Skip to content
Raster stack values...
Clear all

Raster stack values mostly NA but in ArcGIS Pro hardly any NA values in extent

1 Posts
1 Users
Illustrious Member
Joined: 4 months ago
Posts: 57365
Topic starter  

I am predicting a machine learning model onto a set of continuous and categorical spatial explanatory variables in R. When I take a look at the raster stack (called Ras1) values using:


most of the columns have NA values. Since my layers are categorical when I resampled the rasters I used the nearest neighbor method. I can't figure out why all the NA values but it must be something in my code:

rastlist <- list.files(path = 'C:/Users/Desktop/Rasters', pattern='.tif$', all.files=TRUE, full.names=T)

#import all tiff files into raster using lapply
allrasters <- lapply(rastlist, raster)

allrasters <- raster::stack(allrasters)

#read shape file of StudyArea
StudyArea <- terra::vect(x = "C:/Users/Desktop/Rasters/StudyArea.shp", 
layer = "StudyArea")

#match the extent of all rasters to an arbitrary raster in stack

#need to change one raster crs to be the same as another
#allrasters[[2]] <- projectRaster(allrasters[[2]], 
 crs=crs(allrasters[[3]]), method='ngb')

##Creating a function to make a polygon of each raster's extent##

ext<-lapply(allrasters, FUN=fxn)

##Aggregating and dissolving all extents##
full.ext<-aggregate(, ext), dissolve=TRUE)

##Creating a blank raster with the full extent, the desired final projection, and the desired resolution##
blank<-raster(ext=extent(full.ext), nrow=allrasters[[3]]@nrows, ncol=allrasters[[3]]@ncols, crs=allrasters[[3]]@crs)

##Resampling all rasters to desired extent nearest neighbor is used##
rastostack <-lapply(allrasters, FUN = resample, blank, method='ngb')

##Stacking the rasters##
Ras <- raster::stack(rastostack)

#rename layers in raster stack to match the names in the models
name <- c("Age", "Drainage", "HSG", "LU2005", "LU1990", "LU1975", "LU1960", 
"LU1945", "NO3_180", "NO3_400", "DO5", "DO2", "Precip")

#apply those names to the raster
Ras1 <- Ras[[name]]

#check layer name and order

#extracting values from rasters so I can apply model and bias correction
val <- values(Ras1)

#clean up table by rounding and replacing numbers that really mean NA)
val$Drainage <- round(val$Drainage, digits=0)
val$HSG <- round(val$HSG, digits=0)
val$LU1945 <- round(val$LU1945, digits=0)

# replace values of some pixel values which I know to be do NoData
val$LU1945[val$LU1945==128] <- ''
val$LU1945[val$LU1945==0] <- ''

#make categorical variables factors
val$Drainage <- as.factor(val$Drainage)
val$LU1945 <- as.factor(val$LU1945)

#call model
rfcaret_1000 <- readRDS("rfcaret_1000.Rds")

# predict model to values from raster
map<- predict(rfcaret_1000, val, type="raw") 

But when I look at the table val it almost all the cell values are NA so something must be wrong in the process of making the stack or resampling? This post shows a similar problem but I have checked CRS and all layers seem to match.


Unreplied Posts

How to install Ubuntu server 20.04 with UEFI (non gui)

I have done this before but the current ISO installer does not match my notes from previous installs.

ISO: ubuntu-20.04.5-live-server-amd64 using non gui installer via idrac.

get to disk partitioning
select disk
enter image description here

the only option is to add a GPT partition

enter image description here

and no option to format as ESP.

How should I create a /boot partition for UEFI?

All the stuff I turned up in searching used the GUI install where the dialogue was different.

Elements of the coset G/H, where G=$GL^+$(2) and H=SO(2)


In this paper1 in section 2, a method to write the elements of the coset of G/H is provided for GL(4), but I am interested in $GL^+$(2).

My matrix representation of $mathfrak{gl}(2)$ is

a+x& -b+y\
b+y & a-x

My matrix representation of $mathfrak{so}(2)$ is

0& -b\
b & 0

Reading the paper, it states that the element g of G can be decomposed as g=$gamma$h, where

gamma = exp left( begin{bmatrix}
a+x& +y\
y & a-x
end{bmatrix} right) in G/H

and where

h = exp left( begin{bmatrix}
0& -b\
b & 0
end{bmatrix} right) in H

Is this correct, or no?

I am suspicious of the argument, because to me

exp left( begin{bmatrix}
a+x& y\
y & a-x
end{bmatrix} right)exp left( begin{bmatrix}
0& -b\
b & 0
end{bmatrix} right) neq exp left( begin{bmatrix}
a+x& -b+y\
b+y & a-x
end{bmatrix} right)

Thus, $gamma h$ does not appear to realize all elements of $G$. Or do we not care about some missing elements for cosets?


Project not working after moving to WSL directory

My dockerized Lumen project had slow response time so according to advices I moved my project from my classic path C:Mgrlumen_laravellumenmysql to wsl.localhostUbuntuhomesamolumenmysql, then created Z: drive so I can open terminal in that location and run “docker-compose up -d”.
In previous adress it was working.I didnt change any code and have got error. Any advices? Thank you.

error in docker

2023-03-21 22:49:00 AH00112: Warning: DocumentRoot [/var/www/html/public] does not exist

According to docker inspect Mount info it exists

/VAR/WWW/HTML Z:homesamolumenmysql


version: '3.8'


    image: mysql:5.7
    container_name : mysqldb
    restart: always
      - ${DOCKER_MYSQL_VOLUME:-/opt/db_data}:/var/lib/mysql
      - "3306:3306"
     MYSQL_DATABASE: diplomovka

     image: phpmyadmin/phpmyadmin
     container_name: phpmyadmin
       PMA_HOST: mysqldb
       DB_READ_HOST: mysqldb
       - "3400:80"
       - mysqldb

     container_name: Lumen
       context: .
       dockerfile: Dockerfile
       - ./:/var/www/html
     restart: ${DOCKER_RESTART_POLICY:-always}
       - "9000:80"
     working_dir: /var/www/html
       MYSQL_HOST: mysqldb
       MYSQL_USER: root
       MYSQL_PASSWORD: 1234
       MYSQL_PORT: 3306


<VirtualHost *:80>
    ServerName localhost

    DocumentRoot /var/www/html/public

    <Directory "/var/www/html">
        AllowOverride all
        Require all granted

    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined


FROM php:7.4.19-apache

WORKDIR /var/www/html

RUN apt-get update && apt-get install -y  

    && docker-php-ext-install pdo_mysql 
    && docker-php-ext-install mysqli 
    && docker-php-source delete

COPY ./vhost.conf /etc/apache2/sites-available/000-default.conf

COPY ./ ./

RUN curl -sS | php -- --install-dir=/usr/local/bin --filename=composer

RUN chown -R www-data:www-data /var/www/html 
    && a2enmod rewrite

RUN cd /var/www/html && composer install && php artisan key:generate

Using Root Test to see whether $sum_{n=1}^{infty}frac{n^{n+frac{1}{n}}}{(n+frac{1}{n})^{n}}$ converges


This exercise specifically requires that we use the root test to determine whether the series converges or not.

All I’ve done so far is get the sequence in this form:
$$sqrt[n] frac{n^{n+frac{1}{n}}}{(n+frac{1}{n})^{n}} = sqrt frac{n^{(1+n^{-2})n}}{(n+frac{1}{n})^n}=frac{n^{1+n^{-2}}}{n+frac{1}{n}} = frac{n^{frac{n^2+1}{n^{2}}}}{n^{2}+1}$$

But, I’m not even sure if I’m on the right track here. Any guidance is appreciated.

Edit: I showed my effort. I didn’t tell anyone to solve it for me. Not sure why the downvotes..


curious GoldenRatio identity


I would like to verify the following identity but I don’t know how mathematics says that it is equal to numerically.
$$prod _{k=0}^{infty } sqrt{frac{phi ^{2^{-k-1}} left(phi ^{2^{-k}}+1right)}{phi ^{2^{1-k}}+1}}=frac{sqrt{2}}{sqrt[4]{5}}$$


induction on summation with factorial


Let $P(k, n) : exists b_{1}, b_{2}, …, b_{n} in mathbb{N}, (forall i in mathbb{Z^+}, 1 leq i leq k implies b_{i} leq i) land(k = sum limits _{i=1}^n b_{i} cdot i!)$ where $k in mathbb{N}$ and $n in mathbb{Z^+}$.

How to show with induction $forall k in mathbb{N}, forall n in mathbb{N^+}, k < (n+1)! implies P(k,n)$

Any suggestions and hints would be appreciated.


How to repair an XFS volume in a RAID array

In a data server, we have a RAID array of 5 disks joined in an XFS filesystem. The dmesg output is:

IPv6: ADDRCONF(NETDEV_CHANGE): enp134s0f0: link becomes ready
XFS (sda2): Mounting V5 Filesystem
XFS (sda2): failed to locate log tail
XFS (sda2): log mount/recovery failed: error -74
XFS (sda2): log mount failed
clocksource: timekeeping watchdog on CPU21: hpet retried 2 times before success
clocksource: timekeeping watchdog on CPU22: hpet retried 2 times before success
clocksource: timekeeping watchdog on CPU5: hpet retried 2 times before success
clocksource: timekeeping watchdog on CPU23: hpet retried 2 times before success
clocksource: timekeeping watchdog on CPU4: hpet retried 3 times before success
clocksource: timekeeping watchdog on CPU19: hpet retried 2 times before success
clocksource: timekeeping watchdog on CPU10: hpet retried 2 times before success
clocksource: timekeeping watchdog on CPU6: hpet retried 2 times before success
clocksource: timekeeping watchdog on CPU0: hpet retried 2 times before success
usb 1-2.5: USB disconnect, device number 5

and the output of xfs_repair /dev/sda2 is:

bad hash table for directory inode 66571995395 (no data entry): rebuilding
rebuilding directory inode 66571995395
entry ".." in directory inode 66571995396 points to non-existent inode 60129542274
bad hash table for directory inode 66571995396 (no data entry): rebuilding
rebuilding directory inode 66571995396
entry ".." in directory inode 66571995397 points to non-existent inode 4294967426
bad hash table for directory inode 66571995397 (no data entry): rebuilding
rebuilding directory inode 66571995397
Metadata corruption detected at 0x5566785ce66f, inode 0xf80000910 dinode

fatal error -- couldn't map inode 66571995408, err = 117

I will appreciate any help or hint.

Trying to Understand How Multiple Percent Symbols are Processed Around Variables

@echo off

set var1=hello
set var2=var1
set var3=var2
echo on

echo 2. %%var3%%
echo 3. %%%var3%%%

I have echo on so I can try to make better sense of the batch processing, though it really seems to be limited in parsing info.

When echoing a variable, are the inner/center parts processed first?

I’m trying to understand how multiple percents signs are exactly parsed when surrounding a variable.

I know every other percent symbol cancels out the previous one, but there must be more to it because I can never seem to get the result I think I should.

The logic seems to hold true when I do a simple echo using double percents %%, like so:

echo %%var3%%

Result: %var3%

The outer percent symbol is canceled, and it outputs with single percents: %var3%

So, when I use 3 %%%, I would think it would return %%var3%%, since ONLY the middle percent of the 3 %%% should be canceled, yet the result is: %var2%

So, obviously I’m not fully understanding how multiple percents surrounding a variable are processed.

When using 2 %%, does the 2nd % cancel the entire process of reading the value from var3?
Because how else could the result be %var3% rather than %var2%?

And b/c the extra % when using 3 %%%, it re-allows the process of reading the value from var3, which can now extract the value from var3, which is var2, and then simply apply the remaining single %s around it to give the final result: %var2%

Am I close or still grasping at straws? Any clarification or suggestions would be appreciated.

Strassmann’s thoerem and irrationality measure of certain number


In this note from Keith Conrad, he explains an interesting application of Strassmann’s theorem to the divergence of certain linear recurrence integer sequence. More precisely, the sequence defined as $a_0 = a_1 = 1$ and $a_{m} = 2a_{m-1} – 3a_{m-2}$ satisfies $lim_{mto infty} |a_m| = infty$. It seems somewhat easy to prove at first glance (and the actual behavior of $|a_m|$ is exponential), one needs to deal with possible cancellation. The general term is given by
a_m = frac{1}{2} ((1 + sqrt{-2})^{m} + (1 – sqrt{-2})^{m}) = sqrt{3}^m cos (m alpha)

where $alpha = arctan(sqrt{2})$, and if $malpha$ is sufficiently close to the odd multiple of $pi / 2$, then $cos(malpha)$ could be very close to zero. So one may need to show that $cos(malpha)$ can’t be exponentially small with respect to $m$ in some sense, or use $p$-adic method as in Conrad’s note.

What I thought is that once we know $lim_{mtoinfty}|a_m| = infty$, this would tell us that $alpha / pi$ can’t be very close to rational numbers, and may tell something about irrationality measure of $alpha / pi$. I tried some but didn’t get anything useful at this moment. Is it possible to deduce some information on the irrationality measure of $alpha /pi$ using divergence, at least finiteness or infiniteness?


Sharing a webpage with a filled out form

Is there a way to share a webpage with a filled out form to someone else? Meaning, let’s say the page has a HTML form on it, and I fill it out. Can I share it so that when someone else clicks the link I sent them, they will see the form filled out.

Maybe a browser extension or a service that does so? Is there perhaps a way through the url (like text highlighting)?

Would such a thing even be possible?