Skip to content
Raster stack values...
 
Notifications
Clear all

Raster stack values mostly NA but in ArcGIS Pro hardly any NA values in extent

1 Posts
1 Users
0 Likes
1 Views
Guest
Illustrious Member
Joined: 4 months ago
Posts: 57365
Topic starter  

I am predicting a machine learning model onto a set of continuous and categorical spatial explanatory variables in R. When I take a look at the raster stack (called Ras1) values using:

values(Ras1) 

most of the columns have NA values. Since my layers are categorical when I resampled the rasters I used the nearest neighbor method. I can't figure out why all the NA values but it must be something in my code:

rastlist <- list.files(path = 'C:/Users/Desktop/Rasters', pattern='.tif$', all.files=TRUE, full.names=T)

#import all tiff files into raster using lapply
allrasters <- lapply(rastlist, raster)

allrasters <- raster::stack(allrasters)

#read shape file of StudyArea
StudyArea <- terra::vect(x = "C:/Users/Desktop/Rasters/StudyArea.shp", 
layer = "StudyArea")

#match the extent of all rasters to an arbitrary raster in stack
extent(allrasters[[1]])

#need to change one raster crs to be the same as another
#allrasters[[2]] <- projectRaster(allrasters[[2]], 
 crs=crs(allrasters[[3]]), method='ngb')

##Creating a function to make a polygon of each raster's extent##
fxn<-function(ras){
  bb<-bbox(ras)
  bbpoly<-bb_poly(bb)
  st_crs(bbpoly)<-crs(ras)
  return(as_Spatial(bbpoly))
}

ext<-lapply(allrasters, FUN=fxn)

##Aggregating and dissolving all extents##
full.ext<-aggregate(do.call(bind, ext), dissolve=TRUE)

##Creating a blank raster with the full extent, the desired final projection, and the desired resolution##
blank<-raster(ext=extent(full.ext), nrow=allrasters[[3]]@nrows, ncol=allrasters[[3]]@ncols, crs=allrasters[[3]]@crs)

##Resampling all rasters to desired extent nearest neighbor is used##
rastostack <-lapply(allrasters, FUN = resample, blank, method='ngb')

##Stacking the rasters##
Ras <- raster::stack(rastostack)
Ras 

#rename layers in raster stack to match the names in the models
name <- c("Age", "Drainage", "HSG", "LU2005", "LU1990", "LU1975", "LU1960", 
"LU1945", "NO3_180", "NO3_400", "DO5", "DO2", "Precip")

#apply those names to the raster
Ras1 <- Ras[[name]]

#check layer name and order
Ras1@layers

#extracting values from rasters so I can apply model and bias correction
val <- values(Ras1)
val<- as.data.table(val)

#clean up table by rounding and replacing numbers that really mean NA)
val$Drainage <- round(val$Drainage, digits=0)
val$HSG <- round(val$HSG, digits=0)
val$LU1945 <- round(val$LU1945, digits=0)

# replace values of some pixel values which I know to be do NoData
val$LU1945[val$LU1945==128] <- ''
val$LU1945[val$LU1945==0] <- ''

#make categorical variables factors
val$Drainage <- as.factor(val$Drainage)
val$LU1945 <- as.factor(val$LU1945)

#call model
rfcaret_1000 <- readRDS("rfcaret_1000.Rds")

# predict model to values from raster
map<- predict(rfcaret_1000, val, type="raw") 

But when I look at the table val it almost all the cell values are NA so something must be wrong in the process of making the stack or resampling? This post shows a similar problem but I have checked CRS and all layers seem to match.


   
Quote

Unreplied Posts

How to install Ubuntu server 20.04 with UEFI (non gui)

I have done this before but the current ISO installer does not match my notes from previous installs.

ISO: ubuntu-20.04.5-live-server-amd64 using non gui installer via idrac.

get to disk partitioning
select disk
enter image description here

the only option is to add a GPT partition

enter image description here

and no option to format as ESP.

How should I create a /boot partition for UEFI?

All the stuff I turned up in searching used the GUI install where the dialogue was different.

Elements of the coset G/H, where G=$GL^+$(2) and H=SO(2)

$begingroup$

In this paper1 in section 2, a method to write the elements of the coset of G/H is provided for GL(4), but I am interested in $GL^+$(2).

My matrix representation of $mathfrak{gl}(2)$ is

$$
begin{bmatrix}
a+x& -b+y\
b+y & a-x
end{bmatrix}
$$

My matrix representation of $mathfrak{so}(2)$ is

$$
begin{bmatrix}
0& -b\
b & 0
end{bmatrix}
$$


Reading the paper, it states that the element g of G can be decomposed as g=$gamma$h, where

$$
gamma = exp left( begin{bmatrix}
a+x& +y\
y & a-x
end{bmatrix} right) in G/H
$$

and where

$$
h = exp left( begin{bmatrix}
0& -b\
b & 0
end{bmatrix} right) in H
$$

Is this correct, or no?

I am suspicious of the argument, because to me

$$
exp left( begin{bmatrix}
a+x& y\
y & a-x
end{bmatrix} right)exp left( begin{bmatrix}
0& -b\
b & 0
end{bmatrix} right) neq exp left( begin{bmatrix}
a+x& -b+y\
b+y & a-x
end{bmatrix} right)
$$

Thus, $gamma h$ does not appear to realize all elements of $G$. Or do we not care about some missing elements for cosets?

$endgroup$

Project not working after moving to WSL directory

My dockerized Lumen project had slow response time so according to advices I moved my project from my classic path C:Mgrlumen_laravellumenmysql to wsl.localhostUbuntuhomesamolumenmysql, then created Z: drive so I can open terminal in that location and run “docker-compose up -d”.
In previous adress it was working.I didnt change any code and have got error. Any advices? Thank you.

error in docker

2023-03-21 22:49:00 AH00112: Warning: DocumentRoot [/var/www/html/public] does not exist

According to docker inspect Mount info it exists

/VAR/WWW/HTML Z:homesamolumenmysql

docker-compose.yml

version: '3.8'

services:


   mysqldb:
    image: mysql:5.7
    container_name : mysqldb
    restart: always
    volumes:
      - ${DOCKER_MYSQL_VOLUME:-/opt/db_data}:/var/lib/mysql
    ports:
      - "3306:3306"
    environment:
     MYSQL_DATABASE: diplomovka
     MYSQL_ROOT_PASSWORD: 1234

   phpmyadmin:
     image: phpmyadmin/phpmyadmin
     container_name: phpmyadmin
     environment:
       PMA_HOST: mysqldb
       DB_READ_HOST: mysqldb
     ports:
       - "3400:80"
     depends_on:
       - mysqldb

   app:
     container_name: Lumen
     build:
       context: .
       dockerfile: Dockerfile
     volumes:
       - ./:/var/www/html
     restart: ${DOCKER_RESTART_POLICY:-always}
     ports:
       - "9000:80"
     working_dir: /var/www/html
     environment:
       MYSQL_HOST: mysqldb
       MYSQL_USER: root
       MYSQL_PASSWORD: 1234
       MYSQL_PORT: 3306

vhost.conf

<VirtualHost *:80>
    ServerName localhost

    DocumentRoot /var/www/html/public

    <Directory "/var/www/html">
        AllowOverride all
        Require all granted
    </Directory>

    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

Dockerfile

FROM php:7.4.19-apache

WORKDIR /var/www/html

RUN apt-get update && apt-get install -y  
        zlib1g-dev 
        libzip-dev 
        libonig-dev 
        curl 

    && docker-php-ext-install pdo_mysql 
    && docker-php-ext-install mysqli 
    && docker-php-source delete

COPY ./vhost.conf /etc/apache2/sites-available/000-default.conf

COPY ./ ./

RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer

RUN chown -R www-data:www-data /var/www/html 
    && a2enmod rewrite

RUN cd /var/www/html && composer install && php artisan key:generate

Using Root Test to see whether $sum_{n=1}^{infty}frac{n^{n+frac{1}{n}}}{(n+frac{1}{n})^{n}}$ converges

$begingroup$

This exercise specifically requires that we use the root test to determine whether the series converges or not.

All I’ve done so far is get the sequence in this form:
$$sqrt[n] frac{n^{n+frac{1}{n}}}{(n+frac{1}{n})^{n}} = sqrt frac{n^{(1+n^{-2})n}}{(n+frac{1}{n})^n}=frac{n^{1+n^{-2}}}{n+frac{1}{n}} = frac{n^{frac{n^2+1}{n^{2}}}}{n^{2}+1}$$

But, I’m not even sure if I’m on the right track here. Any guidance is appreciated.

Edit: I showed my effort. I didn’t tell anyone to solve it for me. Not sure why the downvotes..

$endgroup$

curious GoldenRatio identity

$begingroup$

I would like to verify the following identity but I don’t know how mathematics says that it is equal to numerically.
$$prod _{k=0}^{infty } sqrt{frac{phi ^{2^{-k-1}} left(phi ^{2^{-k}}+1right)}{phi ^{2^{1-k}}+1}}=frac{sqrt{2}}{sqrt[4]{5}}$$

$endgroup$

induction on summation with factorial

$begingroup$

Let $P(k, n) : exists b_{1}, b_{2}, …, b_{n} in mathbb{N}, (forall i in mathbb{Z^+}, 1 leq i leq k implies b_{i} leq i) land(k = sum limits _{i=1}^n b_{i} cdot i!)$ where $k in mathbb{N}$ and $n in mathbb{Z^+}$.

How to show with induction $forall k in mathbb{N}, forall n in mathbb{N^+}, k < (n+1)! implies P(k,n)$

Any suggestions and hints would be appreciated.

$endgroup$

How to repair an XFS volume in a RAID array

In a data server, we have a RAID array of 5 disks joined in an XFS filesystem. The dmesg output is:

IPv6: ADDRCONF(NETDEV_CHANGE): enp134s0f0: link becomes ready
XFS (sda2): Mounting V5 Filesystem
XFS (sda2): failed to locate log tail
XFS (sda2): log mount/recovery failed: error -74
XFS (sda2): log mount failed
clocksource: timekeeping watchdog on CPU21: hpet retried 2 times before success
clocksource: timekeeping watchdog on CPU22: hpet retried 2 times before success
clocksource: timekeeping watchdog on CPU5: hpet retried 2 times before success
clocksource: timekeeping watchdog on CPU23: hpet retried 2 times before success
clocksource: timekeeping watchdog on CPU4: hpet retried 3 times before success
clocksource: timekeeping watchdog on CPU19: hpet retried 2 times before success
clocksource: timekeeping watchdog on CPU10: hpet retried 2 times before success
clocksource: timekeeping watchdog on CPU6: hpet retried 2 times before success
clocksource: timekeeping watchdog on CPU0: hpet retried 2 times before success
usb 1-2.5: USB disconnect, device number 5

and the output of xfs_repair /dev/sda2 is:

bad hash table for directory inode 66571995395 (no data entry): rebuilding
rebuilding directory inode 66571995395
entry ".." in directory inode 66571995396 points to non-existent inode 60129542274
bad hash table for directory inode 66571995396 (no data entry): rebuilding
rebuilding directory inode 66571995396
entry ".." in directory inode 66571995397 points to non-existent inode 4294967426
bad hash table for directory inode 66571995397 (no data entry): rebuilding
rebuilding directory inode 66571995397
Metadata corruption detected at 0x5566785ce66f, inode 0xf80000910 dinode

fatal error -- couldn't map inode 66571995408, err = 117

I will appreciate any help or hint.

Trying to Understand How Multiple Percent Symbols are Processed Around Variables

@echo off

set var1=hello
set var2=var1
set var3=var2
echo on

echo 2. %%var3%%
echo.
echo 3. %%%var3%%%

I have echo on so I can try to make better sense of the batch processing, though it really seems to be limited in parsing info.

When echoing a variable, are the inner/center parts processed first?

I’m trying to understand how multiple percents signs are exactly parsed when surrounding a variable.

I know every other percent symbol cancels out the previous one, but there must be more to it because I can never seem to get the result I think I should.

The logic seems to hold true when I do a simple echo using double percents %%, like so:

echo %%var3%%

Result: %var3%

The outer percent symbol is canceled, and it outputs with single percents: %var3%

So, when I use 3 %%%, I would think it would return %%var3%%, since ONLY the middle percent of the 3 %%% should be canceled, yet the result is: %var2%

So, obviously I’m not fully understanding how multiple percents surrounding a variable are processed.

When using 2 %%, does the 2nd % cancel the entire process of reading the value from var3?
Because how else could the result be %var3% rather than %var2%?

And b/c the extra % when using 3 %%%, it re-allows the process of reading the value from var3, which can now extract the value from var3, which is var2, and then simply apply the remaining single %s around it to give the final result: %var2%

Am I close or still grasping at straws? Any clarification or suggestions would be appreciated.

Strassmann’s thoerem and irrationality measure of certain number

$begingroup$

In this note from Keith Conrad, he explains an interesting application of Strassmann’s theorem to the divergence of certain linear recurrence integer sequence. More precisely, the sequence defined as $a_0 = a_1 = 1$ and $a_{m} = 2a_{m-1} – 3a_{m-2}$ satisfies $lim_{mto infty} |a_m| = infty$. It seems somewhat easy to prove at first glance (and the actual behavior of $|a_m|$ is exponential), one needs to deal with possible cancellation. The general term is given by
$$
a_m = frac{1}{2} ((1 + sqrt{-2})^{m} + (1 – sqrt{-2})^{m}) = sqrt{3}^m cos (m alpha)
$$

where $alpha = arctan(sqrt{2})$, and if $malpha$ is sufficiently close to the odd multiple of $pi / 2$, then $cos(malpha)$ could be very close to zero. So one may need to show that $cos(malpha)$ can’t be exponentially small with respect to $m$ in some sense, or use $p$-adic method as in Conrad’s note.

What I thought is that once we know $lim_{mtoinfty}|a_m| = infty$, this would tell us that $alpha / pi$ can’t be very close to rational numbers, and may tell something about irrationality measure of $alpha / pi$. I tried some but didn’t get anything useful at this moment. Is it possible to deduce some information on the irrationality measure of $alpha /pi$ using divergence, at least finiteness or infiniteness?

$endgroup$

Sharing a webpage with a filled out form

Is there a way to share a webpage with a filled out form to someone else? Meaning, let’s say the page has a HTML form on it, and I fill it out. Can I share it so that when someone else clicks the link I sent them, they will see the form filled out.

Maybe a browser extension or a service that does so? Is there perhaps a way through the url (like text highlighting)?

Would such a thing even be possible?

Share: