Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

Subsetting Surface Datasets

A

ah828

Anthony Holmes
New Member
Per my previous discussion post:
Regional Ensemble Spin-up Error: "Attempt to use feature that was not turned on when netCDF was built"

I am now attempting to use the ncks command to subset the surface and domain data to my region of interest. Using the following command:

ncks -d LONGXY,21.,52. -d LATIXY,-12.,24. /glade/p/cesmdata/cseg/inputdata/lnd/clm2/surfdata_map/release-clm5.0.18/surfdata_0.9x1.25_hist_78pfts_CMIP6_simyr2000_c190214.nc /glade/scratch/anthholmes/regional/surfdata_0.9x1.25_78pfts_CMIP6_simyr2000_E.Africa_ncks.nc

I receive the error:

Code: 
ncks: ERROR dimension LONGXY is not in input file

Running ncdump -h and looking at the variable keys in Python, it is clear this variable exists, but has two dimensions: lsmlat, and lsmlon. The ./subset_surfdata python tool deals with this by converting the variables to a 1D array as follows:

Code: 
# create 1d variables
    lon0=np.asarray(f1['LONGXY'][0,:])
    lon=xr.DataArray(lon0,name='lon',dims='lsmlon',coords={'lsmlon':lon0})
    lat0=np.asarray(f1['LATIXY'][:,0])
    lat=xr.DataArray(lat0,name='lat',dims='lsmlat',coords={'lsmlat':lat0})

How would one accomplish this using NCO? What other tools are people using to subset surface datasets?
 
slevis

slevis

Moderator
Staff member
I wonder whether the "ERROR dimension LONGXY is not in input file" refers to the fact that you typed "ncks -d" where I suspect "d" means dimension, while LONGXY is a variable with dimensions lsmlat, lsmlon.
 
A

ah828

Anthony Holmes
New Member
slevis said:
I wonder whether the "ERROR dimension LONGXY is not in input file" refers to the fact that you typed "ncks -d" where I suspect "d" means dimension, while LONGXY is a variable with dimensions lsmlat, lsmlon.
Click to expand...
That's a good point, I tried running using the lsmlat and lsmlon dims previously, but receive the following error

Code: 
(xESMF) anthholmes@cheyenne1:~/CTSM/tools/mkmapgrids> ncks -d lsmlon,21.,52. -d lsmlat,-12.,24. /glade/p/cesmdata/cseg/inputdata/lnd/clm2/surfdata_map/release-clm5.0.18/surfdata_0.9x1.25_hist_78pfts_CMIP6_simyr2000_c190214.nc /glade/scratch/anthholmes/regional/surfdata_0.9x1.25_78pfts_CMIP6_simyr2000_E.Africa_ncks.nc
ncks: ERROR nco_lmt_evl_dmn_crd() unable to read user-specified coordinate lsmlat. Ensure this coordinate variable is in file and is a 1-D array.
nco_err_exit(): ERROR Short NCO-generated message (usually name of function that triggered error): nc_get_vara_double()
nco_err_exit(): ERROR Error code is -50. Translation into English with nc_strerror(-50) is "NetCDF: Action prohibited on NC_GLOBAL varid"
nco_err_exit(): ERROR NCO will now exit with system call exit(EXIT_FAILURE)

Even though lsmlon and lsmlat show up as dimensions when I run ncdump -h:

Code: 
(xESMF) anthholmes@cheyenne1:~/CTSM/tools/mkmapgrids> ncdump -h /glade/p/cesmdata/cseg/inputdata/lnd/clm2/surfdata_map/release-clm5.0.18/surfdata_0.9x1.25_hist_78pfts_CMIP6_simyr2000_c190214.nc
netcdf surfdata_0.9x1.25_hist_78pfts_CMIP6_simyr2000_c190214 {
dimensions:
        lsmlon = 288 ;
        lsmlat = 192 ;
        nglcec = 10 ;
        nglcecp1 = 11 ;
        ...
 
slevis

slevis

Moderator
Staff member
...what about trying your ncks command but with "ncks -v" where "v" stands for variable. Does that do anything?
 
A

ah828

Anthony Holmes
New Member
slevis said:
...what about trying your ncks command but with "ncks -v" where "v" stands for variable. Does that do anything?
Click to expand...

It created a NetCDF file with the variable "LATIXY":

Code: 
(xESMF) anthholmes@cheyenne1:~/CTSM/tools/mkmapgrids> ncdump -h /glade/scratch/anthholmes/regional/surfdata_0.9x1.25_78pfts_CMIP6_simyr2000_E.Africa_ncks.nc
netcdf surfdata_0.9x1.25_78pfts_CMIP6_simyr2000_E.Africa_ncks {
dimensions:
        lsmlat = 192 ;
        lsmlon = 288 ;
variables:
        double LATIXY(lsmlat, lsmlon) ;
                LATIXY:long_name = "latitude" ;
                LATIXY:units = "degrees north" ;
 
A

ah828

Anthony Holmes
New Member
slevis said:
Definitely not what you wanted...
I assume that you have already consulted the nco user's guide, but in case you haven't:

NCO Homepage

nco.sourceforge.net nco.sourceforge.net
Click to expand...
Always good to check. I posted this question to NCO Help forum Source Forge as well. My guess is that there is no simple command-line option, but I'll update this thread if something comes up.
 
A

ah828

Anthony Holmes
New Member
ncks does work! The issue was that I was incorrectly specifying the lat and lon. Whereas the Python tool uses decimal degrees, ncks needs you to specify the degrees in latitude values ranging from 0 at the S. Pole to 180 at the N. Pole and for longitude values range from 0 at the Greenwich Meridian to 360.

I used the following code to get the results I wanted:

Code: 
ncks -d lsmlon,21,52 -d lsmlat,78,114 /glade/p/cesmdata/cseg/inputdata/lnd/clm2/surfdata_map/release-clm5.0.18/surfdata_0.9x1.25_hist_78pfts_CMIP6_simyr2000_c190214.nc /glade/scratch/anthholmes/regional/surfdata_0.9x1.25_78pfts_CMIP6_simyr2000_E.Africa_ncks_v2.nc
 
E

evasinha

Eva Sinha
New Member
@aherring and @slevis I experienced the same error mentioned above by Anthony when extracting a surface dataset. I learned from Charlie Zender: "when the lsmlat and lsmlon dimensions do not have corresponding coordinate arrays, then the arguments to -d lsmlat and -d lsmlon need to be indices not actual lats and lons".

The indices can be found using the following command
ncks -C --trd -v LATIXY,LONGXY in.nc | more
Click to expand...
 
You must log in or register to reply here.
Top