Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

Guidance on Running with LME datasets with CESM2.1.3 (CLM5)

wvsi3w

wvsi3w

wvsi3w
Member
Dear Sam @slevis
Thank you for your helpful comment. I think I am making some progress since I am seeing new errors:

This is what I did in my system (Narval):
Code: 
cd projects/def-XXX/USER/inputdata/lnd/clm2/mappingdata/
mkdir grids
cd grids/
wget https://ftp.cgd.ucar.edu/pub/slevis/for_wvsi3w/SCRIPgrid_0.5x0.5_AVHRR_c110228.nc

cd ~/my_cesm_sandbox/components/clm/tools/mkmapdata

export CSMDATA=/home/USER/projects/def-XXX/USER/inputdata

module load ncl
module load nco
module load StdEnv/2020 intel/2020.1.217 openmpi/4.0.3

module load esmf/8.2.0
export ESMFBIN_PATH=/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/MPI/intel2020/openmpi4/esmf/8.2.0/bin

then:
./mkmapdata.sh --gridfile /home/USER/my_cesm_sandbox/components/clm/tools/mkmapgrids/SCRIPgrid_1x1_nomask.nc --res 1x1

and it creates this file (map_0.5x0.5_AVHRR_to_1x1_nomask_aave_da_c250210.nc) but there is also an error:

Code: 
./mkmapdata.sh --gridfile /home/USER/my_cesm_sandbox/components/clm/tools/mkmapgrids/SCRIPgrid_1x1_nomask.nc --res 1x1
./mkmapdata.sh
Script to create mapping files required by mksurfdata_map
query command is ./../../bld/queryDefaultNamelist.pl -silent -namelist clmexp  -justvalue -options sim_year=2000 -csmdata /home/USER/projects/def-XXX/USER/inputdata

Using user specified scrip grid file: /home/USER/my_cesm_sandbox/components/clm/tools/mkmapgrids/SCRIPgrid_1x1_nomask.nc
Output grid resolution is 1x1
Hostname = narval1.narval.calcul.quebec
Machine narval1.narval.calcul.quebec NOT recognized
Creating mapping file: map_0.5x0.5_AVHRR_to_1x1_nomask_aave_da_c250210.nc
From input grid: /home/USER/projects/def-XXX/USER/inputdata/lnd/clm2/mappingdata/grids/SCRIPgrid_0.5x0.5_AVHRR_c110228.nc
For output grid: /home/USER/my_cesm_sandbox/components/clm/tools/mkmapgrids/SCRIPgrid_1x1_nomask.nc


 WARNING: deprecated switch -src_type will be ignored.  The file type will be de
 tected automatically

 WARNING: deprecated switch -dst_type will be ignored.  The file type will be de
 tected automatically
 Starting weight generation with these inputs:
   Source File:
 /home/USER/projects/def-XXX/USER/inputdata/lnd/clm2/mappingdata/grids/
 SCRIPgrid_0.5x0.5_AVHRR_c110228.nc
   Destination File:
 /home/USER/my_cesm_sandbox/components/clm/tools/mkmapgrids/SCRIPgrid_1x1_noma
 sk.nc
   Weight File: map_0.5x0.5_AVHRR_to_1x1_nomask_aave_da_c250210.nc
   Source File is in SCRIP format
   Source Grid is a global grid
   Source Grid is a logically rectangular grid
   Use the center coordinates of the source grid to do the regrid
   Destination File is in SCRIP format
   Destination Grid is a global grid
   Destination Grid is a logically rectangular grid
   Use the center coordinates of the destination grid to do the regrid
   Regrid Method: conserve
   Pole option: NONE
   Ignore unmapped destination points
   Line Type: greatcircle
   Norm Type: dstarea
   Extrap. Method: none

 Completed weight generation successfully.

 Copyright (C) 1995-2019 - All Rights Reserved
 University Corporation for Atmospheric Research
 NCAR Command Language Version 6.6.2
 The use of this software is governed by a License Agreement.
 See http://www.ncl.ucar.edu/ for more details.
(0)     map file: map_0.5x0.5_AVHRR_to_1x1_nomask_aave_da_c250210.nc
(0)     n_s = 143788 max(row)=62616 max(col)=250143
(0)     checking for dups, sorting...
(0)     zeroing out any dups...
(0)     No duplicate weights to remove from mapping file
Creating mapping file: map_0.5x0.5_MODIS_to_1x1_nomask_aave_da_c250210.nc
From input grid: /home/USER/projects/def-XXX/USER/inputdata/lnd/clm2/mappingdata/grids/SCRIPgrid_0.5x0.5_MODIS_c110228.nc
For output grid: /home/USER/my_cesm_sandbox/components/clm/tools/mkmapgrids/SCRIPgrid_1x1_nomask.nc

Input grid file does NOT exist: /home/USER/projects/def-XXX/USER/inputdata/lnd/clm2/mappingdata/grids/SCRIPgrid_0.5x0.5_MODIS_c110228.nc

I tried commenting out this file but it shows other errors for other files too. So I dont think my approach is correct and I guess I should ask for other files too (?)

in the line ~250 of mkmapdata.sh script this is written:
Code: 
grids=(                    \
       "0.5x0.5_AVHRR"     \
       "0.25x0.25_MODIS"   \
       "0.5x0.5_MODIS"     \
       "3x3min_LandScan2004" \
       "3x3min_MODISv2"    \
       "3x3min_MODIS-wCsp" \
       "3x3min_USGS"       \
       "5x5min_nomask"     \
       "5x5min_IGBP-GSDP"  \
       "5x5min_ISRIC-WISE" \
       "5x5min_ORNL-Soil" \
       "10x10min_nomask"   \
       "10x10min_IGBPmergeICESatGIS" \
       "3x3min_GLOBE-Gardner" \
       "3x3min_GLOBE-Gardner-mergeGIS" \
       "0.9x1.25_GRDC" \
       "360x720cru_cruncep" \
       "1km-merge-10min_HYDRO1K-merge-nomask" \
      )

right now we have the first one. But it shows the error of "Input grid file does NOT exist" for the rest of this list.
 
slevis

slevis

Moderator
Staff member
@oleson just running this by you first:
Shouldn't users on non-ncar computers be able to download the files directly from the repository rather than having us place them on our anonymous ftp?
 
wvsi3w

wvsi3w

wvsi3w
Member
slevis said:
@oleson just running this by you first:
Shouldn't users on non-ncar computers be able to download the files directly from the repository rather than having us place them on our anonymous ftp?
Click to expand...
Dear Sam @slevis ,
Do you think we can proceed before Keith answers?
Because I believe if I have those missing grid files we can proceed with the required steps and return to this non-ncar issue later with Keith.
Thanks a lot.
 
O

oleson

Keith Oleson
CSEG and Liaisons
Staff member
Interesting, I don't see SCRIPgrid_3minx3min_USGS_c120912.nc in our inputdata directory either. I think it would be ok to use SCRIPgrid_3x3_USGS_c120912.nc, they have the same creation date.
 
wvsi3w

wvsi3w

wvsi3w
Member
oleson said:
Interesting, I don't see SCRIPgrid_3minx3min_USGS_c120912.nc in our inputdata directory either. I think it would be ok to use SCRIPgrid_3x3_USGS_c120912.nc, they have the same creation date.
Click to expand...
Hello Keith,
Sorry I forgot to answer this.

I tried it with 3x3_USGS and edited the mkmapdata.sh file (I changed 3x3min_USGS to 3x3_USGS) and then I realized after running the "./mkmapdata.sh" it makes a file (map_0.5x0.5_AVHRR_to_1x1_nomask_aave_da_c250310.nc) BUT it is showing some errors like these:

Code: 
Using user specified scrip grid file: /home/USER/my_cesm_sandbox/components/clm/tools/mkmapgrids/SCRIPgrid_1x1_nomask.nc
Output grid resolution is 1x1
(queryDefaultNamelist.pl -silent -namelist clmexp -justvalue -options sim_year=2000 -csmdata /home/USER/projects/def-xxx/USER/inputdata -var scripgriddata -res 3x3 -options lmask=USGS,glc_nec=10 -onlyfiles) ERROR:: invalid resolution entered.
(queryDefaultNamelist.pl -silent -namelist clmexp -justvalue -options sim_year=2000 -csmdata /home/USER/projects/def-xxx/USER/inputdata -var scripgriddata_lrgfile_needed -res 3x3 -options lmask=USGS,glc_nec=10) ERROR:: invalid resolution entered.
(queryDefaultNamelist.pl -silent -namelist clmexp -justvalue -options sim_year=2000 -csmdata /home/USER/projects/def-xxx/USER/inputdata -var scripgriddata_type -res 3x3 -options lmask=USGS,glc_nec=10) ERROR:: invalid resolution entered.

and this error too:
Code: 
[narval2:3461648:0:3461648] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x7fffcaf88bf8)
==== backtrace (tid:3461648) ====
 0 0x0000000000020753 ucs_debug_print_backtrace()  /tmp/ebuser/avx2/UCX/1.8.0/GCCcore-9.3.0/ucx-1.8.0/src/ucs/debug/debug.c:653
 1 0x00000000000130f0 __funlockfile()  :0
 2 0x00000000000eeb66 netcdf_mp_nf90_get_var_1d_fourbyteint_.A()  netcdf4.f90:0
 3 0x0000000001dd02ed esmf_ioscripmod_mp_esmf_scripgetvar_.A()  ESMF_IOScrip.F90:0
 4 0x0000000001c1ef3a esmf_gridmod_mp_esmf_gridcreatefrmscrip_()  ???:0
 5 0x0000000001c2c02f esmf_gridmod_mp_esmf_gridcreatefrmncfile_()  ???:0
 6 0x0000000001fd7c8a esmf_regridweightgenmod_mp_esmf_regridweightgenfile_.A()  ESMF_RegridWeightGen.F90:0
 7 0x000000000041382d MAIN__()  ???:0
 8 0x000000000040e9d2 main()  ???:0
 9 0x0000000000023e1b __libc_start_main()  /cvmfs/soft.computecanada.ca/gentoo/2020/usr/src/debug/sys-libs/glibc-2.30-r8/glibc-2.30/csu/../csu/libc-start.c:308
10 0x000000000040e8ea _start()  ???:0
=================================
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image              PC                Routine            Line        Source
ESMF_RegridWeight  000000000043755A  for__signal_handl     Unknown  Unknown
libpthread-2.30.s  00007F0CC92020F0  Unknown               Unknown  Unknown
libnetcdff.so.7.0  00007F0CC9F02B66  Unknown               Unknown  Unknown
libesmf.so         00007F0CCC83D2ED  Unknown               Unknown  Unknown
libesmf.so         00007F0CCC68BF3A  esmf_gridmod_mp_e     Unknown  Unknown
libesmf.so         00007F0CCC69902F  esmf_gridmod_mp_e     Unknown  Unknown
libesmf.so         00007F0CCCA44C8A  Unknown               Unknown  Unknown
ESMF_RegridWeight  000000000041382D  Unknown               Unknown  Unknown
ESMF_RegridWeight  000000000040E9D2  Unknown               Unknown  Unknown
libc-2.30.so       00007F0CC9058E1B  __libc_start_main     Unknown  Unknown
ESMF_RegridWeight  000000000040E8EA  Unknown               Unknown  Unknown
Error status returned from mkmapdata script


Then I thought maybe I shouldn't have changed the mkmapdata.sh file content. So I tried it once again with the previous name of that file (3x3min_USGS) and this time only the segmentation error showed up.


Code: 
[narval2:3617285:0:3617285] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x7ffe879a0578)
==== backtrace (tid:3617285) ====
 0 0x0000000000020753 ucs_debug_print_backtrace()  /tmp/ebuser/avx2/UCX/1.8.0/GCCcore-9.3.0/ucx-1.8.0/src/ucs/debug/debug.c:653
 1 0x00000000000130f0 __funlockfile()  :0
 2 0x00000000000eeb66 netcdf_mp_nf90_get_var_1d_fourbyteint_.A()  netcdf4.f90:0
 3 0x0000000001dd02ed esmf_ioscripmod_mp_esmf_scripgetvar_.A()  ESMF_IOScrip.F90:0
 4 0x0000000001c1ef3a esmf_gridmod_mp_esmf_gridcreatefrmscrip_()  ???:0
 5 0x0000000001c2c02f esmf_gridmod_mp_esmf_gridcreatefrmncfile_()  ???:0
 6 0x0000000001fd7c8a esmf_regridweightgenmod_mp_esmf_regridweightgenfile_.A()  ESMF_RegridWeightGen.F90:0
 7 0x000000000041382d MAIN__()  ???:0
 8 0x000000000040e9d2 main()  ???:0
 9 0x0000000000023e1b __libc_start_main()  /cvmfs/soft.computecanada.ca/gentoo/2020/usr/src/debug/sys-libs/glibc-2.30-r8/glibc-2.30/csu/../csu/libc-start.c:308
10 0x000000000040e8ea _start()  ???:0
=================================
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image              PC                Routine            Line        Source
ESMF_RegridWeight  000000000043755A  for__signal_handl     Unknown  Unknown
libpthread-2.30.s  00007F2FD2CB30F0  Unknown               Unknown  Unknown
libnetcdff.so.7.0  00007F2FD39B3B66  Unknown               Unknown  Unknown
libesmf.so         00007F2FD62EE2ED  Unknown               Unknown  Unknown
libesmf.so         00007F2FD613CF3A  esmf_gridmod_mp_e     Unknown  Unknown
libesmf.so         00007F2FD614A02F  esmf_gridmod_mp_e     Unknown  Unknown
libesmf.so         00007F2FD64F5C8A  Unknown               Unknown  Unknown
ESMF_RegridWeight  000000000041382D  Unknown               Unknown  Unknown
ESMF_RegridWeight  000000000040E9D2  Unknown               Unknown  Unknown
libc-2.30.so       00007F2FD2B09E1B  __libc_start_main     Unknown  Unknown
ESMF_RegridWeight  000000000040E8EA  Unknown               Unknown  Unknown
Error status returned from mkmapdata script

Is this related to the modules I loaded? do you know how to solve this issue?
 
O

oleson

Keith Oleson
CSEG and Liaisons
Staff member
I went through this process on our machine successfully. The log file indicated that it was looking for SCRIPgrid_3x3_USGS_c120912.nc, not SCRIPgrid_3minx3min_USGS_c120912.nc. I did not have to change mkmapdata.sh. I'm using release-cesm2.1.4. What version are you using?
 
wvsi3w

wvsi3w

wvsi3w
Member
oleson said:
I went through this process on our machine successfully. The log file indicated that it was looking for SCRIPgrid_3x3_USGS_c120912.nc, not SCRIPgrid_3minx3min_USGS_c120912.nc. I did not have to change mkmapdata.sh. I'm using release-cesm2.1.4. What version are you using?
Click to expand...
This is odd. I am using 2.1.3.
I have attached the details of what shows after I run ./mkmapdata.sh

also the steps I took. The only explanation for this error could be these:
either I am using wrong modules, or I am using wrong res because 1x1 in most cases should be 0.9x1.25 but I literally used 1x1.
I don't know if my guess of the reasons for this error is legit.

Is there any other way to go through these steps without making the files? I mean if we could modify pre-existing files of domain and etc for our new simulation. my simulation would be using LME (6hourly dataset 1deg). and another simulation (same time length with monthly temporal res). so do you think we can make some changes or tweaks to the already existing files for these cases without going through all these difficulties?
 

Attachments

  • Error status returned from mkmapdata script.txt
    6 KB · Views: 1
  • steps.txt
    482 bytes · Views: 1
O

oleson

Keith Oleson
CSEG and Liaisons
Staff member
You might have to run in batch mode, using regridbatch.sh, instead of submitting mkmapdata.sh on the command line. It takes a lot of memory.
It's also possible you might have to update your version of esmf, I used 8.6.0.

Is it absolutely essential to run at 1deg X 1deg? Can you run at 0.9x1.25? You indicated you are using 1deg atmospheric forcing data, but the datm should be able to interpolate that to the 0.9x1.25 grid at run time.
 
You must log in or register to reply here.
Top