Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

mkmapdata.sh error for 3x3min_MODIS-wCsp

Status
Not open for further replies.
J

jjang

Jihyeon Jang
New Member
Hello,

I am running mkmapdata.sh and encountered an error whlie it was creating the map file for 3x3min_MODIS-wCsp. (It successfully ran for other grids.)

I am using the released source code of CESM2.2. And intel 19.0.5 and ESMF 8.0.0 are used as in mkmapdata.sh.

Here are error messages:
-----------------------------------------------------------------------
mpiexec_mpt -np 36 /glade/u/apps/ch/opt/esmf-mpi-netcdf/8.0.0/mpt/2.22/intel/19.0.5//bin/binO/Linux.intel.64.mpt.default/ESMF_RegridWeightGen --ignore_unmapped -s /glade/p/cesm/cseg/inputdata/lnd/clm2/mappingdata/grids/SCRIPgrid_3minx3min_MODISwcspsea_c151020.nc -d /glade/work/jjang/cesm/gen_input/MPAS/mpas_scrip/mp120a_scrip.nc -m conserve -w map_3x3min_MODIS-wCsp_to_mpasa120_nomask_aave_da_c210423.nc --src_type SCRIP --dst_type SCRIP --64bit_offset

WARNING: deprecated switch -src_type will be ignored. The file type will be de
tected automatically

WARNING: deprecated switch -dst_type will be ignored. The file type will be de
tected automatically
Starting weight generation with these inputs:
Source File:
/glade/p/cesm/cseg/inputdata/lnd/clm2/mappingdata/grids/SCRIPgrid_3minx3min_MOD
ISwcspsea_c151020.nc
Destination File:
/glade/work/jjang/cesm/gen_input/MPAS/mpas_scrip/mp120a_scrip.nc
Weight File: map_3x3min_MODIS-wCsp_to_mpasa120_nomask_aave_da_c210423.nc
Source File is in SCRIP format
Source Grid is a global grid
Source Grid is a logically rectangular grid
Use the center coordinates of the source grid to do the regrid
Destination File is in SCRIP format
Destination Grid is a global grid
Destination Grid is an unstructured grid
Use the center coordinates of the destination grid to do the regrid
Regrid Method: conserve
Pole option: NONE
Ignore unmapped destination points
Output weight file in 64bit offset NetCDF file format
Line Type: greatcircle
Norm Type: dstarea
Extrap. Method: none

ERROR: Problem on processor 12
. Please see the PET*.RegridWeightGen.Log files for a traceback.
-----------------------------------------------------------------------

In PET12.RegridWeightGen.Log, the messages are written as:
-----------------------------------------------------------------------
20210423 103058.307 INFO PET12 Running with ESMF Version 7.1.0r
20210423 103059.191 ERROR PET12 ESMF_Grid.F90:5304 ESMF_GridCreate Wrong argument specified - - Bad corner array in SCRIP file
20210423 103059.191 ERROR PET12 ESMF_Grid.F90:6480 ESMF_GridCreateFrmScrip Wrong argument specified - Internal subroutine call returned Error
20210423 103059.205 ERROR PET12 ESMF_Grid.F90:6141 ESMF_GridCreateFrmNCFile Wrong argument specified - Internal subroutine call returned Error
20210423 103059.205 ERROR PET12 ESMF_RegridWeightGen.F90:1146 ESMF_RegridWeightGenFile Wrong argument specified - Internal subroutine call returned Error
20210423 103059.205 INFO PET12 Finalizing ESMF
-----------------------------------------------------------------------

Other 3min data didn't have this problem, but I cannot get a mapping file for this grid.

I used 4 cpus per node with mem=109GB on Cheyenne, and I tried increasing the number of nodes to 12, but it didn't work.

Any information will be greatly helpful. Thank you in advance.

Sincerely,
Jihyeon
 
erik

erik

Erik Kluzek
CSEG and Liaisons
Staff member
Hi Jihyeon

The error about "Bad corner array in SCRIP file" might indicate a problem in your SCRIP grid file: /glade/work/jjang/cesm/gen_input/MPAS/mpas_scrip/mp120a_scrip.nc

There might be something like a gridcell that overlaps another grid cell. It's worth looking carefully over your grid file to see if there is such a problem with it. If you can't find a problem with it, I'd also suggest trying to see if the other maps after this one will be created. Create an empty file for the map with today's date and it will go to the other maps. To see which maps work and which don't might give a clue as to the problem.

If you don't see a problem in your gridfile you should contact the ESMF group and give them the cases that fail. There might also be a newer version of ESMF that you can try, or using a different compiler.
 
J

jjang

Jihyeon Jang
New Member
Hi Erik,

Thank you so much for your answer and suggestion.

- I have modified mkmapdata.sh as follows to see if the other maps are created.
grids=( \
"3x3min_USGS" \
"5x5min_nomask" \
"5x5min_IGBP-GSDP" \
"5x5min_ISRIC-WISE" \
"5x5min_ORNL-Soil" \
"10x10min_nomask" \
"10x10min_IGBPmergeICESatGIS" \
"3x3min_GLOBE-Gardner" \
"3x3min_GLOBE-Gardner-mergeGIS" \
"0.9x1.25_GRDC" \
"360x720cru_cruncep" \
"1km-merge-10min_HYDRO1K-merge-nomask" \
)

Other maps are created successfully. I cannot create the mapping file only for 3x3min_MODIS-wCsp.


- I checked with other scrip files for other dynamical core grids (e.g., ne30, f09, f19 grids). But the mapping file for 3x3min_MODIS-wCsp was not created even for those grids.


- So, just for a test, I used the 3x3min_MODIS-wCsp scrip file for the destination file as follows:
-----------------------------------------------------------------------
Source File:
/glade/p/cesm/cseg/inputdata/lnd/clm2/mappingdata/grids/SCRIPgrid_0.5x0.5_AVHRR
_c110228.nc
Destination File:
/glade/p/cesm/cseg/inputdata/lnd/clm2/mappingdata/grids/SCRIPgrid_3minx3min_MOD
ISwcspsea_c151020.nc
Weight File: map_0.5x0.5_AVHRR_to_test_nomask_aave_da_c210426.nc
...
-----------------------------------------------------------------------
But there was the same error, "Bad corner array in SCRIP file" in PET12.RegridWeightGen.Log.


- Lastly, I used intel/18.0.5 and esmf_libs/7.1.0r, but it didn't work.



I am not sure..., but do you think the scrip file for 3x3min_MODIS-wCsp has a problem?


Thank you so much,
Jihyeon
 
J

jjang

Jihyeon Jang
New Member
Hi Erik,

Thank you for your reply.

I took a look at it more, and I realize that I made a mistake.. I didn't use regridbatch.sh, so I didn't set REGRID_PROC for mkmapdata.sh in my own script.

I can create the mapping file for 3x3min_MODIS-wCsp with 8 cores. When I increased the number of cores more than 8 cores, the mapping file was not created with the "bad corner array" error.

Also, I found REGRID_PROC=8 for "-t global" in regridbatch.sh. Without "time env REGRID_PROC=8", mkmapdata.sh sets REGRID_PROC to 36 as
if [ -z "$REGRID_PROC" ]; then
REGRID_PROC=36
fi


So now I can create the mapping file for 3x3min_MODIS-wCsp with REGRID_PROC=8 for my MPAS grid.

I don't know why the scrip file has the bad corner array error when 8 cores are not used. You probably know this reason...


Thank you for your quick answer and your time to fix this problem.


Best regards,
Jihyeon
 
erik

erik

Erik Kluzek
CSEG and Liaisons
Staff member
Hi Jihyeon

So I just created a new CTSM tag that brings in the fixes so that mkmapdata.sh will work. So I recommend using it.

I did notice some strange behavior when using more than 8 processors. Which is why that's what we use for out of the box. I'm thinking it has to do with the high memory requirements for some of these grids.

But in any case I'm glad you have something working now.
 
Z

zhendf

fanfan zheng
New Member
Thanks，I just meet the same error before！ So what I need to do is to adjust the core=8？
 
L

LiwenWu

LiwenWu
Member
erik said:
Hi Jihyeon

So I just created a new CTSM tag that brings in the fixes so that mkmapdata.sh will work. So I recommend using it.

I did notice some strange behavior when using more than 8 processors. Which is why that's what we use for out of the box. I'm thinking it has to do with the high memory requirements for some of these grids.

But in any case I'm glad you have something working now.
Click to expand...
Dear Erik,

I just met the same error when I run mkmapdata.sh and it didn't work after I followed the command "env REGRID_PROC=8". I am wondering where can I find the new CTSM tag you mentioned above? Is it in the github? But I just can't find it in the link you provided above. Looking forward to your reply, any advise would be greatly appreciated.

Best regards,
WuLiwen
 
Status
Not open for further replies.
Top