./mkmapdata.sh failed in mapping one file

[Ceil]

Hi,

I sincerely need your help. Many thanks in advance!
I can successfully remap all files except one file (i.e., /inputdata//lnd/clm2/mappingdata/grids/SCRIPgrid_3minx3min_MODISwcspsea_c151020.nc). I do not know why only this file is failed. I had the errors as follows,

Starting weight generation with these inputs:
Source File:
/cesm2.1.3/my_cesm_sandbox/components/clm/tools/mkmapda
ta/lnd/clm2/mappingdata/grids/SCRIPgrid_3minx3min_MODISwcspsea_c151020.nc
Destination File:
/cesm2.1.3/my_cesm_sandbox/components/clm/tools/mkmapgr
ids/SCRIPgrid_1x1_mq_nomask_c231127.nc
Weight File: map_3x3min_MODIS-wCsp_to_1x1_mq_nomask_aave_da_c231129.nc
Source File is in SCRIP format
Source Grid is a global grid
Source Grid is a logically rectangular grid
Use the center coordinates of the source grid to do the regrid
Destination File is in SCRIP format
Destination Grid is a regional grid
Destination Grid is a logically rectangular grid
Use the center coordinates of the destination grid to do the regrid
Regrid Method: conserve
Pole option: NONE
Ignore unmapped destination points
Output weight file in 64bit offset NetCDF file format
Line Type: greatcircle
Norm Type: dstarea
Extrap. Method: none

ERROR: Problem on processor 4
. Please see the PET*.RegridWeightGen.Log files for a traceback.
Abort(1) on node 4 (rank 4 in comm 496): application called MPI_Abort(comm=0x84000003, 1) - process 4

===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= RANK 0 PID 155499 RUNNING AT b2389
= KILLED BY SIGNAL: 9 (Killed)
===================================================================================

 

[oleson]

Have you looked in your PET log (as noted by "Please see the PET*.RegridWeightGen.Log files for a traceback.")?
What is the command you are using to run this?

 

[Ceil]

Have you looked in your PET log (as noted by "Please see the PET*.RegridWeightGen.Log files for a traceback.")?
What is the command you are using to run this?

Thanks for your reply! The following is PET00.RegridWeightGen.Log:
20231129 170300.093 INFO PET00 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
20231129 170300.093 INFO PET00 !!! THE ESMF_LOG IS SET TO OUTPUT ALL LOG MESSAGES !!!
20231129 170300.093 INFO PET00 !!! THIS MAY CAUSE SLOWDOWN IN PERFORMANCE !!!
20231129 170300.093 INFO PET00 !!! FOR PRODUCTION RUNS, USE: !!!
20231129 170300.093 INFO PET00 !!! ESMF_LOGKIND_Multi_On_Error !!!
20231129 170300.093 INFO PET00 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
20231129 170300.093 INFO PET00 Running with ESMF Version : 8.0.1
20231129 170300.098 INFO PET00 ESMF library build date/time: "Feb 26 2021" "10:04:02"
20231129 170300.098 INFO PET00 ESMF library build location : /cluster/work/users/vegarde/build/ESMF/8.0.1/intel-2020a/esmf-ESMF_8_0_1/
20231129 170300.098 INFO PET00 ESMF_COMM : intelmpi
20231129 170300.099 INFO PET00 ESMF_NETCDF : enabled
20231129 170300.099 INFO PET00 ESMF_PNETCDF : disabled
20231129 170300.099 INFO PET00 ESMF_PIO : enabled
20231129 170300.099 INFO PET00 ESMF_YAMLCPP : enabled
20231129 170300.099 INFO PET00 ESMF_MOAB : enabled

The remaining PET*.RegridWeightGen.Log files have the similar content. I could not get something useful to help run it.

 

[oleson]

Yep, not much help there.
Again, what is the command you are using to run this?

 

[Ceil]

Yep, not much help there.
Again, what is the command you are using to run this?

Hi Oleson, I think that there are something wrong in SCRIPgrid_3minx3min_MODISwcspsea_c151020.nc (I downloaded it from https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata/lnd/clm2/mappingdata/grids/).
So I tried to write the values of variables in SCRIPgrid_3minx3min_MODISwcspsea_c151020.nc to SCRIPgrid_3minx3min_MODISv2_c190503.nc to check, because they have similar data format. I successfully got outputs except when grid_corner_lat is used. Specifically, if I rewrite SCRIPgrid_3minx3min_MODISwcspsea_c151020.nc as follows, I can successfully get outputs:
vi test.py
import netCDF4
datadir = './'
fnm1 = netCDF4.Dataset(datadir + 'SCRIPgrid_3minx3min_MODISwcspsea_c151020.nc.original',mode = 'r')
grid_center_lat = fnm1.variables['grid_center_lat'][:]
grid_center_lon = fnm1.variables['grid_center_lon'][:]
grid_corner_lat1 = fnm1.variables['grid_corner_lat'][:]
grid_corner_lon = fnm1.variables['grid_corner_lon'][:]
grid_dims = fnm1.variables['grid_dims'][:]
grid_imask = fnm1.variables['grid_imask'][:]
fnm2 = netCDF4.Dataset(datadir + 'SCRIPgrid_3minx3min_MODISv2_c190503.nc.1',mode = 'r+')
#--------------------------------
#write out to netcdf
#-------------------------------
fnm2.variables["grid_center_lat"][:] = grid_center_lat
fnm2.variables["grid_center_lon"][:] = grid_center_lon
#fnm2.variables["grid_corner_lat"][:] = grid_corner_lat # probably wrong in the original SCRIPgrid_3minx3min_MODISwcspsea_c151020.nc
fnm2.variables["grid_corner_lon"][:] = grid_corner_lon
fnm2.variables["grid_dims"][:] = grid_dims
fnm2.variables["grid_imask"][:] = grid_imask
fnm2.close()

mv SCRIPgrid_3minx3min_MODISv2_c190503.nc.1 SCRIPgrid_3minx3min_MODISwcspsea_c151020.nc

I think that I could not use grid_corner_lat in SCRIPgrid_3minx3min_MODISv2_c190503.nc (i.e., grid_corner_lat2) to replace grid_corner_lat in SCRIPgrid_3minx3min_MODISwcspsea_c151020.nc (i.e., grid_corner_lat1), because their values are not identical. For example, the following picture is their difference (diff = grid_corner_lat1[:,0]-grid_corner_lat2[:,0]):


I have no idea how to deal with the situation. Could you check with https://svn-ccsm-inputdata.cgd.ucar.../SCRIPgrid_3minx3min_MODISwcspsea_c151020.nc? If there is really something wrong, could you update the dataset?

The command I am using to run it is: ./mkmapdata.sh -r $GRIDNAME -f $GRIDFILE -t regional

 

[Ceil]

Yep, not much help there.
Again, what is the command you are using to run this?

Hi Oleson, I also have another two questions, and I feel very confused.

1. My research is about agricultural management, so it seems that the compset FHIST_BGC (I need to open land surface and atmosphere modules) is the best option for me. But this compset is Defined (not Scientific / Tested), probably meaning not so robust. FWHIST_BGC is Scientific / Tested, but my focus is troposphere. I am not sure whether FWHIST_BGC is suitable for me. I am also worried that FWHIST_BGC will be much more time-consuming. Could you recommend a better compset to me if possible?

2. when I run ./mksurfdata.pl -r usrspec -usr_gname $GRIDNAME -usr_gdate $CDATE -l $CSMDATA, this command calls the following dataset:
/inputdata/lnd/clm2/rawdata/pftcftdynharv.0.25x0.25.LUH2.histsimyr0850-1849.c171012/mksrf_landuse_histclm50_LUH2_850.c171012.nc
But I want to use /inputdata/lnd/clm2/rawdata/pftcftdynharv.0.25x0.25.LUH2.histsimyr1850-2015.c170629/mksrf_landuse_histclm50_LUH2_2010.c170629.nc
How can I change the code to repalce the dataset?
I tried to change compset, but it always uses the same dataset. I tried to edit mksurfdata.pl, the following code probably is the key, but I still do not know how to change the code to replace the dataset because I know little about perl:
my $typ_cmd = "$scrdir/../../bld/queryDefaultNamelist.pl $mkopts -options hgrid=$hgrid,lmask=$lmask,mergeGIS=$merge_gis$mkcrop -var $filnm";

What's more, there are multiple datasets for certain variables, such as
mksrf_GlacierRegion_10x10min_nomask_c160122.nc
mksrf_GlacierRegion_10x10min_nomask_c170616.nc
It seems that the only difference is the data creation date. The default called is mksrf_GlacierRegion_10x10min_nomask_c170616.nc, but how can I change the code to use mksrf_GlacierRegion_10x10min_nomask_c160122.nc?

 

[slevis]

Hi Oleson, I think that there are something wrong in SCRIPgrid_3minx3min_MODISwcspsea_c151020.nc (I downloaded it from https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata/lnd/clm2/mappingdata/grids/).
So I tried to write the values of variables in SCRIPgrid_3minx3min_MODISwcspsea_c151020.nc to SCRIPgrid_3minx3min_MODISv2_c190503.nc to check, because they have similar data format. I successfully got outputs except when grid_corner_lat is used. Specifically, if I rewrite SCRIPgrid_3minx3min_MODISwcspsea_c151020.nc as follows, I can successfully get outputs:
vi test.py
import netCDF4
datadir = './'
fnm1 = netCDF4.Dataset(datadir + 'SCRIPgrid_3minx3min_MODISwcspsea_c151020.nc.original',mode = 'r')
grid_center_lat = fnm1.variables['grid_center_lat'][:]
grid_center_lon = fnm1.variables['grid_center_lon'][:]
grid_corner_lat1 = fnm1.variables['grid_corner_lat'][:]
grid_corner_lon = fnm1.variables['grid_corner_lon'][:]
grid_dims = fnm1.variables['grid_dims'][:]
grid_imask = fnm1.variables['grid_imask'][:]
fnm2 = netCDF4.Dataset(datadir + 'SCRIPgrid_3minx3min_MODISv2_c190503.nc.1',mode = 'r+')
#--------------------------------
#write out to netcdf
#-------------------------------
fnm2.variables["grid_center_lat"][:] = grid_center_lat
fnm2.variables["grid_center_lon"][:] = grid_center_lon
#fnm2.variables["grid_corner_lat"][:] = grid_corner_lat # probably wrong in the original SCRIPgrid_3minx3min_MODISwcspsea_c151020.nc
fnm2.variables["grid_corner_lon"][:] = grid_corner_lon
fnm2.variables["grid_dims"][:] = grid_dims
fnm2.variables["grid_imask"][:] = grid_imask
fnm2.close()

mv SCRIPgrid_3minx3min_MODISv2_c190503.nc.1 SCRIPgrid_3minx3min_MODISwcspsea_c151020.nc

I think that I could not use grid_corner_lat in SCRIPgrid_3minx3min_MODISv2_c190503.nc (i.e., grid_corner_lat2) to replace grid_corner_lat in SCRIPgrid_3minx3min_MODISwcspsea_c151020.nc (i.e., grid_corner_lat1), because their values are not identical. For example, the following picture is their difference (diff = grid_corner_lat1[:,0]-grid_corner_lat2[:,0]):
View attachment 4548

I have no idea how to deal with the situation. Could you check with https://svn-ccsm-inputdata.cgd.ucar.../SCRIPgrid_3minx3min_MODISwcspsea_c151020.nc? If there is really something wrong, could you update the dataset?

The command I am using to run it is: ./mkmapdata.sh -r $GRIDNAME -f $GRIDFILE -t regional

The fact that you do not get a helpful error message and that you encounter the problem with 3minx3min makes me suspect that your job may require more memory than available on your machine. I do not know how to help with that type of issue.

Regarding your other questions:
1) I strongly agree that you should not use "whole atmosphere" compsets for the type of work that you're doing. I would not worry about the tested/supported labels. Very few compsets are tested or supported. It just means that you may encounter problems that we do not know about. But this could even happen with a tested/supported compset. So I recommend pursuing your research using the most appropriate compset. Regardless, you will have to conduct testing before running final simulations and publishing papers.
2) This tool reads a namelist that you could change. The README.developers may have hints. In mksurfdata.pl, notice this line:
# Run mksurfdata_map with the namelist file
You have the option of running "mksurfdata_map < namelist" outside the perl script, after changing the namelist.

 

[Ceil]

The fact that you do not get a helpful error message and that you encounter the problem with 3minx3min makes me suspect that your job may require more memory than available on your machine. I do not know how to help with that type of issue.

Regarding your other questions:
1) I strongly agree that you should not use "whole atmosphere" compsets for the type of work that you're doing. I would not worry about the tested/supported labels. Very few compsets are tested or supported. It just means that you may encounter problems that we do not know about. But this could even happen with a tested/supported compset. So I recommend pursuing your research using the most appropriate compset. Regardless, you will have to conduct testing before running final simulations and publishing papers.
2) This tool reads a namelist that you could change. The README.developers may have hints. In mksurfdata.pl, notice this line:
# Run mksurfdata_map with the namelist file
You have the option of running "mksurfdata_map < namelist" outside the perl script, after changing the namelist.

Hi Slevis, many thanks for your reply and help!

 

[Ceil]

The fact that you do not get a helpful error message and that you encounter the problem with 3minx3min makes me suspect that your job may require more memory than available on your machine. I do not know how to help with that type of issue.

Regarding your other questions:
1) I strongly agree that you should not use "whole atmosphere" compsets for the type of work that you're doing. I would not worry about the tested/supported labels. Very few compsets are tested or supported. It just means that you may encounter problems that we do not know about. But this could even happen with a tested/supported compset. So I recommend pursuing your research using the most appropriate compset. Regardless, you will have to conduct testing before running final simulations and publishing papers.
2) This tool reads a namelist that you could change. The README.developers may have hints. In mksurfdata.pl, notice this line:
# Run mksurfdata_map with the namelist file
You have the option of running "mksurfdata_map < namelist" outside the perl script, after changing the namelist.

Hi Slevis, I have another question and I have already struggled for it for a long time.
I can run CTSM on a regional scale (such as Europe) after I make regional domain and surface data, but I am not sure whether I can use the similar method to run CESM on a regional scale. Specifically, can I run regional CESM after I make regional domain and surface data (not global)? I know a little about the CESM with variable resolution which has two layers (i.e, one is for global, the other is for the region). But if I do not use the CESM-VR, can I run CESM on a regional model? Sadly speaking, I have tried my best but I could not find a complete example to help me run the CESM on a regional scale.

 

[oleson]

I see you have posted this to the high resolution forum, that is probably the best case to get an answer about a coupled regional simulation. You might reply to that thread just to update and send out another notification.