case setup error in ctsm5.3

[xgao304]

Dear Sir/Madam,

I am trying to run ctsm5.3. I am able to successfully create a case with the following command, but I got an error with case.setup.

create_newcase --case $CASE_DIR --compset 2000_DATM%GSWP3v1_CLM50%BGC-CROP_SICE_SOCN_MOSART_SGLC_SWAV \
--res CLM_USRDAT --user-mods-dir $MYDATA_DIR --machine svante --compiler gnu \
--run-unsupported

#case setup
cd $CASE_DIR
./xmlchange NTASKS=-4
./xmlchange RUN_STARTDATE=1983-01-01
./xmlchange STOP_OPTION=nmonths
./xmlchange STOP_N=348
./xmlchange REST_OPTION=nmonths
./xmlchange REST_N=12
./xmlchange DATM_CLMNCEP_YR_START=1983
./xmlchange DATM_CLMNCEP_YR_END=2011
./xmlchange DATM_CLMNCEP_YR_ALIGN=1983
./xmlchange DOUT_S=false
./xmlchange JOB_WALLCLOCK_TIME=2-12:00:00
# set up the forcing data
cp -f $WD/datastreams_HisMRCM/* ./
./case.setup

....
Case created
ERROR: No variable DATM_CLMNCEP_YR_START found in case
ERROR: No variable DATM_CLMNCEP_YR_END found in case
ERROR: No variable DATM_CLMNCEP_YR_ALIGN found in case
ERROR: module command /usr/share/lmod/lmod/libexec/lmod python load gcc/11.3.0 openmpi/4.1.5 netcdf/4_shared_libs hdf5/1.14.1_parallel lapack/3.11.0 esmf/8.4.2 openblas/0.38.28 python/3.9.1 netcdf/4_shared_libs failed with message:
/bin/sh: /usr/share/lmod/lmod/libexec/lmod: No such file or directory
Problem with case setup

I am not sure if the real issue comes from DATM_CLMNCEP_YR_* variables or from module command. Those variables are used in cesm2.1.3 which I can run successfully.

Any information is appreciated.

Xiang

 

[oleson]

The DATM variables are now:
DATM_YR_START
DATM_YR_END
DATM_YR_ALIGN

 

[xgao304]

I am able to compile the ctsm5.3 successfully, but the case does not run with some error messages. Please see the attached log files "atm.log.325716.241203-122425" and "cesm.log.325716.241203-122425". and my running script (test00.sh). Other log files do not contain any content.

simply put, here is the error message:

ERROR: (dshr_mod:dshr_mesh_init) ERROR: model_meshfile UNSET does not exist

Another confusing part is: when I run the script, it will create two case directories. Ideally, there should be just one case directory under "cases", for example:

/xgao/ctsm5.3/cases/BgcCrop2000_BANG_HisD

This directory contains most of the regular stuffs for the case.

Then there is another case directory created one level up the "cases" directory:


/xgao/ctsm5.3/BgcCrop2000_BANG_HisD, where there are two directories "bld" and "run"

[xgao@c036 BgcCrop2000_BANG_HisD]$ ls -al

drwxr-xr-x 13 xgao users 25 Dec 3 12:24 bld
drwxr-xr-x 4 xgao users 19 Dec 3 12:37 run

I am not sure which specification is incorrect for this. Could you help with both error message and directory structure?

Thanks.

Xiang

 

[oleson]

There is only one case directory. Yours appears to be /xgao/ctsm5.3/cases/BgcCrop2000_BANG_HisD
There is only one bld&run directory. Yours appears to be /xgao/ctsm5.3/BgcCrop2000_BANG_HisD
I don't know how your computer or directory structure was setup, probably in the porting process. But in env_build.xml there should be a variable called

"CIME_OUTPUT_ROOT"

This is set to the "Base directory for build and run directories".
And in env_case.xml there is a variable called

"CASEROOT"

This is set to your case directory.
If you want different case and bld&run directories than you have now, then I would post on the Infrastructure Forum. I don't know how those are setup for different machines.

 

[xgao304]

@oleson, thanks for the information. The directory structure issue should be minor.

Could you give us some idea about what could the cause for

ERROR: (dshr_mod:dshr_mesh_init) ERROR: model_meshfile UNSET does not exist

We are able to compile the ctsm. I attached the log file in the previous threads.

Thanks,

Xiang

 

[oleson]

A case with CLM_USRDAT comes with a bunch of xml variables set to UNSET. I see you are using --user-mods-dir to set those. Perhaps you are missing something, maybe one of the mesh files, e.g.,

env_run.xml: <entry id="ATM_DOMAIN_MESH" value="UNSET">
env_run.xml: <entry id="LND_DOMAIN_MESH" value="UNSET">

 

[xgao304]

Are these variables only specific to CTSM5.3? As I can run cesm2.1.3 for the same compset without changing any setting.
My --user-mods-dir contains domain file and surface dataset at 0.1 degree, which were generated based on your instructions a while ago.

the "shell_commands" in the directory contains

./xmlchange LND_DOMAIN_PATH=/net/fs12/d2/xgao/ctsm5.3/cases/sim_setup/Bangladesh/domain_HisMRCM
./xmlchange LND_DOMAIN_FILE=domain.lnd.0.1x0.1_gx1v7_BANGcrews_c240211.nc

I removed ATM_DOMAIN_PATH and ATM_DOMAIN_FILE as it gives me an error message during the case create.

So for ATM_DOMAIN_MESH and LND_DOMAIN_MESH, what files should I specify? Do I need to generate new files at 0.1 degree or I can use either domain or surface dataset?

Thanks,

Xiang

 

[oleson]

Yes, the newer version of the model doesn't use the domain files anymore, it uses mesh files.
LND_DOMAIN_PATH and LND_DOMAIN_FILE still appear in env_run.xml but are not longer used in this new configuration.
ATM_DOMAIN_PATH and ATM_DOMAIN_FILE are no longer in env_run.xml
You need a mesh file for ATM_DOMAIN_MESH and LND_DOMAIN_MESH, you can use the same file for both. It should be at the resolution you are running the land at, so 0.1deg. I think you can generate a mesh file from the domain file you have by following the instructions in section D in this discussion post:

Setting up (high-res sparse) regional-grid CTSM simulations · ESCOMP CTSM · Discussion #1919

Context This started as issue #1773 from which I linked a google doc (now out of date and obsolete) with step-by-step instructions intended initially for a case that @XiulinGao needed to run. To th...

github.com

 

[xgao304]

@oleson,

I am able to generate the mesh files successfully using my domain file at 0.1 degree following the section D.
One part I am a bit confused about the link is the step 4 in section D as follows
-----
NB. This mesh file’s mask = 1 everywhere.
4) Update the mesh file’s mask according to the landmask variable
4a) EITHER DO THIS (shortcut that allows you to skip 4b)
If not changing the land mask, execute only this line:
ncap2 -s 'elementMask(:)=0' lnd_mesh.nc mask_mesh.nc

The mask in mask_mesh.nc represents the inverse of the land mask, so
land points = 0
non-land points = 1
-----

First, my mesh file's mask is not equal to 1 everywhere. I did follow ncap2 command to generate mask_mesh.nc, then the mask in mask_mesh.nc is not consistent with what is described above, since ncap2 command will make everywhere equal to 0.

My run did not go through with the error message (see the attached running script and log files)

"Incompatible surface dataset"

The domain file and surface dataset are created successfully before and I can run cesm2.1.3 the same compset with these datasets without any issue.

I am wondering if I need to following the link to recreate the surface dataset?

Thanks,

Xiang

 

[oleson]

Ok, since you've created the mesh file from the domain file, as you said, it should already have 1s over land and 0s over ocean. So you shouldn't need to do anything further with the mesh file.
Yes, you'll need to recreate the surface dataset you created for use with cesm2.1.3. It will not be compatible with ctsm5.3

 

[xgao304]

@oleson,

Thanks for the quick reply. I have one more question:

I followed your instructions in this link high resolution (0.1 degree) CLM5-BGC-crop regional simulation (reply on May 13, 2023) to create the domain and surface dataset.

I can reasonably assume that the domain file created above and used to create mesh file is safe to use - then I can need to recreate the surface dataset?

Thanks,

Xiang

 

[oleson]

Yes, you'll need to recreate the surface dataset you created for use with cesm2.1.3. It will not be compatible with ctsm5.3

 

[oleson]

You can use tools/mksurfdata_esmf to create the surface dataset. There is a README.md file in that directory.

 

[xgao304]

@oleson,

Thanks a lot. I was doing that now, but we are missing some libraries. So we will build them first and then give it a try.

Xaing