How select a special region to run the CLM from the global model in CLM?

[jteymoori]

In the above instruction, I reached this step: Add this line to user_nl_clm:
fsurdat = ‘surfdata_united_states_hist_16pfts_Irrig_CMIP6_simyr2000_c231004.nc’
I could not find my answer by using an editor. Can you help me or tell me by which scripts I can do it?

 

[oleson]

We are sorry, but we don't understand your question.
Are you unable to add that line to user_nl_clm using an editor?

 

[jteymoori]

Yes, I want to add. this line: " fsurdat = ‘surfdata_united_states_hist_16pfts_Irrig_CMIP6_simyr2000_c231004.nc’ " to user_nl_clm. but I don't know how to do it

 

[oleson]

The user_nl_clm is just a text file that is located in your case directory. Use an editor (vi or emacs, for example) to open the file and then copy and paste the line above into the file.

 

[jteymoori]

Based on this tutorial by Sam Levis created (Setting up (high-res sparse) regional-grid CTSM simulations · ESCOMP/CTSM · Discussion #1919), I followed step-by-step Instructions for running regional CTSM-FATES at regular 1-degree resolution.
I did these below work:

cd /glade/work/jteymoori
mkdir cases
git clone GitHub - ESCOMP/CTSM: Community Terrestrial Systems Model (includes the Community Land Model of CESM)
cd CTSM
./manage_externals/checkout_externals
cd /CTSM/cime/scripts
module load python

Create case:
./create_newcase --case ~/oct05 --mach cheyenne --res CLM_USRDAT --compset COMPSET=2000_DATM%GSWP3v1_CLM50%FATES_SICE_SOCN_RTM_SGLC_SWAV --mpilib mpi-serial --run-unsupported --project UCNN0040

B) Generate fsurdat file
Cd CTSM/tools/site_and_regional
pip install numpy
pip install xarray
./subset_data region --lat1 24.5 --lat2 49.4 --lon1 235 --lon2 294 --reg united_states --create-surface

C) Generate mesh files
ctsm/tools/site_and_regional/subset_data_regional:
module load nco
ncks --rgr infer --rgr scrip=scrip.nc surfdata_united_states_hist_16pfts_Irrig_CMIP6_simyr2000_c231005.nc foo.nc
/glade/u/apps/ch/opt/esmf-netcdf/8.0.0/intel/19.0.5/bin/bing/Linux.intel.64.mpiuni.default/ESMF_Scrip2Unstruct scrip.nc lnd_mesh.nc 0
ncap2 -s 'elementMask(:)=0' lnd_mesh.nc mask_mesh.nc

D) Run the CTSM
cd oct05
./xmlchange NTASKS = -4
./xmlchange NTASKS_PER_INST = 144
./xmlchange JOB_WALLCLOCK_TIME = 12:00:00
./xmlchange JOB_QUEUE = regular
./xmlchange PIO_TYPENAME = pnetcdf
./case.setup

Add line to user_nl_clm:
fsurdat = ‘subset_data_regional/surfdata_united_states_hist_16pfts_Irrig_CMIP6_simyr2000_c231004.nc’

In the last line that I colored it with blue, I faced this error:

If 'fsurdat' is not a typo you can use command-not-found to lookup the package that contains it, like this:
cnf fsurdat

How can I solved this error?

 

[oleson]

The user_nl_clm in your case directory (/glade/u/home/jteymoori/oct05/user_nl_clm) doesn't have any information regarding fsurdat in it. Where did you add that information? It should look like this:

!----------------------------------------------------------------------------------
! Users should add all user specific namelist changes below in the form of
! namelist_var = new_namelist_value
!
! EXCEPTIONS:
! Set use_cndv by the compset you use and the CLM_BLDNML_OPTS -dynamic_vegetation setting
! Set use_vichydro by the compset you use and the CLM_BLDNML_OPTS -vichydro setting
! Set use_cn by the compset you use and CLM_BLDNML_OPTS -bgc setting
! Set use_crop by the compset you use and CLM_BLDNML_OPTS -crop setting
! Set spinup_state by the CLM_BLDNML_OPTS -bgc_spinup setting
! Set co2_ppmv with CCSM_CO2_PPMV option
! Set fatmlndfrc with LND_DOMAIN_PATH/LND_DOMAIN_FILE options
! Set finidat with RUN_REFCASE/RUN_REFDATE/RUN_REFTOD options for hybrid or branch cases
! (includes $inst_string for multi-ensemble cases)
! or with CLM_FORCE_COLDSTART to do a cold start
! or set it with an explicit filename here.
! Set maxpatch_glc with GLC_NEC option
! Set glc_do_dynglacier with GLC_TWO_WAY_COUPLING env variable
!----------------------------------------------------------------------------------
fsurdat = '/glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/surfdata_united_states_hist_16pfts_Irrig_CMIP6_simyr2000_c231005.nc'

 

[jteymoori]

Based on the instruction parapeted by @slevis for Instructions for running regional CTSM-FATES at regular 1-degree resolution, in the case. build step I faced this error:

jteymoori@cheyenne5:~/oct19> ./case.build
Building case in directory /glade/u/home/jteymoori/oct19
sharedlib_only is False
model_only is False
Setting resource.RLIMIT_STACK to -1 from (307200000, -1)
Generating component namelists as part of build
2023-10-19 10:49:43 atm
Create namelist for component datm
Calling /glade/work/jteymoori/CTSM/components/cdeps/datm/cime_config/buildnml
2023-10-19 10:49:43 lnd
Create namelist for component clm
Calling /glade/work/jteymoori/CTSM/cime_config/buildnml
ERROR: Command /glade/work/jteymoori/CTSM/bld/build-namelist failed rc=255
out=
err=::ERROR(Build::Namelist::_parse_next): expect a F90 constant for a namelist instead got: ‘/glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/surfdata_united_states_16pfts_Irrig_CMIP6_simyr2000_c231019.nc’
jteymoori@cheyenne5:~/oct19> client_loop: send disconnect: Connection reset by p eer

What is this error? How could it be solved?

 

[slevis]

The file name in the error message agrees with your user_nl_clm
/glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/surfdata_united_states_16pfts_Irrig_CMIP6_simyr2000_c231019.nc
BUT it differs from the string of the actual file name
/glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/surfdata_united_states_hist_16pfts_Irrig_CMIP6_simyr2000_c231019.nc

If you correct this simple mistake, I hope it will work.

 

[jteymoori]

I created a new case, checked the file name with my user_nl_clm, and ran the scripts. ./case.build, but I faced the same error:

jteymoori@cheyenne3:~/Nov01> ./case.build
Building case in directory /glade/u/home/jteymoori/Nov01
sharedlib_only is False
model_only is False
Setting resource.RLIMIT_STACK to -1 from (307200000, -1)
Generating component namelists as part of build
2023-11-01 16:39:46 atm
Create namelist for component datm
Calling /glade/work/jteymoori/CTSM/components/cdeps/datm/cime_config/buildnml
2023-11-01 16:39:46 lnd
Create namelist for component clm
Calling /glade/work/jteymoori/CTSM/cime_config/buildnml
ERROR: Command /glade/work/jteymoori/CTSM/bld/build-namelist failed rc=255
out=
err=::ERROR(Build::Namelist::_parse_next): expect a F90 constant for a namelist instead got: ‘/glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/surfdata_united_states_hist_16pfts_Irrig_CMIP6_simyr2000_c231101.nc’
jteymoori@cheyenne3:~/Nov01>

 

[oleson]

Your "quotes" should be:

fsurdat = '/glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/surfdata_united_states_hist_16pfts_Irrig_CMIP6_simyr2000_c231101.nc'

not:

fsurdat = ‘/glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/surfdata_united_states_hist_16pfts_Irrig_CMIP6_simyr2000_c231101.nc’

 

[jteymoori]

I submitted my case and I received this email that said:
Job deleted as a result of dependency on job 4115393.chadmin1.ib0.cheyenne.ucar.edu
why my job was deleted and how can I solve this?


my cases

 

[slevis]

In the directory where the simulation was supposed to run:
Did it run at all?

 

[jteymoori]

No I could not run it

 

[slevis]

In order to troubleshoot why the simulation did not run, you will need to look for error messages in the log files. The message in the email is not helpful other than to let you know that the model stopped.

 

[jteymoori]

After submit my case i just received this email:

PBS Job Id: 4115394.chadmin1.ib0.cheyenne.ucar.edu
Job Name: st_archive.Nov08
Aborted by PBS Server
Job deleted as result of dependency on job 4115393.chadmin1.ib0.cheyenne.ucar

I do not know how I can troubleshoot or look for error messages in the log files .How can I do it?

 

[slevis]

To learn about these things, I recommend reading the clm user's guide. You may also find tutorial materials on the CESM/CTSM websites.

 

[jteymoori]

by running : ./preview_run , I have this:


CASE INFO:
nodes: 1
total tasks: 1
tasks per node: 1
thread count: 1
ngpus per node: 0

BATCH INFO:
FOR JOB: case.run
ENV:
Setting Environment ESMF_RUNTIME_PROFILE=ON
Setting Environment ESMF_RUNTIME_PROFILE_OUTPUT=SUMMARY
Setting Environment MPI_DSM_VERBOSE=true
Setting Environment MPI_TYPE_DEPTH=16
Setting Environment MPI_USE_ARRAY=None
Setting Environment OMP_NUM_THREADS=1
Setting Environment OMP_STACKSIZE=1024M
Setting Environment OMP_WAIT_POLICY=PASSIVE
Setting Environment TMPDIR=/glade/scratch/jteymoori
Setting Environment UGCSADDONPATH=/glade/work/turuncu/FV3GFS/addon
Setting Environment UGCSFIXEDFILEPATH=/glade/work/turuncu/FV3GFS/fix_am
Setting Environment UGCSINPUTPATH=/glade/work/turuncu/FV3GFS/benchmark-inputs/2012010100/gfs/fcst

SUBMIT CMD:
qsub -q regular -l walltime=12:00:00 -A UCNN0040 -v ARGS_FOR_SCRIPT='--resubmit' .case.run

MPIRUN (job=case.run):
/glade/scratch/jteymoori/Nov08/bld/cesm.exe >> cesm.log.$LID 2>&1

FOR JOB: case.st_archive
ENV:
Setting Environment ESMF_RUNTIME_PROFILE=ON
Setting Environment ESMF_RUNTIME_PROFILE_OUTPUT=SUMMARY
Setting Environment MPI_DSM_VERBOSE=true
Setting Environment MPI_TYPE_DEPTH=16
Setting Environment MPI_USE_ARRAY=None
Setting Environment OMP_NUM_THREADS=1
Setting Environment OMP_STACKSIZE=1024M
Setting Environment OMP_WAIT_POLICY=PASSIVE
Setting Environment TMPDIR=/glade/scratch/jteymoori
Setting Environment UGCSADDONPATH=/glade/work/turuncu/FV3GFS/addon
Setting Environment UGCSFIXEDFILEPATH=/glade/work/turuncu/FV3GFS/fix_am
Setting Environment UGCSINPUTPATH=/glade/work/turuncu/FV3GFS/benchmark-inputs/2012010100/gfs/fcst

SUBMIT CMD:
qsub -q regular -l walltime=12:00:00 -A UCNN0040 -W depend=afterok:0 -v ARGS_FOR_SCRIPT='--resubmit' case.st_archive

 

[oleson]

If this is your case directory:

/glade/u/home/jteymoori/Nov08

I see that you are using mpi-serial which is only for single-point simulations and it looks like you are trying to run a regional simulation.
I did this in your case directory:

./xmlquery MPILIB

and got:

MPILIB: mpi-serial

I think you would want to go back to the default setting of "mpt"

 

[jteymoori]

I want to run a regional simulation and based on this link "Setting up (high-res sparse) regional-grid CTSM simulations · ESCOMP/CTSM · Discussion #1919" which Mr. Levis introduced to me, I did all steps and I do not know why is simulated a single-point .
How can I do it?

 

[oleson]

I'm not sure why those instructions change MPILIB to mpi-serial for a regional simulation. I will check with @slevis tomorrow.