Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

How select a special region to run the CLM from the global model in CLM?

O

oleson

Keith Oleson
CSEG and Liaisons
Staff member
I checked with @slevis . His setup set mpilib to mpi-serial and then made changes later.
So you should remove the --mpilib mpi-serial specification from your create_newcase command.
 
J

jteymoori

Javad Teymoori
Member
I submitted my case, and I want to see the outputs of this run, it is expected the NC file will be in this directory:
"/glade/scratch/jteymoori/archive/Nov21/lnd/hist"
but in the his, it does not have any files.

just in the run directory: " /glade/scratch/jteymoori/Nov21/run>" it includes some log files.

How can I find the NC files resulted from this run?
 
O

oleson

Keith Oleson
CSEG and Liaisons
Staff member
It looks like maybe you only ran five days. By default, you would need to run at least a month to get history files.
 
O

oleson

Keith Oleson
CSEG and Liaisons
Staff member
./xmlchange STOP_OPTION=nmonths
./xmlchange STOP_N=1

If you haven't already, you might consider going through the CESM tutorial material. These are the types of questions that are answered there.
 
J

jteymoori

Javad Teymoori
Member
Hi
I want to run my case for one year and I used : ./xmlchange STOP_OPTION=nmonths and ./xmlchange STOP_N=12
but when I build my case I received the below email:
PBS Job Id: 4325169.chadmin1.ib0.cheyenne.ucar.edu
Job Name: st_archive.Dec05
Aborted by PBS Server
Job deleted as result of dependency on job 4325168.chadmin1.ib0.cheyenne.ucar.edu

why my job was deleted?
Do you think I should use /xmlchange STOP_OPTION=nyears and ./xmlchange STOP_N=1 or the error is related to another thing?
 
O

oleson

Keith Oleson
CSEG and Liaisons
Staff member
It is deleting your archiving job because your main model run job failed. You should look at the log files in your run directory to see what the error is. Your settings of STOP_OPTION and STOP_N should be fine.
 
J

jteymoori

Javad Teymoori
Member
I ran the model for one year by . /xmlchange STOP_OPTION=nyears and ./xmlchange STOP_N=1
The output of this simulation includes 12 NETCDF files (a NetCDF for each month)
I expected the output to be one yearly NetCDF for one year, not 12 monthly NetCDFs.
How can I have a one-yearly NetCDF for this simulation?
 
O

oleson

Keith Oleson
CSEG and Liaisons
Staff member
The settings of STOP_OPTION and STOP_N don't have anything to do with history file output.
The default history file setting is monthly history file output.
If you want to change this then see the settings beginning with "hist_" on the CLM namelist page. These settings would be added to your user_nl_clm.

CESM2 CLM Namelist Definitions

Namelist definitions for CLM in CESM2
docs.cesm.ucar.edu docs.cesm.ucar.edu
 
J

jteymoori

Javad Teymoori
Member
I ran my model, but for my output, I have some problems; as you see in the attached picture, in some areas (upper side of the map), There are diagonal lines in the upper area of the map, and there is no data between these lines. Maybe the model could not run in these areas.
How can I solve this error? and Why am I faced with this?
 

Attachments

  • USA.png
    USA.png
    142.7 KB · Views: 3
O

oleson

Keith Oleson
CSEG and Liaisons
Staff member
It looks like you've generated a lot of PET log files in your run directory, containing the error:

ERROR PET051 /glade/p/cesmdata/cseg/PROGS/build/26583/esmf-8.5.0/src/Infrastructure/Trace/src/ESMCI_Trace.C:1797 ESMCI::TraceEventRegionExit() Wrong argument specified - Trace regions not properly nested. Attempt to exit region: CNZero-vegbgc-nflux that was never entered.

I haven't seen this before. Maybe this is related to a mesh file problem? @slevis , do you have any ideas here?
The case directory is:
/glade/u/home/jteymoori/Dec05
and the run directory is:
/glade/scratch/jteymoori/Dec05/run
 
O

oleson

Keith Oleson
CSEG and Liaisons
Staff member
@slevis and I worked through the instructions again and realized that there is an assumption there that all of the grid cells are land. In your case, however, you have ocean and land points. So we've revised the instructions to allow for this. You can use the following steps for your particular case. I tried this in a clone of your simulation and got reasonable model output. A caveat is that I'm still getting PET files for some unknown reason, but the model completes and the output looks reasonable.

cp /glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/surfdata_united_states_hist_16pfts_Irrig_CMIP6_simyr2000_c231205.nc landmask.nc

ncrename -v PFTDATA_MASK,landmask landmask.nc

ncks -v PFTDATA_MASK /glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/surfdata_united_states_hist_16pfts_Irrig_CMIP6_simyr2000_c231205.nc landmask.nc

ncrename -v PFTDATA_MASK,mod_lnd_props landmask.nc

ncks --rgr infer --rgr scrip=scrip.nc landmask.nc foo.nc

The following line should use the updated version found in Discussion 1919:
/glade/u/apps/ch/opt/esmf-netcdf/8.0.0/intel/19.0.5/bin/bing/Linux.intel.64.mpiuni.default/ESMF_Scrip2Unstruct scrip.nc lnd_mesh.nc 0

./xmlchange MASK_MESH='YOUR_PATH_TO/lnd_mesh.nc'
 
Last edited by a moderator:
O

oleson

Keith Oleson
CSEG and Liaisons
Staff member
We find that one should also set the following to the lat/lon dimensions of the regional grid. For your simulation this would be:

./xmlchange LND_NX=47
./xmlchange LND_NY=26
./xmlchange ATM_NX=47
./xmlchange ATM_NY=26

There is also a timer bug in the CNDriverMod.F90 module which is causing all of the PET files in your run directory. See here for the issue filed:

github.com

t_startf is within an if statment and t_stopf is not in CNDriverMod.F90 · Issue #2291 · ESCOMP/CTSM

Brief summary of bug In CNDriverMod.F90, there is a t_startf call that is within an if statement, but the t_stopf is outside the if statement. Reported by @olyson , solution by @mvdebolskiy General...
github.com github.com

See here for a fix. Keep in mind that this is for ctsm5.1.dev156, so you will want to check for differences with your version of the code before adding this to your code:

/glade/work/oleson/ctsm5.1.dev156/cime/scripts/Dec05_d156/SourceMods/src.clm/CNDriverMod.F90

I was going to test this fix, but your fsurdat seems to have disappeared.
 
J

jteymoori

Javad Teymoori
Member
I have deleted the Dec05 case due to these errors, and I want to create a new case with your new instructions with the name Dec13, I want to run the below scripts.
Do you think it's true?
in addition, I want to run my model for one year, so I want to have a NetCDF that includes monthly data for one year instead of 12 separated monthly NETCDFs, so I added hist_mfit=12 and hist_nhtfrq=0 for the name list.


:

cd /glade/work/jteymoori
mkdir Dec13
git clone GitHub - ESCOMP/CTSM: Community Terrestrial Systems Model (includes the Community Land Model of CESM)
cd CTSM
./manage_externals/checkout_externals
cd /CTSM/cime/scripts
module load python
./create_newcase --case ~/Dec13 --mach cheyenne --res CLM_USRDAT --compset COMPSET=2000_DATM%GSWP3v1_CLM50%FATES_SICE_SOCN_RTM_SGLC_SWAV --run-unsupported --project UCNN0040
Cd CTSM/tools/site_and_regional
pip install numpy
pip install xarray
./subset_data region --lat1 24.5 --lat2 49.4 --lon1 235 --lon2 294 --reg united_states --create-surface
ctsm/tools/site_and_regional/subset_data_regional:
module load nco
cd /glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/surfdata_united_states_hist_16pfts_Irrig_CMIP6_simyr2000_c231213.nc landmask.nc
ncrename -v PFTDATA_MASK,landmask landmask.nc
ncks -v PFTDATA_MASK /glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/surfdata_united_states_hist_16pfts_Irrig_CMIP6_simyr2000_c231213.nc landmask.nc
ncrename -v PFTDATA_MASK,mod_lnd_props landmask.nc
ncks --rgr infer --rgr scrip=scrip.nc landmask.nc foo.nc

The following line should use the updated version found in Discussion 1919:
/glade/u/apps/ch/opt/esmf-netcdf/8.0.0/intel/19.0.5/bin/bing/Linux.intel.64.mpiuni.default/ESMF_Scrip2Unstruct scrip.nc lnd_mesh.nc 0
ncap2 -s 'elementMask(:)=0' lnd_mesh.nc mask_mesh.nc
cd Dec05
module load python
./xmlchange NTASKS=-4
./xmlchange NTASKS_PER_INST=44
./xmlchange JOB_WALLCLOCK_TIME=12:00:00
./xmlchange JOB_QUEUE=regular
./xmlchange PIO_TYPENAME=pnetcdf
./xmlchange STOP_OPTION=nyears
./xmlchange STOP_N=1
./xmlchange LND_NX=47
./xmlchange LND_NY=26
./xmlchange ATM_NX=47
./xmlchange ATM_NY=26
./case.setup

vi user_nl_clm

fsurdat = '/glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/surfdata_united_states_hist_16pfts_Irrig_CMIP6_simyr2000_c231213.nc'
hist_mfit=12
hist_nhtfrq=0
Esc
:wq

Update lines in env_run.xml:
./xmlchange LND_DOMAIN_MESH='/glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/lnd_mesh.nc'
./xmlchange ATM_DOMAIN_MESH='/glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/lnd_mesh.nc'
./xmlchange MASK_MESH='/glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/mask_mesh.nc'

./case.build
./case.submit
 
Last edited by a moderator:
J

jteymoori

Javad Teymoori
Member
I have deleted the Dec05 case due to these errors, and I want to create a new case with your new instructions with the name Dec13, I want to run the below scripts.
Do you think it's true?
in addition, I want to run my model for one year, so I want to have a NetCDF that includes monthly data for one year instead of 12 separated monthly NETCDFs, so I added hist_mfit=12 and hist_nhtfrq=0 for the name list.


:

cd /glade/work/jteymoori
mkdir Dec13
git clone GitHub - ESCOMP/CTSM: Community Terrestrial Systems Model (includes the Community Land Model of CESM)
cd CTSM
./manage_externals/checkout_externals
cd /CTSM/cime/scripts
module load python
./create_newcase --case ~/Dec13 --mach cheyenne --res CLM_USRDAT --compset COMPSET=2000_DATM%GSWP3v1_CLM50%FATES_SICE_SOCN_RTM_SGLC_SWAV --run-unsupported --project UCNN0040
Cd CTSM/tools/site_and_regional
pip install numpy
pip install xarray
./subset_data region --lat1 24.5 --lat2 49.4 --lon1 235 --lon2 294 --reg united_states --create-surface
ctsm/tools/site_and_regional/subset_data_regional:
module load nco
cd /glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/surfdata_united_states_hist_16pfts_Irrig_CMIP6_simyr2000_c231213.nc landmask.nc
ncrename -v PFTDATA_MASK,landmask landmask.nc
ncks -v PFTDATA_MASK /glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/surfdata_united_states_hist_16pfts_Irrig_CMIP6_simyr2000_c231213.nc landmask.nc
ncrename -v PFTDATA_MASK,mod_lnd_props landmask.nc
ncks --rgr infer --rgr scrip=scrip.nc landmask.nc foo.nc

The following line should use the updated version found in Discussion 1919:
/glade/u/apps/ch/opt/esmf-netcdf/8.0.0/intel/19.0.5/bin/bing/Linux.intel.64.mpiuni.default/ESMF_Scrip2Unstruct scrip.nc lnd_mesh.nc 0
ncap2 -s 'elementMask(:)=0' lnd_mesh.nc mask_mesh.nc

cd Dec13
module load python
./xmlchange NTASKS=-4
./xmlchange NTASKS_PER_INST=44
./xmlchange JOB_WALLCLOCK_TIME=12:00:00
./xmlchange JOB_QUEUE=regular
./xmlchange PIO_TYPENAME=pnetcdf
./xmlchange STOP_OPTION=nyears
./xmlchange STOP_N=1
./xmlchange LND_NX=47
./xmlchange LND_NY=26
./xmlchange ATM_NX=47
./xmlchange ATM_NY=26
./case.setup

vi user_nl_clm

fsurdat = '/glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/surfdata_united_states_hist_16pfts_Irrig_CMIP6_simyr2000_c231213.nc'
hist_mfit=12
hist_nhtfrq=0
Esc
:wq

Update lines in env_run.xml:
./xmlchange LND_DOMAIN_MESH='/glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/lnd_mesh.nc'
./xmlchange ATM_DOMAIN_MESH='/glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/lnd_mesh.nc'
./xmlchange MASK_MESH='/glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/mask_mesh.nc'

./case.build
./case.submit
 
Last edited by a moderator:
O

oleson

Keith Oleson
CSEG and Liaisons
Staff member
This should be cp, not cd:

cd /glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/surfdata_united_states_hist_16pfts_Irrig_CMIP6_simyr2000_c231213.nc landmask.nc

Don't do this:

ncap2 -s 'elementMask(:)=0' lnd_mesh.nc mask_mesh.nc

Do you mean 144 here, not 44:

./xmlchange NTASKS_PER_INST=44

As per the instructions I provided above, this:

./xmlchange MASK_MESH='/glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/mask_mesh.nc'

should be:

./xmlchange MASK_MESH='/glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/lnd_mesh.nc'
 
J

jteymoori

Javad Teymoori
Member
I completely confused
I have deleted the Dec05 case due to these errors, and I want to create a new case with your new instructions with the name Dec13.
I want to run the scripts below.
Do you think it's true?
In addition, I want to run my model for one year, so I want to have a NetCDF that includes monthly data for one year instead of 12 separated monthly NETCDFs, so I added hist_mfit=12 and hist_nhtfrq=0 for the name list.
cd /glade/work/jteymoori
mkdir Dec13
git clone GitHub - ESCOMP/CTSM: Community Terrestrial Systems Model (includes the Community Land Model of CESM)

cd CTSM
./manage_externals/checkout_externals
cd /CTSM/cime/scripts
module load python
./create_newcase --case ~/Dec13 --mach cheyenne --res CLM_USRDAT --compset COMPSET=2000_DATM%GSWP3v1_CLM50%FATES_SICE_SOCN_RTM_SGLC_SWAV --run-unsupported --project UCNN0040
Cd CTSM/tools/site_and_regional
pip install numpy
pip install xarray
./subset_data region --lat1 24.5 --lat2 49.4 --lon1 235 --lon2 294 --reg united_states --create-surface
ctsm/tools/site_and_regional/subset_data_regional:
module load nco
cd /glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/surfdata_united_states_hist_16pfts_Irrig_CMIP6_simyr2000_c231205.nc landmask.nc
ncrename -v PFTDATA_MASK,landmask landmask.nc
ncks -v PFTDATA_MASK /glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/surfdata_united_states_hist_16pfts_Irrig_CMIP6_simyr2000_c231213.nc landmask.nc
ncrename -v PFTDATA_MASK,mod_lnd_props landmask.nc
ncks --rgr infer --rgr scrip=scrip.nc landmask.nc foo.nc

The following line should use the updated version found in Discussion 1919:
/glade/u/apps/ch/opt/esmf-netcdf/8.0.0/intel/19.0.5/bin/bing/Linux.intel.64.mpiuni.default/ESMF_Scrip2Unstruct scrip.nc lnd_mesh.nc 0

cd Dec13
module load python
./xmlchange NTASKS=-4
./xmlchange NTASKS_PER_INST=44
./xmlchange JOB_WALLCLOCK_TIME=12:00:00
./xmlchange JOB_QUEUE=regular
./xmlchange PIO_TYPENAME=pnetcdf
./xmlchange STOP_OPTION=nyears
./xmlchange STOP_N=1
./xmlchange LND_NX=47
./xmlchange LND_NY=26
./xmlchange ATM_NX=47
./xmlchange ATM_NY=26
./xmlchange MASK_MESH='/glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/lnd_mesh.nc'
./case.setup

vi user_nl_clm

fsurdat = '/glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/surfdata_united_states_hist_16pfts_Irrig_CMIP6_simyr2000_c231213.nc'
hist_mfit=12
hist_nhtfrq=0
Esc
:wq

Update lines in env_run.xml:
./xmlchange LND_DOMAIN_MESH='/glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/lnd_mesh.nc'
./xmlchange ATM_DOMAIN_MESH='/glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/lnd_mesh.nc'
./xmlchange MASK_MESH='/glade/work/jteymoori/CTSM/tools/site_and_regional/subset_data_regional/mask_mesh.nc'

./case.build
./case.submit
 
Last edited by a moderator:
You must log in or register to reply here.
Top