Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

Error while creating surface dataset for single point simulation using PTCLM tool

J

Jyoti Singh

Jyoti Singh
Member
I am trying to create surface datasets for a single point simulation having down atmospheric data using tool PTCLM.
Started with adding information of the experimental single pint in sitedata, soildata and pftdata in PTCLMsite data directory of PTCLM

After this foloowed these steps:
# setup the names to be used
setenv CESM_ROOT /glade/work/jyotis/clm5.0
setenv CSMDATA /glade/p/cesmdata/cseg/inputdata
setenv MYSITE 1x1pt_US-UMB
# Setup input data directory
setenv MYDATADIR /glade/work/jyotis/inputdata
mkdir -p $MYDATADIR/$MYSITE

# Running on Cheyenne with these modules loaded:
module load ncarenv/1.3
module load intel/18.0.5
module load ncarcompilers/0.5.0
module load nco/4.9.5
module load netcdf/4.8.1
module load ncl/6.6.2
module load mpt/2.22
module load esmf_libs/7.1.0r
Module load ncview/2.1.7
module load python

# Executing PTCLMmkdata.py
$CESM_ROOT/tools/PTCLM/PTCLMmkdata -s CH-XIA -d $CSMDATA mydatadir $MYDATADIR

# Message on terminal

1646770545075.png1646770545075.png
I can not understand the reason behind the error and how to solve this.
 
O

oleson

Keith Oleson
CSEG and Liaisons
Staff member
Are you submitting this to a compute node on cheyenne or running it interactively?
 
J

Jyoti Singh

Jyoti Singh
Member
I am submitting this job to compute node.

# Executing PTCLMmkdata.py
$CESM_ROOT/tools/PTCLM/PTCLMmkdata -s CH-XIA -d $CSMDATA mydatadir $MYDATADIR
 
O

oleson

Keith Oleson
CSEG and Liaisons
Staff member
I tried this with release-clm5.0.34 and it failed while creating the first mapping file, no useful error messages.
I've had success with this in later versions of the code (ctsm1.0.dev068), so I've tried it there and it is working so far (first six mapping files have been created). So you could try that.
I've also had success in the past with creating the mapping files with release-clm5.0.34, using mkmapdata.sh separately from PTCLM. So I'm trying this and it seems to be working so far:

qcmd -A P93300041 -l walltime=04:00:00 -l select=3:ncpus=1:mpiprocs=1:mem=109GB -- ./mkmapdata.sh -r 1x1_CH-XIA -f /glade/work/oleson/release-clm5.0.34/tools/PTCLM/mydatafiles/1x1pt_CH-XIA/SCRIPgrid_1x1pt_CH-XIA_nomask_c220309.nc -t regional > & ! mkmapdata_CH-XIA.out &

If you keep having trouble, I'll also note that PTCLM is being deprecated in favor of some subsetting python scripts (tools/site_and_regional in the latest version of the code).
 
J

Jyoti Singh

Jyoti Singh
Member
I tried using mkmapdata.sh separately from PTCLM from already created grid files:

qcmd -A P93300041 -l walltime=04:00:00 -l select=3:ncpus=1:mpiprocs=1:mem=109GB -- ./mkmapdata.sh -r 1x1_CH-XIA -f /glade/work/oleson/release-clm5.0.34/tools/PTCLM/mydatafiles/1x1pt_CH-XIA/SCRIPgrid_1x1pt_CH-XIA_nomask_c220309.nc -t regional > & ! mkmapdata_CH-XIA.out &

It runs fines and created some mapping files, however stops in between without any error message.

I tried running PTCLM again and this time requested for walltime. It created all the mapping files and fails to create surafcedata map and the error message in mksurfdata_map.log is
ERROR: pft_idx out of range = 22

I think this is related to the pff information I added in the /glade/work/jyotis/clm5.0/tools/PTCLM/PTCLM_sitedata/PTCLMDATA_pftdata.txt file.

site_code,pft_f1,pft_c1,pft_f2,pft_c2,pft_f3,pft_c3,pft_f4,pft_c4,pft_f5,pft_c5
.
.
.
.
CH-XIA,100,22,0,0,0,0,0,0,0,0 % This is the information I added in the file
I added pot percentage as 100 for irrigated winter wheat (22).

However, I do not know why PFT 22 is out of range.
 
O

oleson

Keith Oleson
CSEG and Liaisons
Staff member
It sounds like you are creating a non-crop surface dataset (16pfts) and you need a crop surface dataset (78pfts).
You can try adding "--crop" to your PTCLM command.
 
J

Jyoti Singh

Jyoti Singh
Member
I ran PTCLM, thi \s time adding --crop:

qcmd -A URTG0006 -l walltime=04:00:00 -l select=3:ncpus=1:mpiprocs=1:mem=109GB -- ./PTCLMmkdata -s CH-XIA -d /glade/p/cesmdata/cseg/inputdata --mydatadir /glade/work/jyotis/inputdata --crop >& ptclmrun.log &

all the surface data maps are created.
 
O

oleson

Keith Oleson
CSEG and Liaisons
Staff member
Great, thanks for the update. I'm going to move this to the CLM Forum which I should have done before.
 
J

Jyoti Singh

Jyoti Singh
Member
I am trying to build a case for single point data after changing env_run.xmil and user_nl_clm. The following error message is printed on the screen:

jyotis@cheyenne2:/glade/work/jyotis/point_runs/CH-XIA_default> ./case.build
Building case in directory /glade/work/jyotis/point_runs/CH-XIA_default
sharedlib_only is False
model_only is False
Setting resource.RLIMIT_STACK to -1 from (307200000, -1)
Generating component namelists as part of build
Creating component namelists
Calling /glade/work/jyotis/clm5.0/cime/src/components/data_comps/datm/cime_config/buildnml
Calling /glade/work/jyotis/clm5.0/cime_config/buildnml
ERROR: Command /glade/work/jyotis/clm5.0/bld/build-namelist failed rc=255
out=
err=::ERROR(Build::Namelist::_parse_next): expect a equal '=' sign or '+=', instead got: '

I do not understand the error.
 
O

oleson

Keith Oleson
CSEG and Liaisons
Staff member
Usually indicates a syntax error in one of your user_* files. I'm not sure if this is the problem, but this is not needed in your user_nl_clm:


&clm_inparm
 
J

Jyoti Singh

Jyoti Singh
Member
This is my user_nl_clm file, can you spot any problem in it?

!----------------------------------------------------------------------------------
! Users should add all user specific namelist changes below in the form of
! namelist_var = new_namelist_value
!
! EXCEPTIONS:
! Set use_cndv by the compset you use and the CLM_BLDNML_OPTS -dynamic_vegetation setting
! Set use_vichydro by the compset you use and the CLM_BLDNML_OPTS -vichydro setting
! Set use_cn by the compset you use and CLM_BLDNML_OPTS -bgc setting
! Set use_crop by the compset you use and CLM_BLDNML_OPTS -crop setting
! Set spinup_state by the CLM_BLDNML_OPTS -bgc_spinup setting
! Set irrigate by the CLM_BLDNML_OPTS -irrig .true. setting
! Set co2_ppmv with CCSM_CO2_PPMV option
! Set dtime with L_NCPL option
! Set fatmlndfrc with LND_DOMAIN_PATH/LND_DOMAIN_FILE options
! Set finidat with RUN_REFCASE/RUN_REFDATE/RUN_REFTOD options for hybrid or branch cases
! (includes $inst_string for multi-ensemble cases)
! or with CLM_FORCE_COLDSTART to do a cold start
! or set it with an explicit filename here.
! Set maxpatch_glcmec with GLC_NEC option
! Set glc_do_dynglacier with GLC_TWO_WAY_COUPLING env variable
!----------------------------------------------------------------------------------
&clm_inparm
paramfile='/glade/work/jyotis/inputdata/editparam/clm5_params.c200717.nc'
fsurdat='/glade/work/jyotis/inputdata/lnd/clm2/surfdata_map/surfdata_1x1pt_CH-XIA_hist_78pfts_CMIP6_simyr2000_c220314.nc'
hist_nhtfrq=-24,-24
hist_mfilt=6000000,6000000
hist_fincl2='ALEAF','ASTEM','AROOT','AGRAIN','SNOW_DEPTH','FCTR','FCEV','FGEV','FSH_V','FSH_G','HBOT','HTOP','FPSN_WC','FPSN_WJ','FPSN_WP','ELAI','PHASE','CRPYLD','DMYIELD','GDDTSOI','NDEP_TO_SMINN','NFIX_TO_SMINN','FERT','SMINN_TO_PLANT','SUPPLEMENT_TO_SMINN','SMINN','C_ALLOMETRY','N_ALLOMETRY','AVAILC','PLANT_NDEMAND','PLANT_NALLOC','PLANT_CALLOC','SMINN_TO_NPOOL','RETRANSN_TO_NPOOL','TOTPFTN','PFT_NTRUNC','TOTVEGN','GDDPLANT','TLAI','TSAI','TSA','LEAFC','LEAFN','FROOTC','FROOTN','NPP','NEE','SEEDC','SEEDN','GPP','GRAINC','GRAINN','LIVESTEMC','LIVESTEMN','LITTERC','TOTCOLC','TOTCOLN','TOTECOSYSC','TOTECOSYSN','TOTLITC','TOTLITN','TOTPFTC','TOTPFTN','TOTSOMC','TOTSOMN','TOTVEGC','TOTVEGN','BTRAN','SOILC','TV','EFLX_LH_TOT','VF','GDDHARV','GDD020','TCROWN','RATEH','RATED','RATES','RATER','FSURV','WDD','LT50','ACCFSURV','COUNTFSURV','A5TMIN','A10TMIN','DOWNREG','EXCESS_CFLUX’,'GRAINC_TO_FOOD'
 
J

Jyoti Singh

Jyoti Singh
Member
Thanks that was the problem.

However, I am facing another problem now, after creating a case and modifying env_run.xml I set up the case and then cretaeds a strem file and modified user_nlk_clm and user_nl_datm then tried ./preview_namelists and following was printed on the screen:

jyotis@cheyenne3:/glade/work/jyotis/point_runs/CH-XIA_def_param> ./preview_namelists
Setting resource.RLIMIT_STACK to -1 from (307200000, -1)
Creating component namelists
Calling /glade/work/jyotis/clm5.0.34/cime/src/components/data_comps/datm/cime_config/buildnml
Running /glade/work/jyotis/clm5.0.34/cime/src/components/data_comps/datm/cime_config/buildnml
Traceback (most recent call last):
File "/glade/work/jyotis/clm5.0.34/cime/src/components/data_comps/datm/cime_config/buildnml", line 251, in <module>
_main_func()
File "/glade/work/jyotis/clm5.0.34/cime/src/components/data_comps/datm/cime_config/buildnml", line 247, in _main_func
buildnml(case, caseroot, "datm")
File "/glade/work/jyotis/clm5.0.34/cime/src/components/data_comps/datm/cime_config/buildnml", line 229, in buildnml
_create_namelists(case, confdir, inst_string, namelist_infile, nmlgen, data_list_path)
File "/glade/work/jyotis/clm5.0.34/cime/src/components/data_comps/datm/cime_config/buildnml", line 123, in _create_namelists
nmlgen.create_stream_file_and_update_shr_strdata_nml(config, case.get_value("CASEROOT"),stream, stream_path, data_list_path)
File "/glade/work/jyotis/clm5.0.34/cime/scripts/Tools/../../scripts/lib/CIME/nmlgen.py", line 451, in create_stream_file_and_update_shr_strdata_nml
strmobj = Stream(infile=stream_path)
File "/glade/work/jyotis/clm5.0.34/cime/scripts/Tools/../../scripts/lib/CIME/XML/stream.py", line 22, in __init__
GenericXML.__init__(self, infile, schema=schema)
File "/glade/work/jyotis/clm5.0.34/cime/scripts/Tools/../../scripts/lib/CIME/XML/generic_xml.py", line 57, in __init__
self.read(infile, schema)
File "/glade/work/jyotis/clm5.0.34/cime/scripts/Tools/../../scripts/lib/CIME/XML/generic_xml.py", line 87, in read
self.read_fd(fd)
File "/glade/work/jyotis/clm5.0.34/cime/scripts/Tools/../../scripts/lib/CIME/XML/generic_xml.py", line 112, in read_fd
self.tree = ET.parse(fd)
File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1182, in parse
tree.parse(source, parser)
File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 656, in parse
parser.feed(data)
File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1653, in feed
self._raiseerror(v)
File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1517, in _raiseerror
raise err
xml.etree.ElementTree.ParseError: junk after document element: line 4, column 6
ERROR: /glade/work/jyotis/clm5.0.34/cime/src/components/data_comps/datm/cime_config/buildnml /glade/work/jyotis/point_runs/CH-XIA_def_param FAILED, see above



I do not understand the error but I think it is related to the stream added in user_nl_datm.
The content of my user_nl_datm is:
taxmode = 'limit'
streams ='datm.streams.txt.CLM1PT.CLM_USRDAT 2007 2007 2008'

and the contents of datm.streams.txt.CLM1PT.CLM_USRDAT is
1647921590000.png
What do you think may be the problem? The same error appears if I try to build the case.
 
O

oleson

Keith Oleson
CSEG and Liaisons
Staff member
I think it is telling you that you can't start a crop simulation other than at the first of the year if you are using an initial file that is from the first of the year (the default).

<entry id="RUN_STARTDATE" value="2007-11-15">

If you need to start at that STARTDATE, then try setting "CLM_FORCE_COLDSTART" to on.
 
You must log in or register to reply here.
Top