DATM setting for CLM compsets

[Dong Peng]

Hello, everyone. I been confused with the DATM setting we have been used in the hist or 1850 or 2000 compsets. The question is: does datm stream file will automatic align with the option of "RUN_STARDATE"? For example, if I set the RUN STARDATE = 2001, but the stream didn't change, then what stream I will use? BTW, I found the stream setting in datm_in was xxx.stream.txt. xxx 1 1901 1920 or 1850 1901 1920. I know the "1 1901 1920" were the align year, first year, and last year respectively. But what's the meaning of 1850?

I tried to correct the stream to match the RUN_startdate and STOP_N, but the case would like to call "unexpercted end-of-file". Here are the details of the case.

"(t_initf) Read in prof_inparm namelist from: drv_in
(t_initf) Using profile_disable= F
(t_initf) profile_timer= 4
(t_initf) profile_depth_limit= 4
(t_initf) profile_detail_limit= 2
(t_initf) profile_barrier= F
(t_initf) profile_outpe_num= 1
(t_initf) profile_outpe_stride= 0
(t_initf) profile_single_file= F
(t_initf) profile_global_stats= T
(t_initf) profile_ovhd_measurement= F
(t_initf) profile_add_detail= F
(t_initf) profile_papi_enable= F
ERROR: (shr_stream_init) ERROR: unexpected end-of-file "

user_nl_datm

streams = "user_datm.streams.txt.CLMGSWP3v1.Solar 2001 2001 2010",
"user_datm.streams.txt.CLMGSWP3v1.Precip 2001 2001 2010",
"user_datm.streams.txt.CLMGSWP3v1.TPQW 2001 2001 2010",
"datm.streams.txt.presaero.clim_1850 2001 2001 2010",
"datm.streams.txt.topo.observed 1 1 1"

datm_in

&datm_nml
decomp = "1d"
factorfn = "null"
force_prognostic_true = .false.
iradsw = 1
presaero = .true.
restfilm = "undefined"
restfils = "undefined"
wiso_datm = .false.
/
&shr_strdata_nml
datamode = "CLMNCEP"
domainfile = "/data4/home/dpeng/CESM/cesm2.1.3/inputdata/share/domains/domain.lnd.fv0.9x1.25_gx1v7.151020.nc"
dtlimit = 1.5, 1.5, 1.5, 1.5, 1.5
fillalgo = "nn", "nn", "nn", "nn", "nn"
fillmask = "nomask", "nomask", "nomask", "nomask", "nomask"
fillread = "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET"
fillwrite = "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET"
mapalgo = "bilinear", "bilinear", "bilinear", "bilinear", "bilinear"
mapmask = "nomask", "nomask", "nomask", "nomask", "nomask"
mapread = "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET"
mapwrite = "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET"
readmode = "single", "single", "single", "single", "single"
streams = "user_datm.streams.txt.CLMGSWP3v1.Solar 2001 2001 2010",
"user_datm.streams.txt.CLMGSWP3v1.Precip 2001 2001 2010",
"user_datm.streams.txt.CLMGSWP3v1.TPQW 2001 2001 2010",
"datm.streams.txt.presaero.clim_1850 2001 2001 2010",
"datm.streams.txt.topo.observed 1 1 1"
taxmode = "cycle", "cycle", "cycle", "cycle", "cycle"
tintalgo = "coszen", "nearest", "linear", "linear", "lower"
vectors = "null"

The user_stream_list only change the file list, such as


"<?xml version="1.0"?>
<file id="stream" version="1.0">
<dataSource>
GENERIC
</dataSource>
<domainInfo>
<variableNames>
time time
xc lon
yc lat
area area
mask mask
</variableNames>
<filePath>
/data4/home/dpeng/CESM/cesm2.1.3/inputdata/atm/datm7/atm_forcing.datm7.GSWP3.0.5d.v1.c170516
</filePath>
<fileNames>
domain.lnd.360x720_gswp3.0v1.c170606.nc
</fileNames>
</domainInfo>
<fieldInfo>
<variableNames>
PRECTmms precn
</variableNames>
<filePath>
/data4/home/dpeng/CESM/cesm2.1.3/inputdata/atm/datm7/atm_forcing.datm7.GSWP3.0.5d.v1.c170516/Precip
</filePath>
<fileNames>
clmforc.GSWP3.c2011.0.5x0.5.Prec.2001-01.nc
...
clmforc.GSWP3.c2011.0.5x0.5.Prec.2010-12.nc
</fileNames>
<offset>
0
</offset>
</fieldInfo>
</file>"

The files from 2001-01 to 2010-12 were matched as the default format, but I collapsed it here.

Thanks for reading.

 

[slevis]

You may set align, first, and last years equal to 1850, 1901, and 1920, when running a historical simulation from 1850 to 1920, cycling forcing data from 1901 to 1920. Though I think that you will end up with 1901 data for model year 1850, 1902 data for model year 1851 and so forth, which may not be an ideal alignment. A better alignment in this case may be 1901, 1901, 1920. If the simulation starts in 1850, the model will figure out which data to pull based on the requested alignment of 1901, 1901, 1920.

 

[slevis]

The unexpected end-of-file is difficult to troubleshoot unfortunately, and it's probably due to a typo from the changes that you introduced.

 

[Dong Peng]

Thanks, Slevis. The issues were solved by changing the env_run parameters.
I typed that:

2023-10-01 10:40:49: xmlchange success <command> ./xmlchange DATM_CLMNCEP_YR_END=2010 </command>
---------------------------------------------------
2023-10-01 10:40:57: xmlchange success <command> ./xmlchange DATM_CLMNCEP_YR_START=2001 </command>
---------------------------------------------------
2023-10-01 10:41:04: xmlchange success <command> ./xmlchange DATM_CLMNCEP_YR_ALIGN=2001 </command>

Then, the bugs were gone, and I could run the case for the simulation period from 2001 to 2010. The setting of user_nl_datm and modified user_stream can hardly work. [ Even The modified user_stream.txt lists were the same as the automatically created streams.txt lists after typing the code above.]

What should I do if I want to set specific stream lists?
For example, we want to evaluate the difference between dry years and wet years, such as 2001(dry), 2010(wet);
we set the experiment as 5 years dry condition versus. 5 years wet condition, then we would like to set the stream list as {[dry], (2001,2001,2001,2001,2001)} versus. {[wet], (2010,2010,2010,2010,2010)}, which the streams list of normal case should be {[normal], (2001,2002,2003,2004,2005)}.

I think it could help us to evaluate the extreme weather conditions by some terms.

 

[JIAA]

Hello all, I also met the similar question. I changed the RUN_STARTDATE=1901-01-01. When I changed the first thress lines in datm_in like below, I got the errors at the time of submit. The log is in the attachment. I found the CRU data in my inputdata is only available up to 1920, so I download the 1921 data from datapool. But it still have errors.
streams = "datm.streams.txt.CLMCRUNCEPv7.Solar 1901 1901 1921",
"datm.streams.txt.CLMCRUNCEPv7.Precip 1901 1901 1921",
"datm.streams.txt.CLMCRUNCEPv7.TPQW 1901 1901 1921",

The default settings in datm_in is :
streams = "datm.streams.txt.CLMCRUNCEPv7.Solar 1901 1901 1920",
"datm.streams.txt.CLMCRUNCEPv7.Precip 1901 1901 1920",
"datm.streams.txt.CLMCRUNCEPv7.TPQW 1901 1901 1920",
"datm.streams.txt.presaero.trans_1850-2000 1849 1849 2014",
"datm.streams.txt.topo.observed 1 1 1",
"datm.streams.txt.co2tseries.20tr 1850 1850 2014"


Another question is when I changed the datm_in like land-noLu experiment (referenced from https://svn-cesm2-expdb.cgd.ucar.edu/public/i.e21.IHIST.f09_g17.CMIP6-land-noLu.001/user_nl_datm ), RUN_STARTDATE=1850-01-01 can run, but RUN_STARTDATE=1901-01-01 got errors.
Changed lines are like below:
streams = "datm.streams.txt.CLMGSWP3v1.Solar 1901 1901 2014",
"datm.streams.txt.CLMGSWP3v1.Precip 1901 1901 2014",
"datm.streams.txt.CLMGSWP3v1.TPQW 1901 1901 2014",
"datm.streams.txt.presaero.trans_1850-2000 1849 1849 2015",
"datm.streams.txt.topo.observed 1 1 1",
"datm.streams.txt.co2tseries.20tr 1750 1750 2014"

Looking forward to your reply! Thank you!

 

[Dong Peng]

Hello all, I also met the similar question. I changed the RUN_STARTDATE=1901-01-01. When I changed the first thress lines in datm_in like below, I got the errors at the time of submit. The log is in the attachment. I found the CRU data in my inputdata is only available up to 1920, so I download the 1921 data from datapool. But it still have errors.
streams = "datm.streams.txt.CLMCRUNCEPv7.Solar 1901 1901 1921",
"datm.streams.txt.CLMCRUNCEPv7.Precip 1901 1901 1921",
"datm.streams.txt.CLMCRUNCEPv7.TPQW 1901 1901 1921",

The default settings in datm_in is :
streams = "datm.streams.txt.CLMCRUNCEPv7.Solar 1901 1901 1920",
"datm.streams.txt.CLMCRUNCEPv7.Precip 1901 1901 1920",
"datm.streams.txt.CLMCRUNCEPv7.TPQW 1901 1901 1920",
"datm.streams.txt.presaero.trans_1850-2000 1849 1849 2014",
"datm.streams.txt.topo.observed 1 1 1",
"datm.streams.txt.co2tseries.20tr 1850 1850 2014"


Another question is when I changed the datm_in like land-noLu experiment (referenced from https://svn-cesm2-expdb.cgd.ucar.edu/public/i.e21.IHIST.f09_g17.CMIP6-land-noLu.001/user_nl_datm ), RUN_STARTDATE=1850-01-01 can run, but RUN_STARTDATE=1901-01-01 got errors.
Changed lines are like below:
streams = "datm.streams.txt.CLMGSWP3v1.Solar 1901 1901 2014",
"datm.streams.txt.CLMGSWP3v1.Precip 1901 1901 2014",
"datm.streams.txt.CLMGSWP3v1.TPQW 1901 1901 2014",
"datm.streams.txt.presaero.trans_1850-2000 1849 1849 2015",
"datm.streams.txt.topo.observed 1 1 1",
"datm.streams.txt.co2tseries.20tr 1750 1750 2014"

Looking forward to your reply! Thank you!

Try typing the xmlchange command lines as I gave before. Remove the lines in user_nl_datm. Then check env_run for the specific options of DATM. The user_nl_datm seems cant work. You need to check the DATM stream-list-txt in CastDoc folder, and the datm_in. It wont change correctly if u just modified in user_nl_datm.

 

[JIAA]

Try typing the xmlchange command lines as I gave before. Remove the lines in user_nl_datm. Then check env_run for the specific options of DATM. The user_nl_datm seems cant work. You need to check the DATM stream-list-txt in CastDoc folder, and the datm_in. It wont change correctly if u just modified in user_nl_datm.

In my opinion, what you are doing（./xmlchange DATM_CLMNCEP_YR_END .etc） is setting the align,start,end years of the stream files. And what I do in user_nl_datm is also for the same purpose. Why do I get an error if I change the end year? I don't understand.

 

[Dong Peng]

In my opinion, what you are doing（./xmlchange DATM_CLMNCEP_YR_END .etc） is setting the align,start,end years of the stream files. And what I do in user_nl_datm is also for the same purpose. Why do I get an error if I change the end year? I don't understand.

Sure. The different ways did the same things to change the DATM settings. But I can't make it through by changing user_nl_datm. That's also what confused me. Maybe you can try both the two methods to achieve the same goal. ref the link of the guidance. ref

 

[JIAA]

Sure. The different ways did the same things to change the DATM settings. But I can't make it through by changing user_nl_datm. That's also what confused me. Maybe you can try both the two methods to achieve the same goal. ref the link of the guidance. ref

Thank you sir! I'll take a closer look.

 

[Dong Peng]

Thanks, Slevis. The issues were solved by changing the env_run parameters.
I typed that:

2023-10-01 10:40:49: xmlchange success <command> ./xmlchange DATM_CLMNCEP_YR_END=2010 </command>
---------------------------------------------------
2023-10-01 10:40:57: xmlchange success <command> ./xmlchange DATM_CLMNCEP_YR_START=2001 </command>
---------------------------------------------------
2023-10-01 10:41:04: xmlchange success <command> ./xmlchange DATM_CLMNCEP_YR_ALIGN=2001 </command>

Then, the bugs were gone, and I could run the case for the simulation period from 2001 to 2010. The setting of user_nl_datm and modified user_stream can hardly work. [ Even The modified user_stream.txt lists were the same as the automatically created streams.txt lists after typing the code above.]

What should I do if I want to set specific stream lists?
For example, we want to evaluate the difference between dry years and wet years, such as 2001(dry), 2010(wet);
we set the experiment as 5 years dry condition versus. 5 years wet condition, then we would like to set the stream list as {[dry], (2001,2001,2001,2001,2001)} versus. {[wet], (2010,2010,2010,2010,2010)}, which the streams list of normal case should be {[normal], (2001,2002,2003,2004,2005)}.

I think it could help us to evaluate the extreme weather conditions by some terms.

@oleson

 

[oleson]

If you want to repeat a single year of atmospheric forcing, then for example, set:
DATM_CLMNCEP_YR_ALIGN=2001
DATM_CLMNCEP_YR_START=2001
DATM_CLMNCEP_YR_END=2001
RUN_STARTDATE = 2001-01-01
and make sure "taxmode" is set to "cycle"
The datm will repeat the 2001 forcing for as many years as you run the simulation.

 

[Dong Peng]

If you want to repeat a single year of atmospheric forcing, then for example, set:
DATM_CLMNCEP_YR_ALIGN=2001
DATM_CLMNCEP_YR_START=2001
DATM_CLMNCEP_YR_END=2001
RUN_STARTDATE = 2001-01-01
and make sure "taxmode" is set to "cycle"
The datm will repeat the 2001 forcing for as many years as you run the simulation.

Thanks, @oleson. Another question is can I use the cmip data as the precipitation stream of DATM after 2014 for CLM5?

 

[slevis]

The answer is YES. Any precip data will work, as long as you put the data in the format and units that the model expects.

 

[Dong Peng]

The answer is YES. Any precip data will work, as long as you put the data in the format and units that the model expects.

@oleson No offense. It would be great if you guys could provide the details of the formats and units of the template file and the setting of the DATM in specific cases, rather than the roughly yes. Thank you for your help. Of course, I can put my files into the same file name and location as well as the default DATM files, but I prefer to find a better solution for this. Besides, we appreciate your help for the whole time, which will always indicated in my related articles' acknowledgment.

 

[slevis]

@Dong Peng
Your request makes sense in theory, but it is not something that we (at NCAR) support. We offer the model as is, including training workshops and documentation. However, when researchers make changes to the model, we may offer ideas and suggestions depending on our expertise and abilities, and that's the extent of our help.

In this Forum you may find others who are willing to help, and that would be great.

I sincerely apologize if my response is disappointing.

 

[Dong Peng]

@Dong Peng
Your request makes sense in theory, but it is not something that we (at NCAR) support. We offer the model as is, including training workshops and documentation. However, when researchers make changes to the model, we may offer ideas and suggestions depending on our expertise and abilities, and that's the extent of our help.

In this Forum you may find others who are willing to help, and that would be great.

I sincerely apologize if my response is disappointing.

Hi S. Levis @slevis . Your answer makes sense. Thanks for your help. I noticed that the issue about updating the DATM input dataset in the highlight of the CLM60 like this Update to CRU-JRA as default datm inputs for CTSM development? The out-of-date datasets seem to have troubles with development. How about build tools to create specific datasets for simulation by users or it could be better if the user would like to share their datasets? Lots of datasets are available for simulation but most of them require the copyright license which NCAR won't accept but the user can afford. I am a researcher rather than a developer, I prefer to make the CESM2 a better simply-use tool for our scientific goals. Thanks.

 

[slevis]

@Dong Peng you are welcome to contact other researchers (such as in the issue that you referenced) and ask if they have helpful tools that they can share with you.