Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

Anomaly forcing run adds extra streams at runtime, causing an error

J

James King

James King
Member
Hi all,

I'm seeing some odd behaviour using anomaly forcing with CLM5. I'm trying to run the compset ISSP245Clm50BgcCrop in CESM2.1.3. I create the case and add the following to user_nl_datm:

anomaly_forcing = 'Anomaly.Forcing.Precip', 'Anomaly.Forcing.Temperature', 'Anomaly.Forcing.Pressure', 'Anomaly.Forcing.Humidity', 'Anomaly.Forcing.Uwind', 'Anomaly.Forcing.Vwind', 'Anomaly.Forcing.Shortwave', 'Anomaly.Forcing.Longwave'

streams = "datm.streams.txt.CLMGSWP3v1.Solar 2001 2001 2013",
"datm.streams.txt.CLMGSWP3v1.Precip 2001 2001 2013",
"datm.streams.txt.CLMGSWP3v1.TPQW 2001 2001 2013",
"datm.streams.txt.presaero.SSP2-4.5 2015 2015 2101",
"datm.streams.txt.topo.observed 1 1 1",
"datm.streams.txt.co2tseries.SSP2-4.5 2015 2015 2500",
"datm.streams.txt.Anomaly.Forcing.Precip 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Temperature 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Pressure 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Humidity 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Uwind 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Vwind 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Shortwave 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Longwave 2015 2015 2100"

I then add user_datm_streams.txt* files to the case directory and run preview_namelists. However, looking at the DATM namelist shows a load of extra streams added and I don't know where these have come from (lines in bold):

streams = "datm.streams.txt.CLMGSWP3v1.Solar 2001 2001 2013",
"datm.streams.txt.CLMGSWP3v1.Precip 2001 2001 2013",
"datm.streams.txt.CLMGSWP3v1.TPQW 2001 2001 2013",
"datm.streams.txt.presaero.SSP2-4.5 2015 2015 2101",
"datm.streams.txt.topo.observed 1 1 1",
"datm.streams.txt.co2tseries.SSP2-4.5 2015 2015 2500",
"datm.streams.txt.Anomaly.Forcing.Precip 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Temperature 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Pressure 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Humidity 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Uwind 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Vwind 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Shortwave 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Longwave 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Precip 2006 2006 2300",
"datm.streams.txt.Anomaly.Forcing.Temperature 2006 2006 2300",
"datm.streams.txt.Anomaly.Forcing.Pressure 2006 2006 2300",
"datm.streams.txt.Anomaly.Forcing.Humidity 2006 2006 2300",
"datm.streams.txt.Anomaly.Forcing.Uwind 2006 2006 2300",
"datm.streams.txt.Anomaly.Forcing.Vwind 2006 2006 2300",
"datm.streams.txt.Anomaly.Forcing.Shortwave 2006 2006 2300",
"datm.streams.txt.Anomaly.Forcing.Longwave 2006 2006 2300"

The case runs for a few days, but trying to run for longer results in this error:

(shr_strdata_advance) ERROR: for stream 15
(shr_strdata_advance) ERROR: dt limit1 76316.000000000000 28.000000000000000 1.5000000000000000
(shr_strdata_advance) ERROR: dt limit2 20150116 0 20150213 0
ERROR: (shr_strdata_advance) ERROR dt limit for stream

I think this error is due to stream 15 being "datm.streams.txt.Anomaly.Forcing.Precip 2006 2006 2300", as the precip anomaly data only has data from 2015-2100, but I don't know how to fix this as I never specified these extra streams with *2006 2006 2300 at the end, and therefore I don't know how to remove them. Where are these lines coming from and how can I get rid of them?

Thanks,

James
 
O

oleson

Keith Oleson
CSEG and Liaisons
Staff member
I haven't seen that behavior before, but I can replicate it in release-cesm2.1.5. I think I solved it by removing this line in user_nl_datm as a final step after having done everything else:

anomaly_forcing = 'Anomaly.Forcing.Precip','Anomaly.Forcing.Temperature','Anomaly.Forcing.Pressure','Anomaly.Forcing.Humidity','Anomaly.Forcing.Uwind','Anomaly.Forcing.Vwind','Anomaly.Forcing.Shortwave','Anomaly.Forcing.Longwave'
 
O

oleson

Keith Oleson
CSEG and Liaisons
Staff member
Before case.build actually. I ran preview_namelists after setting the user_* files as you did and replicated your problem. Then I removed that line in user_nl_datm and ran preview_namelists again, which seemed to fix the problem.
 
J

James King

James King
Member
OK I see. This does seem to get the model to work, but obviously that anomaly forcing line is now not in the namelist. Will the model still read the anomaly forcing files as before?
 
O

oleson

Keith Oleson
CSEG and Liaisons
Staff member
Well, good point, I didn't think of that. I ran the model for a year and it seemed to be reading in the anomalies each month but I'm not positive it's applying those anomalies to the forcing. More later.
 
J

James King

James King
Member
Thanks for the update. I can also try doing this in CESM2.2, but CESM2.1.3 would be ideal for this partiicular set of experiments if we can resolve the issue.
 
J

James King

James King
Member
A suggestion - in the DATM source code that defines the namelist options (/cime/src/components/data_comps/datm/cime_config/namelist_definition_datm.xml) there are a lot of references to SSP options setting the forcing to run over 2006-2300. Replacing this with 2015-2100 in every relevant line I could find still gives a double call to each stream in the DATM namelist, but now they agree with each other, i.e.:

streams = "datm.streams.txt.CLMGSWP3v1.Solar 2001 2001 2013",
"datm.streams.txt.CLMGSWP3v1.Precip 2001 2001 2013",
"datm.streams.txt.CLMGSWP3v1.TPQW 2001 2001 2013",
"datm.streams.txt.presaero.SSP2-4.5 2015 2015 2101",
"datm.streams.txt.topo.observed 1 1 1",
"datm.streams.txt.co2tseries.SSP2-4.5 2015 2015 2500",
"datm.streams.txt.Anomaly.Forcing.Precip 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Temperature 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Pressure 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Humidity 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Uwind 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Vwind 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Shortwave 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Longwave 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Precip 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Temperature 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Pressure 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Humidity 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Uwind 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Vwind 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Shortwave 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Longwave 2015 2015 2100"

Will try running this and will update here if it fixes the issue.
 
J

James King

James King
Member
The atm log for this run shows the anomaly forcing files being read in at the midpoint of each month, which is good, but the files are being called twice. I've only run it for a year so not long enough to tell if this means that the anomalies are being applied twice, which is not good.

E.g. in the log:

(datm_comp_run) atm: model date 20150115 82800s
(datm_comp_run) atm: model date 20150115 84600s
(datm_comp_run) atm: model date 20150116 0s
(shr_dmodel_readstrm) file ub: /work/n02/n02/jking/cesm/CESM2.1.3/cesm_inputdata/atm/datm7/atm_forcing.datm7.GSWP3.0.5d.v1.c170516/Solar/clmforc.GSWP3.c2011.0.5x0.5.Solr.2002-01.nc 122
(shr_dmodel_readstrm) file ub: /work/n02/n02/jking/cesm/CESM2.1.3/cesm_inputdata/atm/datm7/anomaly_forcing/af.pr.CESM.SSP245.2015-2100.nc 2
(shr_dmodel_readstrm) file ub: /work/n02/n02/jking/cesm/CESM2.1.3/cesm_inputdata/atm/datm7/anomaly_forcing/af.tas.CESM.SSP245.2015-2100.nc 2
(shr_dmodel_readstrm) file ub: /work/n02/n02/jking/cesm/CESM2.1.3/cesm_inputdata/atm/datm7/anomaly_forcing/af.ps.CESM.SSP245.2015-2100.nc 2
(shr_dmodel_readstrm) file ub: /work/n02/n02/jking/cesm/CESM2.1.3/cesm_inputdata/atm/datm7/anomaly_forcing/af.huss.CESM.SSP245.2015-2100.nc 2
(shr_dmodel_readstrm) file ub: /work/n02/n02/jking/cesm/CESM2.1.3/cesm_inputdata/atm/datm7/anomaly_forcing/af.uas.CESM.SSP245.2015-2100.nc 2
(shr_dmodel_readstrm) file ub: /work/n02/n02/jking/cesm/CESM2.1.3/cesm_inputdata/atm/datm7/anomaly_forcing/af.vas.CESM.SSP245.2015-2100.nc 2
(shr_dmodel_readstrm) file ub: /work/n02/n02/jking/cesm/CESM2.1.3/cesm_inputdata/atm/datm7/anomaly_forcing/af.rsds.CESM.SSP245.2015-2100.nc 2
(shr_dmodel_readstrm) file ub: /work/n02/n02/jking/cesm/CESM2.1.3/cesm_inputdata/atm/datm7/anomaly_forcing/af.rlds.CESM.SSP245.2015-2100.nc 2
(shr_dmodel_readstrm) file ub: /work/n02/n02/jking/cesm/CESM2.1.3/cesm_inputdata/atm/datm7/anomaly_forcing/af.pr.CESM.SSP245.2015-2100.nc 2
(shr_dmodel_readstrm) file ub: /work/n02/n02/jking/cesm/CESM2.1.3/cesm_inputdata/atm/datm7/anomaly_forcing/af.tas.CESM.SSP245.2015-2100.nc 2
(shr_dmodel_readstrm) file ub: /work/n02/n02/jking/cesm/CESM2.1.3/cesm_inputdata/atm/datm7/anomaly_forcing/af.ps.CESM.SSP245.2015-2100.nc 2
(shr_dmodel_readstrm) file ub: /work/n02/n02/jking/cesm/CESM2.1.3/cesm_inputdata/atm/datm7/anomaly_forcing/af.huss.CESM.SSP245.2015-2100.nc 2
(shr_dmodel_readstrm) file ub: /work/n02/n02/jking/cesm/CESM2.1.3/cesm_inputdata/atm/datm7/anomaly_forcing/af.uas.CESM.SSP245.2015-2100.nc 2
(shr_dmodel_readstrm) file ub: /work/n02/n02/jking/cesm/CESM2.1.3/cesm_inputdata/atm/datm7/anomaly_forcing/af.vas.CESM.SSP245.2015-2100.nc 2
(shr_dmodel_readstrm) file ub: /work/n02/n02/jking/cesm/CESM2.1.3/cesm_inputdata/atm/datm7/anomaly_forcing/af.rsds.CESM.SSP245.2015-2100.nc 2
(shr_dmodel_readstrm) file ub: /work/n02/n02/jking/cesm/CESM2.1.3/cesm_inputdata/atm/datm7/anomaly_forcing/af.rlds.CESM.SSP245.2015-2100.nc 2
(datm_comp_run) atm: model date 20150116 1800s
(datm_comp_run) atm: model date 20150116 3600s
(datm_comp_run) atm: model date 20150116 5400s

Does this look like a fix, or have I broken something else?

James
 
O

oleson

Keith Oleson
CSEG and Liaisons
Staff member
Looking at the differences between the model version where this worked and the version where it didn't, one thing I see that's different is in cime/scripts/lib/CIME/nmlgen.py. Here, the "<" indicates the older version and ">" indicates the new version:

289,290c289,297
< """Get a list of all streams used for the current data model mode."""
< return self.get_default("streamslist")
---
> """Get a list of all streams used for the current data model mode, possibly modified in user_nl_dxxx."""
> streams = self.get_value("streams")
> if streams[0]:
> streams[:] = [text.split(" ",1)[0] for text in streams]
> streams[:] = [text.split("txt.",1)[1] for text in streams]
> else:
> streams = self.get_default("streamslist")
>
> return streams

I'm not exactly sure why that code has changed or what exactly the new code does really, but if I revert back to the "<" version, the anomaly forcing streams no longer appear to contain both the 2006 streams and the 2015 streams. In other words it looks like this:

streams = "datm.streams.txt.CLMGSWP3v1.Solar 2001 2001 2013",
"datm.streams.txt.CLMGSWP3v1.Precip 2001 2001 2013",
"datm.streams.txt.CLMGSWP3v1.TPQW 2001 2001 2013",
"datm.streams.txt.presaero.SSP2-4.5 2015 2015 2101",
"datm.streams.txt.topo.observed 1 1 1",
"datm.streams.txt.co2tseries.SSP2-4.5 2015 2015 2500",
"datm.streams.txt.Anomaly.Forcing.Precip 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Temperature 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Pressure 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Humidity 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Uwind 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Vwind 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Shortwave 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Longwave 2015 2015 2100"


which I think is want you want.

I'll file an issue on this but you can try it and see if you get the same results. If you do, I'd just make sure that the model is picking up any and all other changes you are making via user_nl_*.
 
J

James King

James King
Member
Hi Keith,

Thanks for the update. I've made the change to my version of the source code and the DATM namelist now looks as expected, as do the other namelists I've modified. Hopefully this should now run OK, thanks for your help. I did find that this problem doesn't occur in the other version of CESM2 I have ported to this machine (CESM2.2.2).

James
 
D

Dong Peng

Dong Peng
Member
@oleson and @erik
Hi, I download the anomaly forcing in the ../anomaly forcing/CMIP6-SSP5-8.5/af.allvars.CESM.SSP5-8.5.2015-2100_c20220628.nc But the files cannot be opened by any software like ncview and Panoply. Then I used the user.datm.xxxx to add to datm.in but the case cannot run well because of the NC file cannot be read. Could you please help check the files? Besides, can I have the similar files for SSP534?

log in cesm.log
Error
(shr_stream_getCalendar) ERROR: nf90_open file /data4/home/dpeng/CESM/cesm2.1.
3/inputdata/atm/datm7/anomaly_forcing/CMIP6-SSP5-8.5/af.allvars.CESM.SSP5-8.5.2
015-2100_c20220628.nc
 
O

oleson

Keith Oleson
CSEG and Liaisons
Staff member
I can ncview the files here. Maybe you have an incomplete download, try again. ls -l yields the following file size.

-rw-r--r-- 1 erik cgdtss 3653316048 Jun 28 2022 af.allvars.CESM.SSP5-8.5.2015-2100_c20220628.nc

We don't have files for SSP534.
 
D

Dong Peng

Dong Peng
Member
oleson said:
-I can ncview the files here. Maybe you have an incomplete download, try again. ls -l yields the following file size.

-rw-r--r-- 1 erik cgdtss 3653316048 Jun 28 2022 af.allvars.CESM.SSP5-8.5.2015-2100_c20220628.nc

We don't have files for SSP534.
Click to expand...
Thanks， Oleson. I tried to download the file with the ./check_input_data --download tool, but it seems cannot get the files properly.

Here it's the log of the download file:
-rw-rw-r-- 1 dpeng ebmg 3653316048 Jun 26 09:00 af.allvars.CESM.SSP5-8.5.2015-2100_c20220628.nc

ncview: can't recognize format of input file /data4/home/dpeng/inputdata/atm/datm7/anomaly_forcing/CMIP6-SSP5-8.5/af.allvars.CESM.SSP5-8.5.2015-2100_c20220628.nc

Panoply: There was an error opening the dataset: The file does not seem to be valid CDM.

Check_input_data --download log
Model datm missing file domain1 = '/data4/home/dpeng/CESM/cesm2.1.3/inputdata/atm/datm7/anomaly_forcing/CMIP6-SSP5-8.5/af.allvars.CESM.SSP5-8.5.2015-2100_c20220628.nc'
Trying to download file: 'atm/datm7/anomaly_forcing/CMIP6-SSP5-8.5/af.allvars.CESM.SSP5-8.5.2015-2100_c20220628.nc' to path '/data4/home/dpeng/CESM/cesm2.1.3/inputdata/atm/datm7/anomaly_forcing/CMIP6-SSP5-8.5/af.allvars.CESM.SSP5-8.5.2015-2100_c20220628.nc' using WGET protocol.
SUCCESS

Did not find hash for file atm/datm7/anomaly_forcing/CMIP6-SSP5-8.5/af.allvars.CESM.SSP5-8.5.2015-2100_c20220628.nc in chksum file /data4/home/dpeng/waterloo/cesm2.1.3/outputdata/Future_2020-2030_SSP585_AF85/run/inputdata_checksum.dat
Loading input file list: 'Buildconf/clm.input_data_list'
Loading input file list: 'Buildconf/mosart.input_data_list'
Loading input file list: 'Buildconf/cism.input_data_list'
Loading input file list: 'Buildconf/cpl.input_data_list'
 
O

oleson

Keith Oleson
CSEG and Liaisons
Staff member
I'm out of ideas at this point sorry. I downloaded it myself and it was fine. I'll ask others if they have any ideas. @slevis
 
You must log in or register to reply here.
Top