Guidance on Running with LME datasets with CESM2.1.3 (CLM5)

wvsi3w

wvsi3w
Member
I doubt there is a problem with the finidat file since that file would be interpolated to your new grid/domain at the beginning of the simulation and your simulation runs fine for a number of years.
One question I have is why do you have the anomaly forcing repeated in the file above, i.e.,

"datm.streams.txt.Anomaly.Forcing.Precip 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Temperature 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Pressure 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Humidity 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Uwind 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Vwind 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Shortwave 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Longwave 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Precip 2006 2006 2300",
"datm.streams.txt.Anomaly.Forcing.Temperature 2006 2006 2300",
"datm.streams.txt.Anomaly.Forcing.Pressure 2006 2006 2300",
"datm.streams.txt.Anomaly.Forcing.Humidity 2006 2006 2300",
"datm.streams.txt.Anomaly.Forcing.Uwind 2006 2006 2300",
"datm.streams.txt.Anomaly.Forcing.Vwind 2006 2006 2300",
"datm.streams.txt.Anomaly.Forcing.Shortwave 2006 2006 2300",
"datm.streams.txt.Anomaly.Forcing.Longwave 2006 2006 2300"

Also, I'd check the precipitation anomaly forcing file to make sure it has valid data throughout time, could it be corrupted from a bad download?
Yes that was odd for me. At first, I thought this could be normal.

Also, this is my user_nl_datm file:

Code:
anomaly_forcing = "Anomaly.Forcing.Precip","Anomaly.Forcing.Temperature","Anomaly.Forcing.Pressure","Anomaly.Forcing.Humidity","Anomaly.Forcing.Uwind","Anomal
y.Forcing.Vwind","Anomaly.Forcing.Shortwave","Anomaly.Forcing.Longwave"

fillalgo = "nn", "nn", "nn", "nn", "nn", "nn", "nn", "nn", "nn", "nn", "nn", "nn", "nn"

streams = "datm.streams.txt.CLMGSWP3v1.Solar 2000 2000 2014",
"datm.streams.txt.CLMGSWP3v1.Precip 2000 2000 2014",
"datm.streams.txt.CLMGSWP3v1.TPQW 2000 2000 2014",
"datm.streams.txt.presaero.SSP2-4.5 2015 2015 2101",
"datm.streams.txt.topo.observed 1 1 1",
"datm.streams.txt.co2tseries.SSP2-4.5 2015 2015 2500",
"datm.streams.txt.Anomaly.Forcing.Precip 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Temperature 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Pressure 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Humidity 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Uwind 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Vwind 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Shortwave 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Longwave 2015 2015 2100"

As you see, the anomaly list is not repeated in my namelist. I don't know why datm_in is showing a repeated list, also the repeated part is showing a different range (2006-2300) while my namelist was from 2015-2100!!!

Do you suggest removing these repeated lines manually and starting again from the beginning?

About the "precipitation anomaly forcing file" I checked the "pr" variable in both files (af.allvars.CESM.SSP1-2.6.2015-2100_c20220628.nc & af.allvars.CESM.SSP2-4.5.2015-2100_c20220628.nc), and they are valid.
 

oleson

Keith Oleson
CSEG and Liaisons
Staff member
I don't know why the repeated lines are being generated in your resolved datm_in. You are still usihg release-cesm2.1.3?
What domain file are you using for the anomaly forcing, e.g., in your datm.streams.txt.Anomaly.Forcing.Precip? It should be domain.b.e21.BHIST.f09_g17.CMIP6-historical.010.c200203.nc
 

wvsi3w

wvsi3w
Member
I don't know why the repeated lines are being generated in your resolved datm_in. You are still usihg release-cesm2.1.3?
What domain file are you using for the anomaly forcing, e.g., in your datm.streams.txt.Anomaly.Forcing.Precip? It should be domain.b.e21.BHIST.f09_g17.CMIP6-historical.010.c200203.nc
Yes I am still using 2.1.3 because my second objective hasn't yet finished.

That is odd because the anomaly forcing files initially had this domain (domain.permafrostRCN_P2.c2013.nc) written in them so I thought I shouldnt change it:

e.g. datm.streams.txt.Anomaly.Forcing.Precip :
Code:
<?xml version="1.0"?>
<file id="stream" version="1.0">
<dataSource>
   GENERIC
</dataSource>
<domainInfo>
  <variableNames>
     time    time
        xc      lon
        yc      lat
        area    area
        mask    mask
  </variableNames>
  <filePath>
 /scratch/USER/cesm/inputdata/atm/datm7/anomaly
  </filePath>
  <fileNames>
     domain.permafrostRCN_P2.c2013.nc
  </fileNames>
</domainInfo>
<fieldInfo>
   <variableNames>
     pr  prec_af
   </variableNames>
   <filePath>
 /scratch/USER/cesm/inputdata/atm/datm7/anomaly
   </filePath>
   <fileNames>
    af.allvars.CESM.SSP1-2.6.2015-2100_c20220628.nc
   </fileNames>
   <offset>
      0
   </offset>
</fieldInfo>
</file>

I will change it to what you mentioned and let you know how it goes.
 

wvsi3w

wvsi3w
Member
Dear Keith,
I have tried downloading the domain (domain.b.e21.BHIST.f09_g17.CMIP6-historical.010.c200203.nc), and it fails to find it:
Code:
svn export failed with output:  and errput svn: E170000: URL 'https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata/atm/datm7/anomaly_forcing/domain.b.e21.BHIST.f09_g17.CMIP6-historical.010.c200203.nc' doesn't exist

Checking server None with protocol None
Setting resource.RLIMIT_STACK to 300000000 from (300000000, -1)
Client protocol None not enabled
ERROR: Could not find all inputdata on any server

To get it manually with wget, I also couldn't find the domain file online (is not in this link either: - Revision 70792: /trunk/inputdata/share/domains )
 

oleson

Keith Oleson
CSEG and Liaisons
Staff member
Well, the domain file you are using should work as well, since it has a mask of 1 everywhere on the grid and there is precip anomaly forcing everywhere.
I guess I would check your atm log file and see what atmospheric (not anomaly) forcing is being read in and then check the precip field on that forcing file. Since the model is failing at the same point no matter what the SSP is, it seems like that might point to a problem with the forcing.
 

wvsi3w

wvsi3w
Member
Well, the domain file you are using should work as well, since it has a mask of 1 everywhere on the grid and there is precip anomaly forcing everywhere.
I guess I would check your atm log file and see what atmospheric (not anomaly) forcing is being read in and then check the precip field on that forcing file. Since the model is failing at the same point no matter what the SSP is, it seems like that might point to a problem with the forcing.
Thank you
It was indeed one of the files from Precip. I checked the atm log file, and it ended in these:
Code:
/scratch/USER/cesm/inputdata/atm/datm7/atm_forcing.datm7.GSWP3.0.5d.v1.c170516/Precip/clmforc.GSWP3.c2011.0.5x0.5.Prec.2003-09.nc     169
(shr_dmodel_readstrm) file ub: /scratch/USER/cesm/inputdata/atm/datm7/atm_forcing.datm7.GSWP3.0.5d.v1.c170516/TPHWL/clmforc.GSWP3.c2011.0.5x0.5.TPQWL.2003-09.nc     169
(datm_comp_run) atm: model date 20330921   82800s
(datm_comp_run) atm: model date 20330921   84600s
(datm_comp_run) atm: model date 20330922       0s
(shr_dmodel_readstrm) file ub: /scratch/USER/cesm/inputdata/atm/datm7/atm_forcing.datm7.GSWP3.0.5d.v1.c170516/Solar/clmforc.GSWP3.c2011.0.5x0.5.Solr.2003-09.nc     170
(datm_comp_run) atm: model date 20330922    1800s

So I checked the 2003-09 Precip file with ncview, and it was indeed damaged in the middle. removed it and downloaded it again and continued the run, but again another file was broken (2004-01), so I did the same thing, and it is running now.

Do you know what we should do if this happens a lot? I mean, is there a way to check the forcing files when we download them with some command or some script to see if they are all ok and not damaged?
 

oleson

Keith Oleson
CSEG and Liaisons
Staff member
I'm not sure, maybe something like: ./check_input_data --download --debug, or ./check_input_data --download --chksum?
 

wvsi3w

wvsi3w
Member
I'm not sure, maybe something like: ./check_input_data --download --debug, or ./check_input_data --download --chksum?
Thanks for your help.
I have found another issue which is weird:

Code:
 (shr_strdata_advance) ERROR: for stream           15
 (shr_strdata_advance) ERROR: dt limit1    76316.0000000000
   28.0000000000000        1.50000000000000
 (shr_strdata_advance) ERROR: dt limit2     20150116           0    20150213
           0
 ERROR: (shr_strdata_advance) ERROR dt limit for stream

If you remember last time, I had an error regarding the dtlimit (Guidance on Running with LME datasets with CESM2.1.3 (CLM5)), and you suggested using 1.e36 to get around it.

This time, instead of starting from 2020, I was trying to start my SSP run from 2015, and I faced the same error! This didn't happen when I started with 2020 restart file.

I am using the restart file from that simulation where dtlimit in its user_nl_datm was set to 1.e36 (from 2000-2015).

Prior to this, I was starting from 2020-01-01, and the restart file from 2020-01-01 is from a simulation in which the dtlimit was also set to 1.e36 in the datm user namelist file, but my SSP run (from 2020 onwards) didn't have any problem and is running now.

I only see this error when I am starting from earlier (like 2015).

Do you think this occurs because the 2015-2020 simulation was forced with a SSP and the 2000-2015 simulation was not? (In both, I did set dtlimit to 1.e36, I don't know what is wrong here)

I think I should set dtlimit to 1.e36 if I want to run from 2015-2100 using the restart file from 2015 (?)
 

wvsi3w

wvsi3w
Member
Thanks for your help.
I have found another issue which is weird:

Code:
 (shr_strdata_advance) ERROR: for stream           15
 (shr_strdata_advance) ERROR: dt limit1    76316.0000000000
   28.0000000000000        1.50000000000000
 (shr_strdata_advance) ERROR: dt limit2     20150116           0    20150213
           0
 ERROR: (shr_strdata_advance) ERROR dt limit for stream

If you remember last time, I had an error regarding the dtlimit (Guidance on Running with LME datasets with CESM2.1.3 (CLM5)), and you suggested using 1.e36 to get around it.

This time, instead of starting from 2020, I was trying to start my SSP run from 2015, and I faced the same error! This didn't happen when I started with 2020 restart file.

I am using the restart file from that simulation where dtlimit in its user_nl_datm was set to 1.e36 (from 2000-2015).

Prior to this, I was starting from 2020-01-01, and the restart file from 2020-01-01 is from a simulation in which the dtlimit was also set to 1.e36 in the datm user namelist file, but my SSP run (from 2020 onwards) didn't have any problem and is running now.

I only see this error when I am starting from earlier (like 2015).

Do you think this occurs because the 2015-2020 simulation was forced with a SSP and the 2000-2015 simulation was not? (In both, I did set dtlimit to 1.e36, I don't know what is wrong here)

I think I should set dtlimit to 1.e36 if I want to run from 2015-2100 using the restart file from 2015 (?)
I did set the dtlimit in user_nl_datm to 1.e36 for a case that starts from 2015 (for all of the 13 streams), and it failed with the same error!

I guess I have to start from 2020 :(
 

wvsi3w

wvsi3w
Member
Hello all,
I have done some simulations with SSP scenarios in the past few months, and all of them ran successfully; however, there is a problem:
When you start the SSPs (2015-2100), between 2015 to 2020, a sudden increase in temperature appears, which makes the time-series jump a lot in that initial five years of future simulations. This appears to be related to several issues, one of which is that SSPs read DATM files (in my case 1995-2014 from GSWP3 files) while my simulations from 500 CE to 2015 was using different boundary conditions (as mentioned in this thread these DATM files were monthly resolution 1deg, which required me to do some modifications to the domain and other files in order to run the model with them successfully). So there is a big difference between the default DATM that model reads (GSWP3 in here) and the DATM that I had up until 2015, and this could explain the big jump in temperature time-series.

To resolve this issue, I started the following case, in which I tried to change the DATM so that the anomaly forcing would be applied to my boundary conditions (and not the GSWP3 files):

Code:
cime/scripts/create_newcase --case /MYCASES/FUTURE/DATMchanged_SSP245_Mem1_Deep --compset ISSP245Clm50BgcCrop --res f09_g17 --run-unsupported

cd /MYCASES/FUTURE/DATMchanged_SSP245_Mem1_Deep
./case.setup

./xmlchange STOP_OPTION=nyears,STOP_N=5,JOB_WALLCLOCK_TIME=03:59:00,RUN_STARTDATE=2015-01-01,STOP_DATE=21000101,RESUBMIT=16,DATM_CLMNCEP_YR_END=2014,DATM_CLMNCEP_YR_START=1995,DATM_CLMNCEP_YR_ALIGN=1995

# cp clm2.r file and rpointer to case directory

emacs user_nl_clm
use_init_interp = .true.
soil_layerstruct= '23SL_3.5m_D500'
finidat ='/MYCASES/FUTURE/DATAMchanged_SSP245_Mem1_Deep/DeepMem1_2000to2015_IHist.clm2.r.2015-01-01-00000.nc'

emacs user_nl_datm

anomaly_forcing = "Anomaly.Forcing.Precip","Anomaly.Forcing.Temperature","Anomaly.Forcing.Pressure","Anomaly.Forcing.Humidity","Anomaly.Forcing.Uwind","Anomaly.Forcing.Vwind","Anomaly.Forcing.Shortwave","Anomaly.Forcing.Longwave"

&datm_nml
  decomp = "1d"
  factorfn = "null"
  force_prognostic_true = .false.
  iradsw = 1
  presaero = .true.
  restfilm = "undefined"
  restfils = "undefined"
  wiso_datm = .false.
/
&shr_strdata_nml
  datamode = "CLMNCEP"
  domainfile = "/share/domains/domain.lnd.fv0.9x1.25_gx1v7.151020.nc"
  dtlimit = 1.e36, 1.e36, 1.e36, 1.e36, 1.e36, 1.e36, 1.e36, 1.e36, 1.e36, 1.e36, 1.e36, 1.e36, 1.e36, 1.e36
  fillalgo = "nn", "nn", "nn", "nn", "nn", "nn", "nn", "nn", "nn", "nn", "nn", "nn", "nn", "nn"
  fillmask = "nomask", "nomask", "nomask", "nomask", "nomask", "nomask", "nomask", "nomask", "nomask", "nomask", "nomask", "nomask", "nomask", "nomask"
  fillread = "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET"
  fillwrite = "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET"
  mapalgo = "bilinear", "bilinear", "bilinear", "bilinear", "bilinear", "nn", "bilinear", "bilinear", "bilinear", "bilinear", "bilinear", "bilinear", "bilinear", "bilinear"
  mapmask = "nomask", "nomask", "nomask", "nomask", "nomask", "nomask", "nomask", "nomask", "nomask", "nomask", "nomask", "nomask", "nomask", "nomask"
  mapread = "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET"
  mapwrite = "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET"
  readmode = "single", "single", "single", "single", "single", "single", "single", "single", "single", "single", "single", "single", "single", "single"
streams = "datm.streams.txt.CLMGSWP3v1.Solar 1995 1995 2014",
"datm.streams.txt.CLMGSWP3v1.Precip 1995 1995 2014",
"datm.streams.txt.CLMGSWP3v1.TPQW 1995 1995 2014",
"datm.streams.txt.presaero.SSP2-4.5 2015 2015 2101",
"datm.streams.txt.topo.observed 1 1 1",
"datm.streams.txt.co2tseries.SSP2-4.5 2015 2015 2500",
"datm.streams.txt.Anomaly.Forcing.Precip 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Temperature 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Pressure 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Humidity 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Uwind 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Vwind 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Shortwave 2015 2015 2100",
"datm.streams.txt.Anomaly.Forcing.Longwave 2015 2015 2100"
  taxmode = "cycle", "cycle", "cycle", "cycle", "cycle", "extend",  "cycle", "cycle", "cycle", "cycle", "cycle", "cycle", "cycle", "cycle"
  tintalgo = "coszen", "nearest", "linear", "linear", "lower", "linear", "nearest",  "linear", "linear",  "linear", "linear",  "linear", "coszen", "coszen"
  vectors = "null"
/

# COPY THE "USER_DATM.STREAMS.TXT.ANOMALY.FORCING" FILES FROM PREVIOUS CASE DIRECTORY AND PASTE IT INTO YOUR NEW CASE DIRECTORY

# emacs existing user_datm.stream*
# change the domain path and name to the domain file that is adjusted to my DATAM files:

  <filePath>
     /SHARED_FOLDER/mem1/1deg
  </filePath>
  <fileNames>
     domain.lnd.fv0.9x1.25_gx1v7.151020.nc
  </fileNames>

# cd my DATM path for Precip, Solar, TPWHL and rename the years 1995-2014 to "clmforc.GSWP3.c2011.0.5x0.5." format. and copy them into the GSWP3 directory

./case.build
./case.submit

But this job failed with the following error:

Code:
(shr_dmodel_readstrm) ERROR in data sizes      288     192       1  259200
 ERROR: (shr_dmodel_readstrm) ERROR in data sizes
.
.
Stack trace terminated abnormally.
[rc13020:75919:0:75919] Caught signal 11 (Segmentation fault: Sent by the kernel at address (nil))
Image              PC                Routine            Line        Source
.
.
Stack trace terminated abnormally.
[rc13020:75921:0:77991] Caught signal 11 (Segmentation fault: Sent by the kernel at address (nil))
Image              PC                Routine            Line        Source
.
.
Stack trace terminated abnormally.
[rc13020:75920:0:75920] Caught signal 11 (Segmentation fault: address not mapped to object at address (nil))
==== backtrace (tid:  75920) ====
 0 0x0000000000038790 __sigaction()  ???:0
.
.
[rc13020:76028:0:76028] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x1)
[rc13020:76028:1:78135] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x10b)


I assume this is because the SSP anomaly files are not compatible with my datm files. Do you know how to resolve this? I genuinely need to use my own datm files from 1995-2014 for the SSP anomaly to be applied on them. How do I make the anomaly file (or maybe even its domain file) compatible with my datm files?

Thank you for your help and support in advance.
 

wvsi3w

wvsi3w
Member
the key part of the error is "288 192 1 259200" which is saying my 1 deg datm files (192 x 288 = 55296) is not equal to the 0.5 deg resolution GSWP3 files (259200 = 720 × 360). This created that incompatibility I guess.

here I don't know if I need to modify my datm files to become 0.5 deg (also my domain needs to be modified too), or I need to modify the SSP anomaly and its domain to be 1 deg.

or there is another option you know?
 

wvsi3w

wvsi3w
Member
An update on this issue:
I re-grid the datm files I have to be 0.5 deg resolution and it is running now (not sure if it finishes without issue or not).

Also in the user_nl_datm file I made a mistake that tintalgo for Longwave should be "linear" and not "coszen".

lets see if it works.
 

oleson

Keith Oleson
CSEG and Liaisons
Staff member
The anomaly forcing is at 1deg and typically applied to 0.5 deg GSWP3 data. So they don't have to be at the same resolution. As long as you use domain files appropriate for each set of forcing. E.g., the typical domain file setting for the GSWP3 data is (in, for example, datm.streams.txt.CLMGSWP3v1.Precip):

<domainInfo>
<variableNames>
time time
xc lon
yc lat
area area
mask mask
</variableNames>
<filePath>
/glade/p/cgd/tss/CTSM_datm_forcing_data/atm_forcing.datm7.GSWP3.0.5d.v1.c170516
</filePath>
<fileNames>
domain.lnd.360x720_gswp3.0v1.c170606.nc
</fileNames>
</domainInfo>

and the typical (depends on your model version) domain file setting for anomaly forcing is (in, for example, datm.streams.txt.Anomaly.Forcing.Humidity):

<domainInfo>
<variableNames>
time time
xc lon
yc lat
area area
mask mask
</variableNames>
<filePath>
/glade/p/cgd/tss/people/oleson/anomaly_forcing
</filePath>
<fileNames>
domain.b.e21.BHIST.f09_g17.CMIP6-historical.010.c200203.nc
</fileNames>
</domainInfo>

One other thing is that all of anomaly forcing should have tintalgo = nearest. Maybe it doesn't matter, but I'm not sure what the datm is doing with your setting of coszen for the solar anomaly. coszen should only be used for your solar forcing data.
 

wvsi3w

wvsi3w
Member
The anomaly forcing is at 1deg and typically applied to 0.5 deg GSWP3 data. So they don't have to be at the same resolution. As long as you use domain files appropriate for each set of forcing. E.g., the typical domain file setting for the GSWP3 data is (in, for example, datm.streams.txt.CLMGSWP3v1.Precip):

<domainInfo>
<variableNames>
time time
xc lon
yc lat
area area
mask mask
</variableNames>
<filePath>
/glade/p/cgd/tss/CTSM_datm_forcing_data/atm_forcing.datm7.GSWP3.0.5d.v1.c170516
</filePath>
<fileNames>
domain.lnd.360x720_gswp3.0v1.c170606.nc
</fileNames>
</domainInfo>

and the typical (depends on your model version) domain file setting for anomaly forcing is (in, for example, datm.streams.txt.Anomaly.Forcing.Humidity):

<domainInfo>
<variableNames>
time time
xc lon
yc lat
area area
mask mask
</variableNames>
<filePath>
/glade/p/cgd/tss/people/oleson/anomaly_forcing
</filePath>
<fileNames>
domain.b.e21.BHIST.f09_g17.CMIP6-historical.010.c200203.nc
</fileNames>
</domainInfo>

One other thing is that all of anomaly forcing should have tintalgo = nearest. Maybe it doesn't matter, but I'm not sure what the datm is doing with your setting of coszen for the solar anomaly. coszen should only be used for your solar forcing data.
Dear Keith,
Thank you so much for the response.

About the "tintalgo = nearest", do you think that would significantly impact my results if I use "linear" instead?
I couldn't find relevant explanation in the user guide link for this question.
 

oleson

Keith Oleson
CSEG and Liaisons
Staff member
The anomaly forcing is a monthly value and I believe you would want to apply the same monthly value to all of the forcing in that month, which is what nearest would do. If you used linear then you would be creating anomalies whose monthly mean would not be the same as the provided monthly value. At least that's how I interpret the choice of nearest.
 

wvsi3w

wvsi3w
Member
Dear Keith,
I have tried it with both linear and nearest, and here are the results for permafrost extent; you can see in the linear case there is a sudden drop in between 2015-2020:

linear.png

While when we use "tintalgo = nearest" for all of the streams, this happens: a sudden, weird jump:

nearest.png

What do you think is creating this issue here? should we change some of the streams tintalgo to something else maybe?

+the only difference between these two sets of cases are the tintalgo.
+the lines in the timeseries represent different permafrost extent definitions.

+ this was my tintalgo for first case (which resulted in sudden drop in 2015-2020):
tintalgo = "coszen", "nearest", "linear", "linear", "lower", "linear", "nearest", "linear", "linear", "linear", "linear", "linear", "coszen", "coszen"

and for my second case (big sudden jump after 2015), all of these were "nearest"
 

oleson

Keith Oleson
CSEG and Liaisons
Staff member
I would start by looking at the forcing itself to diagnose the discontinuity. For example, look in the atm logs to see what year of forcing is being used for the last year of the historical simulation, and what year of forcing is being used for the first year of the SSP, that the anomaly forcing is being applied to. And check that you are using the same forcing datasets for the historical and SSP. And check to see that the correct time slice of the anomaly forcing is being used at the beginning of the SSP simulation.
 

wvsi3w

wvsi3w
Member
I would start by looking at the forcing itself to diagnose the discontinuity. For example, look in the atm logs to see what year of forcing is being used for the last year of the historical simulation, and what year of forcing is being used for the first year of the SSP, that the anomaly forcing is being applied to. And check that you are using the same forcing datasets for the historical and SSP. And check to see that the correct time slice of the anomaly forcing is being used at the beginning of the SSP simulation.
Dear Keith,
Thanks for the helpful response.
I should mention that since all of the simulations are finished, I only have access to gz files for log files, and as you know, these gz files dont give us useful information. If I want to check the atm log file I should start a new SSP simulation and make it fail after a few runs so that I can check that log file.

But I can explain my setup:
The anomaly forcings in SSPs are being applied to my own 1995-2014 datm7 files (Solar, Precip, TPHWL). My own datm7 files are monthly, 1 deg.
The historical simulations (from 1850 to 2014) were prescribed with my own transient datm7 files covering the whole period (1850-2014, monthly, 1 deg). so the start and end of my historical simulations are 1850 and 2014.

+My own datm7 files are coming from another model (our global model) and we use those forcings as boundary conditions, so we already had datm7 from year 500 CE to 2014, we only needed to separate the runs into Pre-industrial and historical, which we did and for historical we used the right CO2 and aerosols too. so when I started the SSPs I eventually used a chunk from 1995-2014 of my own datm7 files.

should I change the start and end of DATM for SSPs?


P.S. I have to mention that I did try changing the start and end of SSPs using nearest tintalgo, in a case I used 1980-1999 (because I thought maybe there are some irregularities in datm7 files from 2000 onwards), but the results didn't change much. big jump appeared again even with 1980-1999 datm7 files.
 

wvsi3w

wvsi3w
Member
Dear Keith,
Thanks for the helpful response.
I should mention that since all of the simulations are finished, I only have access to gz files for log files, and as you know, these gz files dont give us useful information. If I want to check the atm log file I should start a new SSP simulation and make it fail after a few runs so that I can check that log file.

But I can explain my setup:
The anomaly forcings in SSPs are being applied to my own 1995-2014 datm7 files (Solar, Precip, TPHWL). My own datm7 files are monthly, 1 deg.
The historical simulations (from 1850 to 2014) were prescribed with my own transient datm7 files covering the whole period (1850-2014, monthly, 1 deg). so the start and end of my historical simulations are 1850 and 2014.

+My own datm7 files are coming from another model (our global model) and we use those forcings as boundary conditions, so we already had datm7 from year 500 CE to 2014, we only needed to separate the runs into Pre-industrial and historical, which we did and for historical we used the right CO2 and aerosols too. so when I started the SSPs I eventually used a chunk from 1995-2014 of my own datm7 files.

should I change the start and end of DATM for SSPs?


P.S. I have to mention that I did try changing the start and end of SSPs using nearest tintalgo, in a case I used 1980-1999 (because I thought maybe there are some irregularities in datm7 files from 2000 onwards), but the results didn't change much. big jump appeared again even with 1980-1999 datm7 files.
I checked one of the older cases in which I was using 1980-1999 as start and end of my DATM for SSPs, and there is a log file fortunately.
In the atm log file I can see that despite the fact that we assigned the user_datm_streams to go and read our own forcing from that year, it goes and reads default datm7 files.

Code:
(datm_comp_init)  initialize gsmaps
(datm_comp_init) copy domains
(datm_comp_init) allocate AVs
(datm_comp_init)  flds_strm = strm_swdn:strm_precn:strm_tbot:strm_wind:strm_pbot:strm_shum:strm_lwdn:strm_co2diag:strm_prec_af:strm_tbot_af:strm_pbot_af:strm_shum_af:strm_u_af:strm_
v_af:strm_swdn_af:strm_lwdn_af
(datm_comp_init)  restart filenames from rpointer
(datm_comp_init)  reading nearset_SSP245_Mem1_Deep.datm.rs1.2025-01-01-00000.bin
(shr_stream_restRead) case name        : nearset_SSP245_Mem1_Deep
(shr_stream_restRead) case description : SDATM strdata
(shr_stream_restRead) File created     : 2026-07-17 18:08:07
(shr_stream_restRead) Number of streams    14
(shr_stream_restRead) reading time axis for stream restart filename        1       1 clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-01.nc clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-01.nc
(shr_stream_restRead) reading time axis for stream restart filename        1       2 clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-02.nc clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-02.nc
(shr_stream_restRead) reading time axis for stream restart filename        1       3 clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-03.nc clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-03.nc
(shr_stream_restRead) reading time axis for stream restart filename        1       4 clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-04.nc clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-04.nc
(shr_stream_restRead) reading time axis for stream restart filename        1       5 clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-05.nc clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-05.nc
(shr_stream_restRead) reading time axis for stream restart filename        1       6 clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-06.nc clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-06.nc
(shr_stream_restRead) reading time axis for stream restart filename        1       7 clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-07.nc clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-07.nc
(shr_stream_restRead) reading time axis for stream restart filename        1       8 clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-08.nc clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-08.nc
(shr_stream_restRead) reading time axis for stream restart filename        1       9 clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-09.nc clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-09.nc

This is weird. I did define different files and paths in the user_stream files before!!! Somehow the model ignores that and goes straight to the default datm files that are in the scratch directory!!!!

I think I should rename and copy my own datm7 files into scratch so that this time the model reads them (?)
 

wvsi3w

wvsi3w
Member
I checked one of the older cases in which I was using 1980-1999 as start and end of my DATM for SSPs, and there is a log file fortunately.
In the atm log file I can see that despite the fact that we assigned the user_datm_streams to go and read our own forcing from that year, it goes and reads default datm7 files.

Code:
(datm_comp_init)  initialize gsmaps
(datm_comp_init) copy domains
(datm_comp_init) allocate AVs
(datm_comp_init)  flds_strm = strm_swdn:strm_precn:strm_tbot:strm_wind:strm_pbot:strm_shum:strm_lwdn:strm_co2diag:strm_prec_af:strm_tbot_af:strm_pbot_af:strm_shum_af:strm_u_af:strm_
v_af:strm_swdn_af:strm_lwdn_af
(datm_comp_init)  restart filenames from rpointer
(datm_comp_init)  reading nearset_SSP245_Mem1_Deep.datm.rs1.2025-01-01-00000.bin
(shr_stream_restRead) case name        : nearset_SSP245_Mem1_Deep
(shr_stream_restRead) case description : SDATM strdata
(shr_stream_restRead) File created     : 2026-07-17 18:08:07
(shr_stream_restRead) Number of streams    14
(shr_stream_restRead) reading time axis for stream restart filename        1       1 clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-01.nc clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-01.nc
(shr_stream_restRead) reading time axis for stream restart filename        1       2 clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-02.nc clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-02.nc
(shr_stream_restRead) reading time axis for stream restart filename        1       3 clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-03.nc clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-03.nc
(shr_stream_restRead) reading time axis for stream restart filename        1       4 clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-04.nc clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-04.nc
(shr_stream_restRead) reading time axis for stream restart filename        1       5 clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-05.nc clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-05.nc
(shr_stream_restRead) reading time axis for stream restart filename        1       6 clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-06.nc clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-06.nc
(shr_stream_restRead) reading time axis for stream restart filename        1       7 clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-07.nc clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-07.nc
(shr_stream_restRead) reading time axis for stream restart filename        1       8 clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-08.nc clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-08.nc
(shr_stream_restRead) reading time axis for stream restart filename        1       9 clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-09.nc clmforc.GSWP3.c2011.0.5x0.5.Solr.1980-09.nc

This is weird. I did define different files and paths in the user_stream files before!!! Somehow the model ignores that and goes straight to the default datm files that are in the scratch directory!!!!

I think I should rename and copy my own datm7 files into scratch so that this time the model reads them (?)
I am sorry.
Ignore my previous message. This is completely wrong. (The time limit to delete and edit my message was passed).
I remembered that I had renamed and copied my own datm7 files into scratch before. There is nothing wrong with that atm log I presented.

I am sorry for the confusion, I am currently running sevaral simulations I lost track of which was which.
 
Back
Top