How to run CTSM5.2 in CPLHIST mode?

[KeerZ]

Hi all,

I am trying to run a simulation in the ISSP585 CPLHIST mode (compset long name: SSP585_DATM%CPLHIST_CLM50%SP_SICE_SOCN_MOSART_SGLC_SWAV) with CTSM5.2. I noticed that the coupler history files for CTSM have new NetCDF names (e.g. **.cpl.hx.atm.3h.avrg.**.nc instead of **.cpl.ha2x3h.**.nc) and variable names, and the dimensions for each variable are also different from the old coupler history files.

This thread mentions that although the format of the coupler history files has changed for CTSM, it is still possible to use the old coupler history files as long as we rename the files and variables to match the new format for CTSM. The coupler history files I'd like to use are from case "b.e21.BSSP585cmip6.f09_g17.CMIP6-SSP5-8.5.102" and I have both monthly and daily files (at /glade/derecho/scratch/keerzhang/archive/SSP585cpl/b.e21.BSSP585cmip6.f09_g17.CMIP6-SSP5-8.5.102/cpl/).

I've tried these:

1) change the datm.streams.xml to match the old file names and variable names of coupler history files. See this case: /glade/u/home/keerzhang/CTSM5.2.005EAC_3/cime/scripts/TestCase/ISSP585Sp.ctsm52.ChangeStream

This case failed with an error message in the cesm log:".../bin/bash: module: line 1: syntax error: unexpected end of file
/bin/bash: error importing function definition for `module'
dec2107.hsn.de.hpc.ucar.edu 1280: /bin/bash: module: line 1: syntax error: unexpected end of file
dec2107.hsn.de.hpc.ucar.edu 1280: /bin/bash: error importing function definition for `module'
dec2107.hsn.de.hpc.ucar.edu 1281: /bin/bash: module: line 1: syntax error: unexpected end of file..." (log files: /glade/derecho/scratch/keerzhang/ISSP585Sp.ctsm52.ChangeStream/run)

2) change the names of the old cplhist files and variables to match the new format for CTSM. See this case:
/glade/u/home/keerzhang/CTSM5.2.005EAC_3/cime/scripts/TestCase/ISSP585Sp.ctsm52.ChangeCplhistFiles.

This case failed with an error message in the cesm log:"....dec2062.hsn.de.hpc.ucar.edu 0: Abort with message Specified netCDF file does not exist. in file /glade/derecho/scratch/csgteam/temp/spack/derecho/23.09/builds/spack-stage-parallelio-2.6.2-zyhuubh2c6tdzo3o3zugo55lv6atxzzv/spack-src/src/clib/pioc_support.c at line 2843
dec2062.hsn.de.hpc.ucar.edu 1: Abort with message Specified netCDF file does not exist. in file /glade/derecho/scratch/csgteam/temp/spack/derecho/23.09/builds/spack-stage-parallelio-2.6.2-zyhuubh2c6tdzo3o3zugo55lv6atxzzv/spack-src/src/clib/pioc_support.c at line 2843..." (log files: /glade/derecho/scratch/keerzhang/ISSP585Sp.ctsm52.ChangeCplhistFiles/run)

For both test cases, I set flds_presndep = .false. since the old coupler history files do not have any variables for "NOx nad NHx deposition". I am attaching my setup details of these two cases. I wonder if running ISSP585 CPLHIST simulations is feasible with CTSM5.2. If yes, can anyone share some experience or successful cases with me?

Thanks a lot!!

 

[oleson]

I took a quick glance at this. Since it appears to still be reading in the old coupler history files for ndep, e.g.,

(shr_stream_getCalendar) opening stream filename = /glade/derecho/scratch/keerzhang/archive/SSP585cpl/b.e21.BSSP585cmip6.f09_g17.CMIP6-SSP5-8.5.102/cpl/Test/b.e21.BSSP585cmip6.f09_g17.CMIP6-SSP5-8.5.102.cpl.ha2x1d.2095-01.nc

I wonder if setting DATM_PRESNDEP to SSP5-8.5 instead of cplhist would help.

 

[KeerZ]

I took a quick glance at this. Since it appears to still be reading in the old coupler history files for ndep, e.g.,

(shr_stream_getCalendar) opening stream filename = /glade/derecho/scratch/keerzhang/archive/SSP585cpl/b.e21.BSSP585cmip6.f09_g17.CMIP6-SSP5-8.5.102/cpl/Test/b.e21.BSSP585cmip6.f09_g17.CMIP6-SSP5-8.5.102.cpl.ha2x1d.2095-01.nc

I wonder if setting DATM_PRESNDEP to SSP5-8.5 instead of cplhist would help.

This works for both test cases! Thank you, Keith!

It is easier to modify datm.streams.xml to match the old file names and variables, so I will use option 1)

I noticed that the offset value for the "CPLHISTForcing.Solar" stream is -900 for the new cplhist files for CTSM (see /glade/u/home/keerzhang/CTSM5.2.005EAC_3/cime/scripts/TestCase/ISSP585Sp.ctsm52.ChangeCplhistFiles_2/CaseDocs/datm.streams.xml), but was 2700 for the old cplhist files for CESM2.1 (see /glade/u/home/keerzhang/CESM2.1.4_DC/cime/scripts/ModEmi/SSP585/ISSP585Sp.f09_g17.ModE/CaseDocs/datm.streams.txt.CPLHISTForcing.Solar).

I am unsure if this offset value is really used by the model (see this document for a very old version of DATM model). If it is indeed used, should I change the offset value for "CPLHISTForcing.Solar" stream in datm.streams.xml to 2700 because I am using old cplhist files?

 

[KeerZ]

A potentially useful note:

For option 2, I initially only prepared five-day cplhist files in the new format and intended to run the test case for only three days. I also modified the datm.streams.xml to only include five-day forcing files and thought this should be fine.

However, the case keeps failing with this error message:"dec0082.hsn.de.hpc.ucar.edu 3: (shr_strdata_advance) ERROR: for stream 1 and calendar NO_LEAP
dec0082.hsn.de.hpc.ucar.edu 3: (shr_strdata_advance) ERROR: dtime, dtmax, dtmin, dtlimit =
dec0082.hsn.de.hpc.ucar.edu 3: 3.125000000000000E-002 361.052083333333 3.125000000000000E-002
dec0082.hsn.de.hpc.ucar.edu 3: 3.00000000000000
dec0082.hsn.de.hpc.ucar.edu 3: (shr_strdata_advance) ERROR: ymdLB, todLB, ymdUB, todUB = 20950101
dec0082.hsn.de.hpc.ucar.edu 3: 1800 20950101 4500
dec0082.hsn.de.hpc.ucar.edu 3: ERROR: (shr_strdata_advance) ERROR dt limit for stream, see atm.log output"

The reason for the error seems to be that the model needs to have the cplhist files for the whole year 2095 because I have set DATM_YR_START=2095,DATM_YR_END=2095.

After I prepared all cplhist files in 2095 and specified them in the datm.streams.xml, the problem was solved.

 

[oleson]

Thanks for the note, that was useful, I didn't know that.
I don't have any experience with using old cplhist files with the new CPLHIST mode, so I'm not sure what the answer is. I think one way to determine this, since this is with respect to the solar files, is to output model history at half-hourly time step for a few days and check that the incoming solar, reflected solar, are lined up properly with the model's solar zenith angle. In other words, the daily cycle of incoming and reflected radiation are reasonable.

 

[KeerZ]

Thanks for the note, that was useful, I didn't know that.
I don't have any experience with using old cplhist files with the new CPLHIST mode, so I'm not sure what the answer is. I think one way to determine this, since this is with respect to the solar files, is to output model history at half-hourly time step for a few days and check that the incoming solar, reflected solar, are lined up properly with the model's solar zenith angle. In other words, the daily cycle of incoming and reflected radiation are reasonable.

Got it. Thanks. I will do some tests and post the results!

 

[KeerZ]

Thanks for the note, that was useful, I didn't know that.
I don't have any experience with using old cplhist files with the new CPLHIST mode, so I'm not sure what the answer is. I think one way to determine this, since this is with respect to the solar files, is to output model history at half-hourly time step for a few days and check that the incoming solar, reflected solar, are lined up properly with the model's solar zenith angle. In other words, the daily cycle of incoming and reflected radiation are reasonable.

Here are my test results:
I ran three test simulations. Case 1 uses CESM2.1.4 in the cplhist mode, and the default offset for the solar stream is 2700. Case 2 uses CTSM5.2 in the cplhist mode, and the default offset for the solar stream is -900. Case 3 uses CTSM5.2 in cplhist mode, and I changed the offset for the solar stream to be 2700.

I ran them for three days and output the FSDS and solar zenith angle (COSZEN) at each time step. All of them are driven by the same atm data from case "b.e21.BSSP585cmip6.f09_g17.CMIP6-SSP5-8.5.102". I plotted the time series of FSDS/600 (I scaled it to have the same magnitude as COSZEN) and COSZEN in a daily cycle to see if their values match. It seems that the FSDS for Case 2 is a little too early, and the FSDS and COSZEN match better in Cases 1 and 3 with an offset of 2700. For Cases 1 and 3, the FSDS values at the same time step are the same. Therefore, I feel that setting the offset value to 2700 makes more sense when we are using the old version of cplhist files.


However, the COSZEN in Cases 2 and 3 which use CTSM can be the same for two timesteps, as denoted by the zigzag curves. I am curious, is this behavior expected?

 

[oleson]

I think that in the zigzag case radiation is being calculated every hour, while in the smooth case radiation is calculated every time-step. This is controlled by a namelist item in datm_in, iradsw. A value of 1 means calculate radiation every time step, while a value of -1 means calculate radiation every hour. You could check the values in your cases to confirm.