Information required in the CLM5 Spinup procedure.

[knreddy]

Hello,
I am using the "release-cesm2.1.1" to run the CLM5BGCCROP model using the following compset: "2000_DATM%GSWP3v1_CLM50%BGC-CROP_SICE_SOCN_SROF_SGLC_SWAV". I plan to explore the crops from the Indian subcontinent, their planting windows, irrigation applied to the crops, and energy, water, and carbon cycles during their growing season. The query I have is to carry on such an experiment do I require to Spin-up the model for over 600 to 1000 years? Or would it be sufficient to run the model for a few extra years than I intend to analyze and discard the initial years as a spinup period?

Another follow-up question is:
I have given a spinup run as directed in the following link "1.5.5. Spinup of CLM5.0-BGC-Crop — ctsm release-clm5.0 documentation".
The steps I followed are:
$ ./create_newcase --case ~/BGC_Spinup_2 --res f09_g17 --compset 1850_DATM%GSWP3v1_CLM50%BGC-CROP_SICE_SOCN_SROF_SGLC_SWAV --machine PADUM --run-unsupported
$ ./xmlchange CLM_ACCELERATED_SPINUP=on
$ ./case.setup.
$ ./case.build --skip-provenance-check
$ changed the RUN_TYPE to "startup"
$ changed the CLM_FORCE_COLDSTART to value="on"
$ ./xmlchange RESUBMIT=3,STOP_N=50,STOP_OPTION=nyears
$ assigned the #PBS -l walltime=120:00:00 and #PBS -l select=8:ncpus=24:mpiprocs=24
$ ./case.submit

The question I have is this the right approach for doing a spin-up? After I complete this run, I will use the pointer files to initialize a spin-up for 400+ years in normal mode.
Am I following the right steps to achieve the goals I have set? Do I need to do anything differently?

Thanks in advance.

 

[oleson]

That compset should come out of the box in release-cesm2.1.1 with an initial file that is spunup at present day for CLM5 BGC CROP. If you are just exploring changes to the model and their impact on energy, water, and carbon cycles then you probably don't need spinups for each configuration you are trying, particularly if you are just looking at turbulent/radiative fluxes, LAI, etc., and not deep soil carbon/nitrogen.

The spinup sequence you propose looks reasonable for conducting an 1850 spinup in preparation for a historical simulation. You'll need to evaluate the model's equilibrium after both the first and second steps. Instead of pointer files, you would specify the initial file in the second step using finidat in user_nl_clm.

If you want to spinup at present day then you would do the same thing but using the 2000_DATM... compset.

 

[knreddy]

Thank you so much @oleson for the information.
This does help me conduct the experiments I intend to carry out. I am interested in the turbulent and radiative fluxes and crop phenology. If a spinup is not required for the simulations, that would save me computational time. Thank you for a quick response.

 

[Han Lubin]

Hello Oleson,

I am also confused about spinup period. I try to simulate global crop yields from 1980 to 2010 using CESM2.2 to run compset I2000Clm50BgcCropGs and res f09_g17. Here is my confusion about the steps.

(1)Instead of spinup, I choose "clmi.I2000Clm50BgcCrop.2011-01-01.1.9x2.5_gx1v7_gl4_simyr2000_c190312.nc" as my initial file and set RUN_STARTDATE=1980-01-01, DATM_CLMNCEP_YR_START=1980, DATM_CLMNCEP_YR_END=2010, DATM_CLMNCEP_YR_ALIGN=1980. I think the initial file I set has spunup by your group ,and I don't need another spinup. Do I run the right step?

(2)Compared to (1), if I want to recycle 30 times form 1960 to 1979 for spinup in I2000Clm50BgcCropGs, then is it reasonable to use the 600-year spunup file as initial file to simulate global crop yields for 1980-2010(I2000)? If true, is the difference between (1) and (2) simply a repetition of the process of spinup that your group have already completed?

(3)Same goal, if I choose I1850Clm50BgcCropGs to spinup 600 years for equilibrium, do I should recycle 30 times from 1901 to 1920 and then set RUN_STARTDATE=1921-01-01, DATM_CLMNCEP_YR_START=1921, DATM_CLMNCEP_YR_END=2010, DATM_CLMNCEP_YR_ALIGN=06010101 to simulate global crop yields for 1980-2010? Which situation would I use IHISTClm50BgcCrop and run it for spinup?

Thanks in advance!

 

[oleson]

First of all, I think you would probably want to use IHISTClm50BgcCrop for your simulation, not I2000Clm50BgcCropGs. The IHIST compset would give you transient landcover, CO2, aerosol and nitrogen deposition, year-specific atmospheric forcing, etc (which are likely important for crop yields), while I2000 would give you year-specific atmospheric forcing but year 2000 land surface conditions.

(1) The initial file you list is for year 2000 (simyr2000), has a spatial resolution of 1.9x2.5, and was created by a release version of CLM (release-clm5.0.20). Those aspects differ from the simulation you are proposing (starting at year 1980, spatial resolution of 0.9x1.25, and using cesm2.2). I think your simulation will still run because the model will interpolate that initial file to your starting year and datasets. These differences may or may not be important for crop yields.

(2) That would give you an initial file that is equilibrated to 1960-1979 atmospheric conditions but would have I2000 aspects otherwise (e.g., CO2).

(3) Ideally, use I1850Clm50BgcCropGs to spinup over 1901 to 1920 atmospheric forcing and 1850 land surface conditions (e.g., land cover and CO2). Then use the initial file generated from that spinup to run a full historical using IHISTClm50BgcCrop for 1850-2010. You could possibly skip the spinup because the IHIST compset comes with an initial file that is spunup using CLM5 at 1850 conditions. The initial condition provided appears to be at 0.9x1.25 resolution. A reason why you might want to do the spinup is because of the model differences between cesm2.2 and the release version of CLM5. But if computational resources are a problem, then it would probably be reasonable to skip the spinup. I don't think there are major differences in CLM between cesm2.2 and the release version.

 

[Han Lubin]

Many thanks for your reply, Mr. Oleson! I realize I make mistakes in the simulation, and by the way, I wanna ask more questions.

(1)According to your advices, I realize there is wrong by setting surface file "surfdata_0.9x1.25_78pfts_simyr2000_c170428.nc" to simulate 1980-2010 global crop yields, which only represents the year 2000 (simyr2000). For simulating the distribution of surface CFTs from 1980 to 2010, the surface file should be changed to "landuse.timeseries_0.9x1.25_hist_78pfts_simyr1850-2015_c170428.nc". By the way, I also find the name of some data with “CMIP6”, such as "landuse.timeseries_0.9x1.25_hist_78pfts_CMIP6_simyr1850-2015_c170824.nc", and what is the difference between with "CMIP6" and without it?

(2)Sometimes, I wonder in simulations whether the start date is important, since the surface file (fsurdat), initial file (finidat), and the year-specific atmospheric forcing are all numerical files that have already been set? For example, to get the crop yields in 1990, it seems feasible to run the simulation from 1980 or from 1985 by using IHISTClm50BgcCrop. Do you have any suggestions on the start date setting for CLM5?

(3)I use "I" compset (IHIST) to simulate global crop yields based on DATM%GSWP3V1 + CLM50%BGC-CROP. If I want to set "F" compset (FHIST, CAM60 + CLM50%BGC-CROP) to simulate them, do I need to reset spinup? Or can the initial file your group provided for "I" compset be used in "F" compset as well?

(4)Considering time-consuming (600 model years) and computational resources for global equilibrium, if there is no pre-set initial file in INPUTDATA and I need to make my own spinup, how to shorten the real running time and ensure the accuracy of spinup? At high latitudes, soil carbon takes much longer to equilibrate because of cold, but for some crops, especially corn and rice, there is no CFT, so do I need to keep the high latitudes equilibrium?

Thanks in advance!

 

[oleson]

(1) It is generally bad practice to go searching in inputdata and plug in datasets you find there. Using properly chosen compsets will provide you with the datasets that are compatible with the version of the model you are using. The IHist compsets will give you a surface dataset file and a transient landcover file, e.g., for cesm2_2_0 at 1deg:

fsurdat = '/glade/p/cesmdata/cseg/inputdata/lnd/clm2/surfdata_map/release-clm5.0.18/surfdata_0.9x1.25_hist_78pfts_CMIP6_simyr1850_c190214.nc'

flanduse_timeseries = '/glade/p/cesmdata/cseg/inputdata/lnd/clm2/surfdata_map/landuse.timeseries_0.9x1.25_hist_78pfts_CMIP6_simyr1850-2015_c170824.nc'

You need both and they are not interchangable.

(2) The start date is important in historical or future transient simulations. If you want to start the model in 1980, you need to specify RUN_STARTDATE=1980-01-01. If you want to start the model in 1985, you need to specify RUN_STARTDATE=1985-01-01. In general, for spinups, the start year isn't important since you are looping over forcing and after the first go-thru the year is meaningless. So spinups usually start with RUN_STARTDATE=0001-01-01

(3) There will be differences between "observed" (GSWP3V1) and coupled model climate (F-case). There will be a period of adjustment when initializing an F-case with an initial file from an observationally-forced simulation. Generally, you may want to start your F-case simulation a few years before the first year you want analyze.

(4) Sure, if you ignore equilibrium at high latitudes, the model should spinup faster.

 

[Han Lubin]

Thank you so much, Mr. Oleson! Have a good day!

 

[Han Lubin]

Hello Mr.Oleson,
Sorry to bother you again, follow your advice, I run several model years for the equilibrium between land and atmosphere before I simulate the crop yield in 1986-2010. I set the case I with compset HIST_DATM%GSWP3v1_CLM50%BGC-CROP_SICE_SOCN_MOSART_CISM2%NOEVOLVE_SWAV. But after ./case.submit, the case I is killed with such error message:

m_GlobalSegMap::initp_: non-positive value of ngseg error, stat =0
01A.MCT(MPEU)::die.: from m_GlobalSegMap::initp_()

I don't find any solution in CESM forum, so I set another case (case II) with HIST_DATM%GSWP3v1_CLM50%BGC-CROP_SICE_SOCN_MOSART_SGLC_SWAV. The only difference between case I and II is whether the land ice module is turned on or not. But case II works fine.

So is the problem with the land ice module?

Here is my set in case II

./create_newcase --case maize_spunup_1850_test --compset HIST_DATM%GSWP3v1_CLM50%BGC-CROP_SICE_SOCN_MOSART_SGLC_SWAV --res f09_g17 --run-unsupported --compiler intel --mach bscca3

cd maize_spunup_1850_test

./xmlchange NTASKS=-4
./xmlchange NTHRDS=1
./xmlchange ROOTPE=0
./xmlchange NTASKS_ICE=-1
./xmlchange NTHRDS_ICE=1
./xmlchange ROOTPE_ICE=256
./xmlchange NTASKS_OCN=-1
./xmlchange NTHRDS_OCN=1
./xmlchange ROOTPE_OCN=256
./xmlchange NTASKS_WAV=-1
./xmlchange NTHRDS_WAV=1
./xmlchange ROOTPE_WAV=256

./case.setup

vi user_nl_clm
finidat='/public1/home/sc93599/CESM/my_cesm_sandbox/inputdata/lnd/clm2/initdata_map/clmi.I1850Clm50BgcCrop-ciso.1366-01-01.0.9x1.25_gx1v7_simyr1850_c200428.nc'
use_init_interp = .true.

./xmlchange RUN_TYPE=startup
./xmlchange DEBUG=TRUE
./xmlchange STOP_OPTION=nyears
./xmlchange STOP_N=1
./xmlchange RESUBMIT=6
./xmlchange RUN_STARTDATE=1979-01-01
./xmlchange DATM_CLMNCEP_YR_START=1979
./xmlchange DATM_CLMNCEP_YR_END=1985
./xmlchange DATM_CLMNCEP_YR_ALIGN=1979
./xmlchange DOUT_S=TRUE

./case.build

./check_input_data
./check_case

./case.submit

By the way, I use CESM2.2 and attach the file with error message. Look forward to your reply.
Thanks in advance！

 

[Han Lubin]

Sorry for the cesm.log is too large, I attach cesm.log.zip ...

 

[oleson]

The error message isn't very informative unfortunately (to me at least).
I tried this without changing the start date and it ran fine I also tried it with the change to the start date and it ran fine.
Note that changing the start date changed the initial file to:

/glade/p/cesmdata/cseg/inputdata/lnd/clm2/initdata_map/clmi.I2000Clm50BgcCrop.2011-01-01.1.9x2.5_gx1v7_gl4_simyr2000_c190312.nc

Changing the initial file to 1850 file you specified also ran fine.
The one thing I didn't change was the PE layout. Is there a reason you are changing it?
Otherwise, I think I would try using the year 2000 initial file (which is more reasonable for your simulation than 1850) and/or run with SGLC.

 

[oleson]

I also tried with your PE layout and it ran fine, albeit slowly.
I used release-cesm2.2.0 for these tests.

 

[Han Lubin]

Hello Mr.Oleson
Thank you for your kindness! Unfortunately, comparing to CESM2 TIMING, PERFORMANCE AND LOAD BALANCING DATA in CESM2 official website, I find the case run so slowly follow my PE layout... I change the PE layout in another case cloned from case II mentioned above, as close as possible to the settings in HIST_DATM%GSWP3v1_CLM50%BGC-CR

./xmlchange NTASKS=-3
./xmlchange ROOTPE=64
./xmlchange NTHRDS=1

./xmlchange NTASKS_ATM=-1
./xmlchange ROOTPE_ATM=0
./xmlchange NTHRDS=1

./xmlchange NTASKS_IAC=1
./xmlchange ROOTPE_IAC=0
./xmlchange NTHRDS=1

It ran fine, but still slowly. In theory, I think it doesn't take me long time to run "I" compset with CLM, MOSART and DATM. There are not many grid need to simulate. Is there a mistake in my PE layout? Could you please give me any advice?

In addition, I also noticed finidat='clmi.I2000Clm50BgcCrop.2011-01-01.1.9x2.5_gx1v7_gl4_simyr2000_c190312.nc' when I change startdate. Will the model choose a more appropriate initial file to 2000 or 1850 depending on start date?

Finally, I re-did the test, and when I use the year 2000 initial file and SGLC, it ran fine, but when I use the year 2000 initial file and CISM2%NOEVOLVE, it reported the same error. Next I'll try to find out if setting SGLC will have an influence on the crop yield, which I currently think should be almost none.

Here is some config_*.xml. Thanks in advance for your reply!

 

[oleson]

Regarding the speed of your run, are you still running in DEBUG mode? That will slow down the model significantly.
Otherwise, your choice of resolution (f09_g17) means that there are 288X192 gridcells of which 21,013 are land. Furthermore, there are 62,100 landunits, 590,024 columns, and 884,206 patches to be simulated (according to the lnd log). On our machine, the setup comes out of the box requesting 51 nodes (1836 processors). With that setup I get about 100 model years per wallclock day. You could throw more processors at it or consider a coarser resolution (f19_g17) if the model is still too slow.
Right, it does appear to change the initial file based on a change in the RUN_STARTDATE for this particular compset and resolution. It looks like there is some logic in the build namelist code that finds an initial file that is closest to the start date.
I agree that setting SGLC should not have any effect on crop yields.

 

[Han Lubin]

Mr. Oleson, Thank you so much for your advice!
The case ran faster than before, and I wasn't aware that debug mode was slowing down the case.

 

[Han Lubin]

Hello Mr.Oleson,

Sorry to bother you. I am trying to analyse the difference of crop yield between compset I with CAM60+CLM%Bgc-Crop and compset II with DATM+CLM%Bgc-Crop. I think there are two ways to get it. One way for compsetI and compsetII is to set ./xmlchange RUN_TYPE=startup and initial file with “clmi.I2000Clm50BgcCrop.2011-01-01.1.9x2.5_gx1v7_gl4_simyr2000_c190312.nc”.Another way is to set ./xmlchange RUN_TYPE=hybrid and initial file with “b.e20.BHIST.f09_g17.20thC.297_01_v3.clm2.r.1979-01-01-00000.nc”. So I'd like to ask:

(1)If I set ./xmlchange START_DATE=1979-01-01 and get crop yields in 1986-2010, are both results for one way and another way reasonable?
(2)I try to run this case with CAM60+CLM%Bgc-Crop+initial file “b.e20.BHIST.f09_g17.20thC.297_01_v3.clm2.r.1979-01-01-00000.nc”, but I encountered such error to stop the run.

"lnd_import ERROR: Longwave down sent from the atmosphere model is negative or zero"

After searching the same error in the forum, I think this error is usually due to the weather data is bad download. But I don't realize which file is wrong, could you please give me some advice?

Thanks in advance!

 

[oleson]

(1) It doesn't seem unreasonable since you appear to be allowing for a six year spinup in both cases (1979-1985). I assume both of those initial files come out of the box from the two cases you setup.
(2) Difficult to determine if this is really from a bad download of a file. You'll have to provide the exact steps you took for this simulation beginning with the version of the model you are using and your create_newcase command, especially any modifications you've made. I can then try to replicate the error here.

 

[Han Lubin]

Mr. Oleson, Thanks for your reply! For question (1), I set case I with CAM60+CLM%Bgc-Crop+initial file “b.e20.BHIST.f09_g17.20thC.297_01_v3.clm2.r.1979-01-01-00000.nc” and case II with CAM60+CLM%Bgc-Crop+initial file “clmi.I2000Clm50BgcCrop.2011-01-01.1.9x2.5_gx1v7_gl4_simyr2000_c190312.nc” to get crop yields in 1986-2010 from 1979 (./xmlchange START_DATE=1979-01-01). Which of the two cases is more appropriate? Or maybe both are reasonable?

For question 2, details are as follows.

./create_newcase --case maize_be20_irr_test --compset HIST_CAM60_CLM50%BGC-CROP_SICE_SOCN_MOSART_SGLC_SWAV --res f09_g17 --run-unsupported --compiler intel --mach bscca3

cd maize_be20_irr_test

./xmlchange NTASKS=-3
./xmlchange ROOTPE=192
./xmlchange NTHRDS=1

./xmlchange NTASKS_ATM=-3
./xmlchange ROOTPE_ATM=0
./xmlchange NTHRDS=1

./xmlchange NTASKS_IAC=1
./xmlchange ROOTPE_IAC=0
./xmlchange NTHRDS=1

./case.setup

vi user_nl_clm
hist_nhtfrq=0,0
hist_mfilt=1,12
hist_fincl2='GRAINC_TO_FOOD'
hist_dov2xy=.true.,.false.
irrigate=.true.
use_init_interp = .true.

./xmlchange RUN_TYPE=hybrid
./xmlchange RUN_REFDIR='/public1/home/sc93599/CESM/my_cesm_sandbox/inputdata/cesm2_init/b.e20.BHIST.f09_g17.20thC.297_01_v3/1979-01-01/b.e20.BHIST.f09_g17.20thC.297_01_v3.clm2.r.1979-01-01-00000.nc'
./xmlchange RUN_REFCASE=b.e20.BHIST.f09_g17.20thC.297_01_v3
./xmlchange RUN_REFDATE=1979-01-01
./xmlchange STOP_OPTION=nyears
./xmlchange STOP_N=1
./xmlchange RESUBMIT=35
./xmlchange RUN_STARTDATE=1979-01-01
./xmlchange DOUT_S=TRUE

./case.build

./check_input_data
./check_case

./case.submit

I try to run this case with CAM60+CLM%Bgc-Crop+initial file “b.e20.BHIST.f09_g17.20thC.297_01_v3.clm2.r.1979-01-01-00000.nc”, but I encountered such error to stop the run.

"lnd_import ERROR: Longwave down sent from the atmosphere model is negative or zero"

I set another case with CAM60+CLM%Bgc-Crop+initial file “clmi.I2000Clm50BgcCrop.2011-01-01.1.9x2.5_gx1v7_gl4_simyr2000_c190312.nc” for test, but also encountered the same error. Could you please give me some advice?

I use CESM2.2.0 and attach the file with error message. Thanks in advance!

 

[oleson]

The compset and resolution you are using in release-cesm2.2.0:

--compset HIST_CAM60_CLM50%BGC-CROP_SICE_SOCN_MOSART_SGLC_SWAV --res f09_g17

comes out of the box with

START_DATE=1979-01-01

and this initial file:

/glade/p/cesmdata/cseg/inputdata/lnd/clm2/initdata_map/clmi.BHIST.2000-01-01.0.9x1.25_gx1v7_simyr2000_c200728.nc

So, that's the initial file I would use. I'm not sure what your reasons are for running a hybrid with RUN_REFCASE=b.e20.BHIST.f09_g17.20thC.297_01_v3, or for using clmi.I2000Clm50BgcCrop.2011-01-01.1.9x2.5_gx1v7_gl4_simyr2000_c190312.nc.

That said, my own run using this initial file crashed with the same error. Since this is coming from the atmospheric model, I would suggest posting to the CAM forum. For reference, here are the steps that I took, you might try repeating these:

git describe
cesm2.2.0

cd cime/scripts
./create_newcase --case cesm220_FHISTClm50BgcCrop --compset HIST_CAM60_CLM50%BGC-CROP_SICE_SOCN_MOSART_SGLC_SWAV --res f09_g17 --run-unsupported
cd cesm220_FHISTClm50BgcCrop
./xmlchange STOP_OPTION=nmonths
./xmlchange STOP_N=1
./case.setup
qcmd -- ./case.build
./case.submit

 

[Xiaoyu yu]

Hello Oleson,

I'm also confused about spinup procedure. I want to simulate the physical properties of snow from 1980 to 2010 in regional scale by using CESM2.2 to run compset I2000Clm50Sp and res f09_g17. Here is my confusion about the steps：

(1) First creat a spin-up case (I2000Clm50_spinup) to evaluate the model's equilibrium and change CLM_ACCELERATED_SPINUP=on, CLM_FORCE_COLDSTART=on.
Considering the choice of SP mode and the characteristics of snow, it may only need a short time to reach equilibrium, for example, DATM_CLMNCEP_YR_START=1980,DATM_CLMNCEP_YR_END=1990.

Then creat a common case (I2000Clm50Sp) running from 1980 to 2010, and set the finidat in user_nl_clm as the output of spin-up case (/archive/I2000Clm50_spinup/rest/1990-01-01-00000/I2000Clm50_spinup.clm2.r.1990-01-01-00000.nc).

Is this the right approach for doing a spin-up? I have seen another approach which creats a spin-up case running in 10 years to reach equilibrium and then creat a common case in branch type by setting RUN_REFCASE to the spin-up case and copying initial and restart files to 'run' directory. What is the difference between this two approaches?

(2) If I want the spatial resolution of output to be 0.5 degree as the GSWP3, what should be changed? like setting res as CLM_USRDAT and changing the domain and surface file (surfdata_360x720cru_16pfts_Irrig_CMIP6_simyr2000_c170824.nc)?

Any suggestions will be appreciated! Thanks in advance.