question about CLM5 spin up

[hyf412694462]

For Q1, I suggest that you try it, i.e. modify the .ncl script to do what you want. Otherwise, rerun with hist_mfilt = 1.

For Q2, you will find that we provide default initial condition files when you set up your simulation. That is the extent of what we offer, unfortunately.

Dear Dr. Slevis,

Thank you very much for your information. Now I can utilize the .ncl script successfully.
I am very new for the usage of CLM5.0. I have three another questions which confused me a lot, I would like to inquire a help.
Q1, For example, if I have already spun up the model for 200yr but the equilibrium state is still not achieved, then I want to continue to spin up the model for like 100yr. How can I do this？ Should I only change CONTINUE_RUN to True and change the STOP_N to 100 and then ./case.submit or I need further specify the finidat or change the RUN_TYPE? I am not sure about the difference between 'startup' and 'branch'.

Q2, when I finished spinning up the model and got the spun up restart file by conducting the following simulation:

./create_newcase --case I1850Clm50BgcCrop_spinup_01 --res f19_g17 --compset  I1850Clm50BgcCrop  --run-unsupported
cd I1850Clm50BgcCrop_spinup_01
./xmlchange STOP_OPTION='nyears'
./xmlchange STOP_N=200
./xmlchange REST_N=1
./xmlchange REST_OPTION='nyears'
./xmlchange CLM_ACCELERATED_SPINUP='on'
./case.setup
./case.build
./case.submit

In my thinking, this spun up restart file stand for the conditions of 1850. Then I would like to use this spun up restart file to conduct transient simulation from 1850 to 2014 to try to reproduce the real situation in this period. The question is there are no atmospheric forcing data from 1850 to 1900, so I wonder how to design this experiment from 1850 to 2014?

Q3, the machine I used to run CLM is a Ubutun server (no batch system) with 40 Intel(R) Xeon(R) Silver 4210 CPU. One CPU has 10 cores and 20 threads. I wonder how to set MAX_TASKS_PER_NODE and MAX_MPITASKS_PER_NODE or I need to set some other variables in config_machines.xml to utilize all cores and threads of 15 Intel CPU to run the model?

I would greatly appreciate your help.

 

[slevis]

Q1: As you said, you should change CONTINUE_RUN to True. and submit. Branch is like "continue" except it allows minor changes.
Q2: We have datm data from 1901 to 2014 (maybe you are using the same data). From 1850 to 1900 we cycle the 1901 to 1920 data. In 1901 we start using all the data. That transition is an example where I use "hybrid" because continue and branch I think will not work.
Q3: I do not know the answer to this question. Maybe @erik?

 

[hyf412694462]

Q1: As you said, you should change CONTINUE_RUN to True. and submit. Branch is like "continue" except it allows minor changes.
Q2: We have datm data from 1901 to 2014 (maybe you are using the same data). From 1850 to 1900 we cycle the 1901 to 1920 data. In 1901 we start using all the data. That transition is an example where I use "hybrid" because continue and branch I think will not work.
Q3: I do not know the answer to this question. Maybe @erik?

Dear slevis,

I appreciate very much your quick response, which is very helpful. I tried your advice (change CONTINUE_RUN to True and ./case.submit) and it worked fine. I can understand your Q2 response, thanks a lot.
For Q3, Still I would like to receive any information from you.

Best,

 

[oleson]

You may post your Q3 on the infrastructure/porting forum:

Infrastructure (CIME, porting, machines, scripts)

CIME, case control system, build and configure, machines and supporting scripts.

bb.cgd.ucar.edu

 

[hyf412694462]

You may post your Q3 on the infrastructure/porting forum:

Infrastructure (CIME, porting, machines, scripts)

CIME, case control system, build and configure, machines and supporting scripts.

bb.cgd.ucar.edu

Ok, thank you for your suggestion, I will post Q3 on the infrastructure/porting forum.

 

[wvsi3w]

Dear @oleson I have checked my SpinupStability.ncl (which is in this path: my_cesm_sandbox/components/clm/tools/contrib/) and by running it I noticed that the directory it is pointing to is set to yours (Input directory does not exist or not found: /glade/scratch/oleson/archive/clm50_release-clm5.0.15_2deg_GSWP3V1_1850AD/lnd/hist/) and I checked the script too it is showing data_dir and case id of yours which I guess I have to change them to mine (?) to perform the test. But I think the paths that I need to change in the script are more than a few, right? I mean what should someone do in this case?

Because this is what the script is saying:

The variables examined are TOTECOSYSC,TOTSOMC,TOTVEGC,TLAI,GPP,TWS,H2OSNO.
; The percentage of land area in TOTECOSYSC disequilibrium is also examined (not for single point).
; Thresholds are defined below (i.e., global_thresh_*) and can be changed for individual
; variables.
; To run this script, just enter in your case name and username below.
;  AND set the annual_hist flag to True if your case has annual mean history files or set
;  annual_hist flag to False if your case has monthly mean history files.
;  AND set the region (supported options: Global, Arctic, SPT).
;  AND set the subper (subsampling period in years, number of years that atm forcing repeats).
; The script assumes that your history files are in /glade/scratch/$username/archive/$caseid/lnd/hist

But the lnd/hist of my case is not set in that type of structure, so should I change the whole path to mine? or it will ruin the test if I change that path completely?

Moreover, the variables I need to test and plot are not only these that the script is mentioning so can I add my desired variables to the test? and how.


I am using CESM2.1.3 in Canada (Narval and Beluga systems).

 

[slevis]

I would start by changing the obvious things that you saw, plus whatever the script documentation suggested. If you still get an error, we may need to rethink.

 

[wvsi3w]

I would start by changing the obvious things that you saw, plus whatever the script documentation suggested. If you still get an error, we may need to rethink.

Dear Sam,
Thanks for the response.
I changed it according to my directory and case id.
But it fails with this error:

fatal:file (lstfile) isn't defined
fatal:["Execute.c":8637]:Execute: Error occurred at or near line 98 in file SpinupStability.ncl

and it cannot access the files in lnd/hist directory:

cannot access '/home/USER/scratch/cases/ND_Narval_1deg_All_variables/lnd/hist/ND_Narval_1deg_All_variables.clm2.h0.*-*-*-*.nc': No such file or directory
warning:_NclOpenFile: cannot open file <missing>; No such file or directory

I think this is because my history files are monthly (I really don't know why I have monthly instead of yearly):

ND_Narval_1deg_All_variables.clm2.h0.1000-01.nc  ND_Narval_1deg_All_variables.clm2.h0.1066-09.nc  ND_Narval_1deg_All_variables.clm2.h0.1133-05.nc
ND_Narval_1deg_All_variables.clm2.h0.1000-02.nc  ND_Narval_1deg_All_variables.clm2.h0.1066-10.nc  ND_Narval_1deg_All_variables.clm2.h0.1133-06.nc
ND_Narval_1deg_All_variables.clm2.h0.1000-03.nc  ND_Narval_1deg_All_variables.clm2.h0.1066-11.nc  ND_Narval_1deg_All_variables.clm2.h0.1133-07.nc

So there are two questions:

1) can I edit the SpinupStability.ncl script so that it accepts monthly data? if not, what should I do. I can not rerun the whole thing with yearly history file it will take a lot of time. If you say there is no way and I have to rerun this ND spin up part from the beginning (year 1000) then I guess I have to. Or maybe I should use some command to make an average of each year in this lnd/hist/ directory of mine and then run the script? wouldn't this affect the output of the script? One more thing, am I referring correctly to the history files frequency because my AD branch has 20yearly output, so do you think I also need to have 20yearly instead of 1yearly outputs?

2) Out of curiosity, how this SpinupStability script plots the year 1 till 1200 when it is only using ND history files? I mean ND is starting from 1000 and I see in the plots of other people who used the SpinupStability script that the year on those plots are from 1 till the end. So I guess it will grab some info from the AD by itself?


Thank you @slevis and @oleson for your support

 

[wvsi3w]

(0)     EQUILIBRIUM SUMMARY
(0)     =======================================================================
(0)     TOTECOSYSC is in equilibrium. Eq. Yr. = 60
(0)     TOTSOMC is in equilibrium. Eq. Yr. = 40
(0)     TOTVEGC is in equilibrium. Eq. Yr. = 60
(0)     TLAI is in equilibrium. Eq. Yr. = 0
(0)     GPP is in equilibrium. Eq. Yr. = 40
(0)     TWS is in equilibrium. Eq. Yr. = 40
(0)     WARNING: H2OSNO is NOT in equilibrium or is missing
(0)     At least 97 percent of the land surface is in TOTECOSYSC equilibrium. Eq. Yr. = 220
(0)     Percent of the land surface not in equilibrium (  1.30% )
(0)     Congratulations! Your simulation is in equilibrium

I tested the script with the AD spinup (from year 1 to year 885) and it is saying it reached equilibrium but we all know we should try it when ND is done.
I should mention that the frequency of the files in this case was 20y which means all I have is like this:

spinup1degAD_Narval.clm2.h0.0000-01-01-00000.nc  spinup1degAD_Narval.clm2.h0.0300-01-01-00000.nc  spinup1degAD_Narval.clm2.h0.0600-01-01-00000.nc
spinup1degAD_Narval.clm2.h0.0020-01-01-00000.nc  spinup1degAD_Narval.clm2.h0.0320-01-01-00000.nc  spinup1degAD_Narval.clm2.h0.0620-01-01-00000.nc
spinup1degAD_Narval.clm2.h0.0040-01-01-00000.nc  spinup1degAD_Narval.clm2.h0.0340-01-01-00000.nc  spinup1degAD_Narval.clm2.h0.0640-01-01-00000.nc

So it worked when we have 20yearly output for history files. So I believe I should do the following (correct me if I am wrong):

- save the output of my currently finished ND case from monthly frq to 20yearly (maybe instead of rerunning the whole case with a different frequencies I can use some command to make an average of 20y?)
- put all the outputs of AD and ND in one place so that it plots the whole spinup from year 1 to the end (because I tried the script for a branch case that I had and it started the plot from the start of the branch "866 to 1000") ?

 

[oleson]

The script will work with monthly history files:
"; This version operates on either monthly mean or multi-annual mean multi-variable history files"

You need to set annual_hist = False.

"
; To run this script, just enter in your case name and username below.
; AND set the annual_hist flag to True if your case has annual mean history files or set
; annual_hist flag to False if your case has monthly mean history files.
"

You also need to run the script separately for each phase of the spinup, e.g., AD and ND.

 

[wvsi3w]

The script will work with monthly history files:
"; This version operates on either monthly mean or multi-annual mean multi-variable history files"

You need to set annual_hist = False.

"
; To run this script, just enter in your case name and username below.
; AND set the annual_hist flag to True if your case has annual mean history files or set
; annual_hist flag to False if your case has monthly mean history files.
"

You also need to run the script separately for each phase of the spinup, e.g., AD and ND.

OMG, I am so blind to not see that flag! So sorry.

Ok I tried it and it says it is not in equilibrium yet:

EQUILIBRIUM SUMMARY
(0)     =======================================================================
(0)     TOTECOSYSC is in equilibrium. Eq. Yr. = 1100
(0)     TOTSOMC is in equilibrium. Eq. Yr. = 1000
(0)     TOTVEGC is in equilibrium. Eq. Yr. = 1060
(0)     TLAI is in equilibrium. Eq. Yr. = 1000
(0)     GPP is in equilibrium. Eq. Yr. = 1000
(0)     TWS is in equilibrium. Eq. Yr. = 1000
(0)     WARNING: H2OSNO is NOT in equilibrium or is missing
(0)     FATAL: Not enough of the land surface is in equilibrium (  5.73% >   3.00%)
(0)     FATAL: Your simulation is not in equilibrium, 8 hours have been deducted from your PTO bank, try again

So, I believe I should add more years to the ND part (by any chance do you know how many more years would be needed to reach below 3% for the land surface?)

 

[oleson]

It will likely take another few hundred years. However, the length of time is likely due to deep soil carbon at high latitudes. If you don't care so much about that, then you could probably accept 5% disequilibrium.

 

[wvsi3w]

It will likely take another few hundred years. However, the length of time is likely due to deep soil carbon at high latitudes. If you don't care so much about that, then you could probably accept 5% disequilibrium.

Dear Keith,
Thanks for the hint.
Unfortunately, I should try it with more years since soil carbon plays an important role in my study.

Another question is I noticed my forcing years were 1901 till 1902 (DATM_CLMNCEP_YR_END=1902), so instead of subsampling = 20, I should put 2, right? (I thought maybe I should put 1 since 1901-1902 seems one year, butit also failed with the same message), it failed when I changed subper=20 to 2:

warning:_NclOpenFile: Can not open file </home/USER/scratch/cases/ND_Narval_1deg_All_variables/lnd/hist/ND_Narval_1deg_All_variables.clm2.h0.1000-01.nc>; file format not supported or file is corrupted
warning:A valid instance of variable TOTECOSYSC was not found in one or more elements of the file list
(0)     *** FATAL ***
fatal:Variable (nyrs) is undefined
fatal:["Execute.c":8637]:Execute: Error occurred at or near line 10551 in file $NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl

fatal:["Execute.c":8637]:Execute: Error occurred at or near line 167 in file SpinupStability.ncl

prior to this it worked with subper=20 which is odd as my CLMNCEP_YR_END was 1902 !

 

[wvsi3w]

Dear Keith,
Thanks for the hint.
Unfortunately, I should try it with more years since soil carbon plays an important role in my study.

Another question is I noticed my forcing years were 1901 till 1902 (DATM_CLMNCEP_YR_END=1902), so instead of subsampling = 20, I should put 2, right? (I thought maybe I should put 1 since 1901-1902 seems one year, butit also failed with the same message), it failed when I changed subper=20 to 2:

warning:_NclOpenFile: Can not open file </home/USER/scratch/cases/ND_Narval_1deg_All_variables/lnd/hist/ND_Narval_1deg_All_variables.clm2.h0.1000-01.nc>; file format not supported or file is corrupted
warning:A valid instance of variable TOTECOSYSC was not found in one or more elements of the file list
(0)     *** FATAL ***
fatal:Variable (nyrs) is undefined
fatal:["Execute.c":8637]:Execute: Error occurred at or near line 10551 in file $NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl

fatal:["Execute.c":8637]:Execute: Error occurred at or near line 167 in file SpinupStability.ncl

prior to this it worked with subper=20 which is odd as my CLMNCEP_YR_END was 1902 !

I think you explained it here and it doesn't matter what our DATM end year is we still need to put 20 for subper (?)

 

[wvsi3w]

I think you explained it here and it doesn't matter what our DATM end year is we still need to put 20 for subper (?)

please ignore my last message as I found other people using values for subper that is less than 20. so my hypothesis was wrong.

Questions about BGC spinup

Dear all, Hi! I am trying to run a I2000Bgc compset spinup following the section 1.5.5 Spinup of CLM5.0-BGC of Clm5.0 User's guide.I want to spinup using the forcing data from 1979 to 1983.and try to simulate the model year from 1984 to 2014. Here are some setting: ./create_newcase -case...

bb.cgd.ucar.edu

 

[oleson]

Generally speaking, it should be set to the number of years of forcing you are looping over. However, it will also run if you set it to a multiple of your forcing years, so subper=20 would work also. It would sample every 20 years, instead of every 2 years, but it would still be sampling the same year of forcing each time.

 

[wvsi3w]

Generally speaking, it should be set to the number of years of forcing you are looping over. However, it will also run if you set it to a multiple of your forcing years, so subper=20 would work also. It would sample every 20 years, instead of every 2 years, but it would still be sampling the same year of forcing each time.

Ok, thank you, good to know that.

I have a question that might seem trivial:
I know that when your job is canceled or fails or isn't completed and cancels for some reason you can do CONTINUE_RUN=TRUE so that it continues from where it got canceled (failed). For other purposes like adding variables to namelist also you can do branch or hybrid run types.

But, as you know I have a case (this ND case we discussed) that is in the year 1200 and it has finished and completed the ND spinup but is still not in good equilibrium so I have a question:
Am I allowed to continue running this ND case (which did not cancel, nor fail) to run it from 1200 till 1400 (running it for another 200y), by changing some XML options like "RESUBMIT=39", "STOP_N=5", "STOP_OPTION=nyears", " CONTINUE_RUN=TRUE" ??? and if so should I set the "RUN_STARTDATE" from 1000-01-01 to what it should start from which is 1200-01-01??? is it normal to do this in these situations?

If not, so I think I should start a new case and copy the finidat and pointers and etc from this ND case to the new case then (?)

I am asking these because I am worried to ruin this ND case which took a long time to finish, so I should better be sure before changing anything.
Thanks for your support.