Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

Mistake in AD spin-up (I need your help)

wvsi3w

wvsi3w

wvsi3w
Member
Hello,
I have noticed that I made a huge mistake doing the AD spin-up which I will explain below:

First and before doing the spinup I tested the CESM (2.1.3) for the land (CLM5) with this compset "IHistClm50BgcCrop" (1deg) with the below detail in user_nl_clm:

Code: 
soil_layerstruct= '23SL_3.5m'
use_init_interp = .true.

&clm_inparm
    hist_empty_htapes = .true.
    hist_fincl1 = 'TSOI', 'TSOI_ICE'
/

With the above details, I have removed all other variables and just included the two soil temperatures. I thought by removing all variables in the history file and adding these two variables I could speed up my test (which it didn't). Anyway, then, I started my AD spin-up (1000y) without removing all those variables from the history files (and without adding the two T variables), and I thought the output would create every variable that is available (including the TSOI). You know when you do a simple test run with these land compsets you will get about 400 variables in the output and TSOI is one of them (also a lot of other variables are there, like CH4, HEAT_CONTENT, ALT, TSL, ...)

But, little did I know that it did not create all of the variables in this AD spinup phase and only showed a bunch of variables like "TOTVEGC, TOTVEGN, GPP, NPP, TLAI, ...".

This is awkward and I know I should have checked the output in the beginning. I am asking if there is a way to add the TSOI now? the spinup for AD is near its end and I should do the ND spin later. I didn't know by doing the default AD spin-up for this compset I wont get the TSOI and other 400 variables in the output. I am so sad and disappointed of myself.

I suppose whatever variable you assign in the user_nl_clm would be there in the output (in case you don't want to go with the default which I explained lacks the other 400 variables), so I have some questions regarding that:

1- Do you think it is possible to add TSOI right now (it is in its 800y of AD, 200y remaining, +200y ND)? I am not sure if this is a good idea, but I think that the temperature signal might need a couple of centuries (or maybe more) to reach the bottom boundary (in this case 43.8m), so do you think I can use the restart file of this current spin-up, let's say I use the year 700 of this current AD spin-up as finidat in the namelist of a new case where I turn on TSOI and other variables in it and continue the run from there (?) But I am not sure if this is a good idea since the AD spin-up is not finished yet and we don't have the effects of ND spin-up on the outputs so I don't know, what do you think?

2- Is there a way to not do a lengthy spin-up (AD+ND)? because AD-ND spin-ups are for decomposition and I guess there might be other land-related compsets that don't require 1000y of spin-up. I know that SP are not BGCs and might be one of the compsets for my case but I don't think I can use satellite phenology for my study that involves subsurface dynamics (Temperature, ...). If so, I can do the new shorter spin-up using that from the beginning while adding the TSOI, ... to the namelist.

3- In case there are no compset available to skip the lengthy AD-ND spin-up part; I should redo the spin-up with all the desired variables that I need to be spun-up, right? because as far as I learned, if I don't include all the variables that I want in my spin-up namelist then when I am going to do the transient run those variables would not be spun-up and would start from arbitrary values, am I correct? So I guess to be prepared, I should put almost all of my desired variables in the namelist to see them spun-up later in the outputs of the final phase, otherwise there is no use to the spin-up which has only a few unwanted variables in it (?)
 
slevis

slevis

Moderator
Staff member
1. Yes, you can add new history fields partway through a simulation. I'm not confident whether it will work when you restart the simulation (i.e. as a CONTINUE run). But it will work if you use the "branch" RUN_TYPE.
 
slevis

slevis

Moderator
Staff member
3. The history fields specified in hist_fincl statements only affect which variables appear in history files. The model still gets spun up as a whole model, regardless of which fields you select to send to history.
 
wvsi3w

wvsi3w

wvsi3w
Member
slevis said:
1. Yes, you can add new history fields partway through a simulation. I'm not confident whether it will work when you restart the simulation (i.e. as a CONTINUE run). But it will work if you use the "branch" RUN_TYPE.
Click to expand...
Thank you Sam for your helpful answer,

I have followed this example (Exercise 2: Modify run type — CESM Tutorial).
I created the case "1deg_SpinupAD_Narval_All_Variables" and did the following:

Code: 
./xmlchange RUN_TYPE=branch
./xmlchange RUN_REFCASE=spinup1degAD_Narval
./xmlchange RUN_REFDATE=0885-01-01

Then I copied the restart file from year 885 of the previous case:
"spinup1degAD_Narval.clm2.r.0885-01-01-00000.nc" and "spinup1degAD_Narval.clm2.rh0.0885-01-01-00000.nc"

And I modified the user_nl_clm of this case:

Code: 
soil_layerstruct= '23SL_3.5m'use_init_interp = .true.&clm_inparm    hist_empty_htapes = .true.    hist_fincl1 = 'TSOI', 'TSOI_ICE', 'FCH4', 'CH4PROD', 'ALT', 'ALTMAX', 'DSL', 'LAI', 'FCEV', 'FSH_G', 'FSH_V', 'GPP', 'NPP', 'GR', 'CPOOL', 'HEAT_CONTENT1', 'ICE_CONTENT1', 'TG', 'TSOI_10CM', 'TBOT', 'TOTCOLC', 'TOTCOLCH4', 'TOTCOLN', 'TOTVEGC', 'TOTVEGN', 'TSA', 'STORVEGC', 'STORVEGN', 'SUCSAT', 'BSW', 'TOTECOSYSC', 'TOTECOSYSN', 'TOTSOMC', 'TOTSOMN', 'TWS', 'TSL', 'TV', 'TLAI', 'WIND', 'SOILICE', 'SOILC_vr', 'SOILN_vr', 'ZSOI', 'DZSOI', 'WATSAT', 'HKSAT', 'ZLAKE', 'DZLAKE'

But In that link of exercise, it says to copy the restart file from the previous run (doesn't say to copy rpointer files, which makes sense because we set refcase in our branch case and it should point to the case and refdate we assigned and there is no need for rpointers anymore (right?))

Anyway, should I set the finidat in my user_nl_clm of branch case to the restart file that I copied here? because the exercise doesn't mention this finidat change for restart file.
 
wvsi3w

wvsi3w

wvsi3w
Member
wvsi3w said:
Thank you Sam for your helpful answer,

I have followed this example (Exercise 2: Modify run type — CESM Tutorial).
I created the case "1deg_SpinupAD_Narval_All_Variables" and did the following:

Code: 
./xmlchange RUN_TYPE=branch
./xmlchange RUN_REFCASE=spinup1degAD_Narval
./xmlchange RUN_REFDATE=0885-01-01

Then I copied the restart file from year 885 of the previous case:
"spinup1degAD_Narval.clm2.r.0885-01-01-00000.nc" and "spinup1degAD_Narval.clm2.rh0.0885-01-01-00000.nc"

And I modified the user_nl_clm of this case:

Code: 
soil_layerstruct= '23SL_3.5m'use_init_interp = .true.&clm_inparm    hist_empty_htapes = .true.    hist_fincl1 = 'TSOI', 'TSOI_ICE', 'FCH4', 'CH4PROD', 'ALT', 'ALTMAX', 'DSL', 'LAI', 'FCEV', 'FSH_G', 'FSH_V', 'GPP', 'NPP', 'GR', 'CPOOL', 'HEAT_CONTENT1', 'ICE_CONTENT1', 'TG', 'TSOI_10CM', 'TBOT', 'TOTCOLC', 'TOTCOLCH4', 'TOTCOLN', 'TOTVEGC', 'TOTVEGN', 'TSA', 'STORVEGC', 'STORVEGN', 'SUCSAT', 'BSW', 'TOTECOSYSC', 'TOTECOSYSN', 'TOTSOMC', 'TOTSOMN', 'TWS', 'TSL', 'TV', 'TLAI', 'WIND', 'SOILICE', 'SOILC_vr', 'SOILN_vr', 'ZSOI', 'DZSOI', 'WATSAT', 'HKSAT', 'ZLAKE', 'DZLAKE'

But In that link of exercise, it says to copy the restart file from the previous run (doesn't say to copy rpointer files, which makes sense because we set refcase in our branch case and it should point to the case and refdate we assigned and there is no need for rpointers anymore (right?))

Anyway, should I set the finidat in my user_nl_clm of branch case to the restart file that I copied here? because the exercise doesn't mention this finidat change for restart file.
Click to expand...
Oh, I get it, I should have copied the restart files to the run directory of my new branch case. sorry
 
wvsi3w

wvsi3w

wvsi3w
Member
I am putting this here for my reference and in case someone needs to know the steps to take in Narval cluster Canada for this error I encountered and explained above:

To add variables (you missed) in a spin-up (AD) using branch run type:

#run the modules on Narval:
module load StdEnv/2020 xml-libxml/2.0205 intel/2020.1.217 openmpi/4.0.3 netcdf-fortran-mpi/4.6.0 cmake/3.23.1 pnetcdf/1.12.2 netcdf-mpi/4.7.4 perl/5.30.2

cd my_cesm_sandbox/cime/scripts/

#create another case similar to spinup AD case (which was "spinup1degAD_Narval"):
./create_newcase --case /home/$USER/scratch/cases/1deg_SpinupAD_Narval_All_Variables --compset IHistClm50BgcCrop --res f09_g17 --machine narval --walltime 03:59:00 --run-unsupported

cd /home/$USER/scratch/cases/1deg_SpinupAD_Narval_All_Variables

./xmlchange RUN_TYPE=branch

#set the ref of case and date according to the AD case which got stopped (failed, or canceled):
./xmlchange RUN_REFCASE=spinup1degAD_Narval
./xmlchange RUN_REFDATE=0885-01-01

#since your AD is not finished you need to turn on AD for this branch run type too:
./xmlchange CLM_ACCELERATED_SPINUP="on"

#set all other env_run options similar to the AD that you are branching from:
./xmlchange DATM_CLMNCEP_YR_END=1902
./xmlchange NTASKS=-8
./xmlchange NTASKS_ESP=1
./xmlchange STOP_OPTION=nyears,STOP_N=5

./case.setup

./xmlchange JOB_WALLCLOCK_TIME=03:59:00
./xmlchange RESUBMIT=22



#edit user namelist (make sure there are no duplicates or wrong variables names, and pay attention to that last dash "/" and make sure to include the variables from lnd_in file on the last AD spin-up which you are branching from because if you don't it fails (in my case the last 11 variables are from previous run which I stopped because it lacked the other variables listed below) ) :
emacs user_nl_clm

soil_layerstruct= '23SL_3.5m'
use_init_interp = .true.

&clm_inparm
hist_empty_htapes = .true.
hist_fincl1 = 'TSOI', 'TSOI_ICE', 'FCH4', 'CH4PROD', 'ALT', 'ALTMAX', 'DSL', 'FCEV', 'FSH_G', 'FSH_V', 'GR', 'HEAT_CONTENT1', 'HEAT_CONTENT1_VEG', 'HEAT_C
ONTENT2', 'ICE_CONTENT1', 'TG', 'TSOI_10CM', 'TBOT', 'TOTCOLC', 'TOTCOLCH4', 'TOTCOLN', 'TSA', 'TSA_ICE', 'STORVEGC', 'STORVEGN', 'TSL', 'TV', 'WIND', 'SOILIC
E', 'SOILC_CHANGE', 'SOILC_vr', 'SOILN_vr', 'TOTECOSYSC', 'TOTECOSYSN', 'TOTSOMC', 'TOTSOMN', 'TOTVEGC', 'TOTVEGN', 'TLAI', 'GPP', 'CPOOL', 'NPP', 'TWS'
/

#copy the restart file from AD spin-up case into the run directory of your current branch case:
cd cases/spinup1degAD_Narval/rest/0885-01-01-00000/
cp * ../../../../1deg_SpinupAD_Narval_All_Variables/run/

#make sure "CONTINUE_RUN" and "GET_REFCASE" are turned off and false because when you do a branch run it shouldnt continue any run because you are putting refrence case for it and when you do that you need to put GET_REFCASE to false because you will copy all the restart files from rest directory of that previous run into your current run as you wish, so no need to have GET_REFCASE here.



./case.build

./case.submit --mail-user YOURMAIL --mail-type all
 
You must log in or register to reply here.
Top