Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

Using interpinic to set up a simulation which includes the land ice model

Hi.  I would like to run some CESM1.2 simulations which include the land ice model so that I can have mass balance terms in the output.  I am working on Yellowstone, and would like to use a compset like B1850C5CN, but which includes the land ice model.  I’ve defined this new compset in config_compsets.xml as:1850_CAM5_CLM40%CN_CICE_POP2_RTM_CISM1_SWAVWhen running, I want to use the default initial conditions for the B1850C5CN compset but, as I understand it, I have to use the command "interpinic" to regrid the “*clm2.r*” file before it will work with this new compset.  I have a few questions about this process (forgive me if these are overly basic questions):1.     Are these the most up-to-date instructions for doing this: http://www.cesm.ucar.edu/models/cesm1.2/clm/models/lnd/clm/doc/UsersGuide/x11704.html?  Two other sources of information are the interpinic instructions in the land ice model user guide (http://www.cesm.ucar.edu/models/cesm1.2/cism/) and the interpinic README file, and each mentions doing things a little differently (e.g. the user guide says to change the “override_missing” variable to false; the README says to use NetCDF4.3 when building).2.     Once I do this, I have a new “*clm2.r*” file.  How do I tell the model to use this file along with the default initial conditions for the B1850C5CN compset when starting new simulations?Thanks.Michael Erb
 

sacks

Bill Sacks
CSEG and Liaisons
Staff member
Hi Michael,Regarding question 1: If you do 'module load netcdf' on yellowstone, you should now get netcdf 4.3 by default, so that piece is no longer an issue. For this particular use case, you DO want to follow the instructions in the CISM user's guide - specifically, setting override_missing to false. (That isn't included in the other instructions, because the main time that is used is when interpolating from a non-glc_mec run to a glc_mec run, as you are doing here.) If there are other specific discrepancies you're not sure about, I'm happy to clarify those, too.Regarding question 2: You can point to the new clm2.r* file by adding the following line in your user_nl_clm file (which will be in your case directory after you run cesm_setup):finidat = '/path/to/your/new/file.nc'Also: The new glacier columns will take something like 100 years or somewhat more to spin-up before they will start giving robust surface mass balance values. So you have two choices:(a) Discard the first 100+ years of your BG simulation. This is the simplest to do if you have the computer time to burn, or if you were planning to discard a large number of years of your initial simulation anyway to allow for spin-up time.(b) Run about 120 years of an IG compset (CLM with CISM, forced by a data atmosphere) first to get a somewhat spun-up state for glacier columns. Then blend this with the existing initial conditions for B1850C5CN using interpinic. Then you can throw away significantly fewer years of the BG simulation. If you would like to do (b), I can provide more details on how to do this.
 
Thanks, Bill.  I used interpinic to make the regridded surface dataset, but when I started up a run, I got an error in the cesm log file saying:* FATAL ERROR : (glimmer_ncio.F90:349) Error opening file b40_1850_2d_r07c5cn_160jp.cism.r.0070-01-01-00000.nc: No such file or directoryI had tried to start a hybrid run using all of the initial datasets that the B1850C5CN compset would use, plus my new "*clm2.r*" file.  I don't have the CISM file that the error mentions.  I looked in my env_run.xml file, and here are the lines related to CISM:
   


   


   


   Do I need to set that "CISM_OBSERVED_IC" variable to true, or should I be looking somewhere else for the solution?Michael
 

sacks

Bill Sacks
CSEG and Liaisons
Staff member
Hi Michael,Ah, yes, you do need to set CISM_OBSERVED_IC to true in this case. Sorry, the documentation of that probably is not as clear as it could be. Basically: When you're doing a hybrid run, CESM will try to find initial conditions for all components based on the REFCASE. But in this case, the REFCASE didn't have initial conditions for CISM, so you need to tell it not to try to find initial conditions from the REFCASE, but to instead use the out-of-the-box, observed initial conditions for CISM. You can do that by setting CISM_OBSERVED_IC to TRUE.
 
Thanks Bill.  That solved the problem.  A question about specifying the new "clm2.r" file: You mentioned above that I should add the finidat command to my user_nl_clm file to point to the new surface dataset, so I did that.  I ran the model for a month and it seems to work fine.  However, in my run directory, all of the initial restart files (populated after building the model) appear to point to the B1850C5CN defaults, and not to my new file.  The CESM user guide says:"It is important to note that runs that are initialized as branch or hybrid runs, will require restart/initial files from previous model runs (as specified by the variables, $RUN_REFCASE and $RUN_REFDATE). These required files must be prestaged by the user to the case $RUNDIR (normally $EXEROOT/run) before the model run starts. This is normally done by just copying the contents of the relevant $RUN_REFCASE/rest/$RUN_REFDATE.00000 directory."So do I need to copy that "clm2.r" file into the run directory directly (simply using the "cp" command)?  I did some tests, and the model runs if I do that, but the output is different from the first method I mentioned above (just adding the finidat command).  If I neither copy the surface dataset nor use the finidat command, the model fails with exit code 255.
 

sacks

Bill Sacks
CSEG and Liaisons
Staff member
Hi Michael,Your comment made me realize that I am not 100% certain of the operation of finidat when doing a hybrid case (i.e., I am not 100% certain that CLM listens to finidat in this case). That said, if your BG case runs at all, then I am pretty sure that it is picking up the correct file - i.e., the file you generated with interpinic. If you tried to run the model with the original restart file from the refcase, the model should die in initialization, and you should see a message in your lnd.log file to the effect of "wrong number of landunits" (or columns or pfts).I don't understand some of your specific questions, though:However, in my run directory, all of the initial restart files (populated after building the model) appear to point to the B1850C5CN defaults, and not to my new file.I'm not sure what you mean by this. If you point me to a run directory that exhibits this, I could answer it more definitively. do I need to copy that "clm2.r" file into the run directory directlyNo: As long as you set finidat to point to the full path of the file, it doesn't matter if this file sits in your run directory or somewhere else.I did some tests, and the model runs if I do that, but the output is different from the first method I mentioned above (just adding the finidat command).This is disturbing to me, but I don't fully understand the setup you are describing of the two different cases. If you'd like more input / advice on this, can you point me to the two different case directories and run directories that result in different output?
 
Hi Bill,To explain, I tried two different cases which I thought would produce identical results.  Case 1 (testing_model_finidat): I specified the "clm2.r" file with the finidat command, like you suggested.  Case 2 (testing_model_cp): I copied the "clm2.r" file directly into the run directory before running.  I ran both cases for a month, and both produce output, but are not the same.If you want to take a look, my process on Yellowstone is as follows:---cd /glade/u/home/merb/startruns./initial_simulation_for_interpinic.sh    # This runs a simulation for one month to produce an output file to regrid the surface data onto.cd /glade/u/home/merb/interpinic./run_interpinic.sh                              # Running interpinic.  I also have a txt file in this directory which contains the screen output from running this script.  It mentions that some variables were skipped or not found.  Is this a problem?cd /glade/u/home/merb/startruns./new_simulation.sh                           # This runs the simulation.  Both the finidat command the the cp command (toward the end) in this script are currently commented, but I uncomment one or the other depending on the case I'm running.  There are text files for the screen output for this script as well.---Directories (for reference):Case directory: /glade/u/home/merb/cases/Run directory: /glade/scratch/merb/Archive: /glade/scratch/merb/archive/The two experiments are called "testing_model_finidat..." and "testing_model_cp..."Two other things I tried (just to test):  1) Using both the finidat and cp commands resulted in the same output as just using the cp command.  2) Using neither command led to exit code 255.By the way, should I be running a hybrid case for this?  I'm still not completely clear about the difference between hybrid and branch simulations, so would there be any reason to run a branch simulation instead?  I chose hybrid because it seemed to be the default when using the B1850C5CN compset.Thanks for the help.
 

sacks

Bill Sacks
CSEG and Liaisons
Staff member
Hi Michael,Thanks a lot for your careful documentation of this. I'll admit, though, that I'm still at a bit of a loss as to why you're seeing differences here.That said: I have gotten confirmation that the correct way to do this is to set finidat via user_nl_clm - and it shouldn't matter if the file referenced by finidat is in your run directory or somewhere else. In particular, I found out that the contents of the rpointer.lnd file are irrelevant for a hybrid run.If you want to confirm that your case is reading the restart file properly, you can search for 'reading restart file ' in your lnd.log file.I'm still confused why your two cases give different answers. As I understand it, they are identical, except that in the cp case, finidat points to a file in your run directory that you have named to be the same as the original from the refcase (you can see that finidat value in your lnd_in file in the run directory); whereas in the 'finidat' case, you have changed the name and path of this file. If that's truly the only difference, then I cannot see why the two runs would differ. But again, setting finidat via user_nl_clm is the accepted way to do this. Regarding your other questions:It is normal to see messages about variables not found or being skipped during interpinic.A hybrid case is appropriate here. I always have trouble remembering when to use a hybrid vs branch myself, but I think branch runs are more appropriate if you are trying to restart exactly from a previous case, but with some tweaked namelist settings. In your case, you are modifying restart files (and adding CISM), so I think hybrid is more appropriate.
 
Hi Bill,I think I've narrowed down the difference between the two methods I tried. When I diff the "rof" logfiles for the two simulations, one logfile (when I simply copied the "clm2.r" file into the run directory) complains that it can't find eight variables, such as RTM_VOLR_LIQ. Then, later in the logfile, the value for its first "budget check" is 0, rather than a specific value. It looks to me like it can't find those variables, so it starts with values of zero, which affects the whole simulations through the butterfly effect (though hopefully the climatologies are the same).I've attached screenshots to show the difference between these two "rof" logfiles. The method where I copied the "clm2.r" file into the run directory is on the left. The method where I used finidat is on the right. There are differences in other logfiles, but nothing that seemed particularly relevant.The variables the logfile mentions are the same ones that interpinic couldn't find, with messages like this when I ran interpinic:"Variable NOT found on output file: RTM_VOLR_LIQ"I’m guessing that with the finidat method, those variables (e.g. RTM_VOLR_LIQ) are read from the original “clm2.r” file (the one that gets used with the B1850C5CN compset). In the “cp” method, that original file gets copied over, so the model starts with zeros for those variables.So, my question: when I run the initial simulation (which I use for interpinic), should the model be producing those eight variables (e.g. RTM_VOLR_LIQ)? Presently it doesn’t.Michael
 

sacks

Bill Sacks
CSEG and Liaisons
Staff member
Hi Michael,Great detective work! I think I finally understand what's going on.The background to this is that RTM was split out from CLM a few years ago, to be its own component. This means that it now has its own restart files, namelist input files, etc. However, the model version that produced the initial conditions that you're using had RTM as a sub-component of CLM, so the RTM restart variables were on the CLM restart file.So, to answer your last question: It is expected that newer versions of CLM do NOT produce those 8 variables, since they are now produced by RTM, on its own restart file.As for the behavior you are seeing: I agree with your assessment. To restate it: You are using a setup that is kind of in limbo between the old setup (RTM restart variables on CLM's restart file) and the new setup (RTM as its own component, with its own restart files and namelist input files). So, for this simulation, RTM is expecting to find restart variables on the original CLM restart file (taken directly from the refcase). When you do the "cp" method, you overwrote that restart file that RTM was expecting to use with one that did not have any data for the RTM restart variables.So this reinforces the recommendation to use the "finidat" method rather than the "cp" method, and now provides the rationale for why.
 
Great, I'll use the finidat method.I ran a simulation for a year to see what ice flux data I would get.  In the glc history file, output variables (such as "acab") are only present over Greenland.  Is it possible to output "acab" over Antarctica?
 

sacks

Bill Sacks
CSEG and Liaisons
Staff member
Hi Michael,No, you currently cannot get downscaled acab over Antarctica, since CISM isn't running over Antarctica. However, you should be able to see these variables on the CLM grid (separated by elevation class) in the coupler history file. Look for variables with 'x2g' in their name. Let me know if you have trouble finding the right ones.
 

sacks

Bill Sacks
CSEG and Liaisons
Staff member
Currently, it is not possible to have CISM run in Antarctica. That is something that we're working towards.
 
Hi Bill,Regarding the accumlation/ablation variables you mentioned in the coupler history file, are there any besides the "X2G_FGSS_QICE##" variable at the 10 elevation classes?  Are any of the "G2X" variables important to determining accumulation/ablation?
 

sacks

Bill Sacks
CSEG and Liaisons
Staff member
Those are the only variables directly on accumulation/ablation. Sl_topo## tells you the topographic height of each elevation class in CLM, which is needed for the vertical interpolation in the downscaling routine (i.e., downscaling the CLM grid to the CISM grid); and Sl_tsrf## tells you the surface temperature (used as an upper boundary condition - not at all related to accumulation/ablation).None of the g2x variables are important for this purpose. For the code base you're using, those g2x variables are simply diagnostics that do not feed back to the rest of the system, and may be incorrect. (They are corrected and hooked into other components in the development code base, using CLM4.5.)
 
Hi Bill.  Thanks for all of the help so far.  Another question: I don't want the ice sheet to evolve as the model runs.  There are two variables (seen here: http://www.cesm.ucar.edu/models/cesm1.2/cesm/doc/modelnl/nl_cism.html) which sound related to this: "evolve_ice" and "evolution".  "evolve_ice=0" sounds like what I want, but I'm not sure what "evolution" does.  Do I need to make any changes to it?
 

sacks

Bill Sacks
CSEG and Liaisons
Staff member
Simply set evolve_ice = 0. You do not need to change "evolution".(In older model versions this was done by setting evolution = -1, but that has been replaced by setting evolve_ice = 0. Now, if you set evolve_ice = 0, evolution is ignored.)
 
Top