t85 restart run problem

[klaus_wyser@smhi_se]

Has anybody ever succeeded in using the restart files from the t85 run for the 1990 control climate (b30.009) that are available from the EGS portal?

We have compiled and successfully run a startup run with CCSM in t85 configuration. We then tried to start a branch run from a set of restart files that we downloaded from the EGS portal. The run stopped when CAM was trying to read the restart file (end of atm.log):

(GETFIL): attempting to find local file b30.009.cam2.r.0510-01-01-00000
(GETFIL): using b30.009.cam2.r.0510-01-01-00000 in current working directory
(OPNFIL): Successfully opened file b30.009.cam2.r.0510-01-01-00000 on unit=
2
READ ioerror 67 on i/o unit = 2
ENDRUN: called without a message string

We tried 3 different sets of restart files, same result. We had a similar problem already with t42, where some restart runs would start while others would end with the same type of error. We noticed that some of the t42 restart files (from b30.004) are larger than others, and the larger ones would work while the smaller ones wouldn't. Could it be that the larger ones contained more variables/fields?

Further digging revealed that the smaller, non-working t42 restart files originate from a run with ccsm_3_0_beta09 while the larger and working files are from version ccsm_3_0_beta14. The t85 restart files are all from ccsm_3_0_beta08!

Our guess is that something has happened during the development of CCSM that destroyed the backward compatibility of the restart capability. Possibly, the newer version saves additional fields, and the restart tries to read these variables that are not present in the old files. Could it be that way? If yes, is there a workaround so that the t85 restart files can be used? For t42 we can use restart files after year 592, that's when the version change occurred, but for t85 we have no option.

Kind regards,
Klaus

 

[mvertens]

I believe that the problem you are encountering is that the T85 run was actually performed with an older version of the code base that produced only roundoff level differences with the released code base but was not backwards compatible in terms of CAM/CLM restart files. The way to get around this problem is to do a hybrid run (using CAM/CLM netcdf initial files) rather than a branch run (using CAM/CLM binary restart files). See the User's Guide
http://www.ccsm.ucar.edu/models/ccs...sGuide/node7.html#SECTION00074200000000000000
for instructions on how to perform a hybrid run.

 

[klaus_wyser@smhi_se]

Thanks for the feedback, it kind of confirms what I was supposing. I have already thought of a hybrid run, the only problem is that no cam/clm*.i.*nc are included in the restart tar files on the ESG portal, so this option is no option at this time. If NCAR could make the initial (*.nc) files available that would be great help.

Klaus

 

[kauff]

The b30.009 CCSM3.0 control run used a pre-release version of the code. The restart files created during the control run are incompatible with the released CCSM3.0 code (in particular, the CAM restart files), thus one cannot do a "branch" or "restart" run from control run output data using released code. (A similar problem may exist for part or all of other CCSM3.0 control runs.)

It is possible, however, to do a "hybrid" run from control run output data using released code.

All the data necessary to do a hybrid run was created during the model run and exists on NCAR's mss, but the original "restart tar files" (created during the model run and dated circa 2004) found on the mss and also on the ESG don't contain the clm & cam "init" files necessary to do a hybrid run. "Restart tar files" merely duplicate/collect restart data that exists elsewhere (in various different directories) into one handy tar file.

I created new restart tar files (dated March 2006) which also contain the init files necessary to do the hybrid run.
These new files now exist on NCAR's mss in mss:/CCSM/csm/b30.009/restart.tars
and will soon appear on the ESG.

Note:
Unlike branch or restart runs, hybrid runs cannot be "bit-for-bit identical" to the original run. That said, there are other reasons why eventually bit-for-bit identical runs generally are not possible even when doing branch or restart runs, for example:
o) a compiler upgrade results in different executable code
o) library upgrades result in different executable code
o) a particular computer is no longer available