Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

Problem running a branch run

Dear CESM Team,I am trying to run a branch run that begins 1 hour after the beginning of the startup run and I am getting an error which I have been unable to fix. ------------------------------------------------------------------------- CCSM PRESTAGE SCRIPT STARTING - CCSM input data directory, DIN_LOC_ROOT_CSMDATA, is /export/scratch/cesm/inputdata - Case input data directory, DIN_LOC_ROOT, is /export/scratch/cesm/inputdata/usr_created_inputdata/EMbranch - Checking the existence of input datasets in DIN_LOC_ROOT The following files were not found, this is informational onlyInput Data List Files Found:/export/scratch/cesm/cases/0.1T1Nv1/Buildconf/clm.input_data_list/export/scratch/cesm/cases/0.1T1Nv1/Buildconf/docn.input_data_list/export/scratch/cesm/cases/0.1T1Nv1/Buildconf/cam.input_data_list/export/scratch/cesm/cases/0.1T1Nv1/Buildconf/cice.input_data_list/export/scratch/cesm/cases/0.1T1Nv1/Buildconf/cpl.input_data_listFile status unknown: /export/scratch/cesm/output/0.1T1Nv1/atm/hist/0.1T1Nv0.cam2.h0.00000-01-01-03600.nc   CCSM PRESTAGE SCRIPT HAS FINISHED SUCCESSFULLY------------------------------------------------------------------------- The code is running and the simulation finishes successfully but the changes I wish to implement to the branch run are not implemented. I am certain of this because the results of the startup run and the branch run are idential. I have placed clm.input_data_list, docn.input_data_list, cam.input_data_list, cice.input_data_list, cpl.input_data_list, 0.1T1Nv0.cam2.h0.00000-01-01-03600.nc iboth in the DIN_LOC_ROOT and the RUNROOT folders but they seem not to be recognised by the program. Would you happen to know what the problem might be and how it could be solved?  Thank you very much in advance.
 

mmills

CSEG and Liaisons
Staff member
The output of the build script that you post looks normal to me. Sometimes the script reports files that are missing, but this can usually be ignored. In your case, perhaps the list of missing files is actually empty, as the next line reports files that were found.You mention changes you wish to implement ot the branch run that are not implemented. Are these namelist changes? My limited understanding of branch runs is that that is their main purpose. The procedure for implementing namelist changes varies depending on which version of CESM you are using. If the changes involve more than just namelist changes, you may need to rebuild the model.You may have more luck getting a response from someone more familiar with branch runs if you post under a general CESM forum, rather than a WACCM forum. Please include the version of the model you are running and your motivation for doing a branch run.
 

santos

Member
I would agree with Mike that you should ask this kind of question in the general CESM forums; I'm probably the most familiar with this script, out of the people who regularly check the WACCM forums, but I was on vacation, so you would have gotten a response sooner in the general forums.The output of this script is a little deceptive. The input_data_list files that it specifies were all found. The file "0.1T1Nv0.cam2.h0.00000-01-01-03600.nc" was the only thing not found, with status "unknown", but that usually only means that it is not in the inputdata directory; it's frequently not a problem at all, especially when you have an absolute path like this.Whatever the problem with your branch run is, it must be something else. Can you post the compset you were using, the results you were expecting, and the changes you made that failed to get those results?
 

santos

Member
P.S.: I intend to change the scripts so that this output is not so confusing in the future. So it is good to get feedback like this.
 
Hello Santos. Sorry I did not reply earlier, I was in a summer school with no internet connection.I am using the F_2000_WACCM compset. My intent was to increase the temperature of a specific area of the atmosphere every hour for 23 hours. I throught that the best way to achieve this was to run a startup run for 1 hour (starting at 0 UT) and then create a branch run that would use as input the restart files from the startup run (the run starts at 1 UT). So I setup the branch run in the same way that is mentioned in the user's manual, created the new case as branch run case, moved the restart files to the buildrun/run folder of the branch run, increased the temperature in the region I wanted and run the model. Then for the branch run starting at 2 UT the input was the restart files of the first branch run (I followed the same procedure that I used before). I repeated this procedure for branch runs starting at 3 UT - 23 UT. To ensure that the branch runs were working, I run a startup run, where the temperature was increased only once at 0 UT and then I let it run for 24 hours. After comparing the results from the two simulations (by plotting the global temperature are different altitudes for 0 UT - 24 UT) (24 plots), I found that they were the same. I hope this description was better than my last one.
 

santos

Member
Hmm, that is an interesting way to do this. I suppose I have two suggestions:
  1. If you have to increase the heating rates in this way, I would make sure that you are increasing the temperature in the right place. Only the restart files are used in a branch run, so any change you make to the initial condition file ("ncdata") will be ineffective. 
  2. However, I would do this differently. I would use SourceMods, by writing a short routine that produces a heating tendency for the relevant part of the atmosphere, and then adding a line to physpkg.F90 to call that routine (in tphysac or tphysbc, perhaps). This requires a little bit more familiarity with CAM's infrastructure, but it shouldn't be too hard to use the state components (e.g. state%lon) to determine if a grid cell is in the region you want to heat, and then add the heating to a physics tendency (ptend%s). The main benefit of this is that it would make it easier to reproduce what you're doing in future runs, since you wouldn't have to keep modifying restart files and branching the model.
 
Thank you very much for your advice Mr Santos. I will try it and see if it works.For the record I did make changes on the restart files but it still seems to not be working. One question. you mentioned that any changes that I make to the ncdata file will be ineffective. Does that mean this file is not used during the branch run? If not what is the purpose of listing it in the namelist file?Also what I really want to do is to have a cooling perturbation moving over the surface of the planet for 24 hours. So the perturbation is supposed to be moving. Changing the temperature every hour was just the best way I could think of doing it. Is there perhaps a better way to do it?
 

santos

Member
"Does that mean this file is not used during the branch run? If not what is the purpose of listing it in the namelist file?"Yes, actually, this file is not used. I believe that it is only present in the namelist because the CAM build-namelist script does not actually know whether or not your run is a restart or branch run, so ncdata is always added just in case.It's a bit puzzling why your modifications to the restart file would not work. You might want to double check that the file you're using actually is the one that's modified, of course, and that you've modified the right variable (I believe that this is PT, potential temperature?). During a restart, your log file should have some lines that look like this example, reading a file called "rpointer.atm": (OPNFIL): Successfully opened file ./rpointer.atm on unit=  92
 (GETFIL): attempting to find local file camrun.cam.r.1995-01-01-07200.ncI think that this should be in the atm.log, but it might be cpl.log. In any case, the file listed in the "GETFIL" line should match the name of whatever file you actually modified.If you don't modify the restart file, and use SourceMods instead, you can probably create the cooling perturbation in the physics by checking state%lat, state%lon for horizontal location, state%zm or state%pmid for vertical location (altitude or pressure), and by using the functions in time_manager.F90 for time (e.g. get_curr_time/get_curr_date). I'm assuming in this case that you have a predetermined course that you want the cooling perturbation to follow.
 
Dear Mr. Santos,I have checked the atm log, and the line that you mentioned is there. I have attached the log file because it is too big to post. However it still does not seem to be working.Beth
 

santos

Member
Hmm. Would you mind checking something else in the log? There are lines that look something like this in the log: nstep, te    87600   0.33229085352065673E+10   0.33229162183499045E+10   0.42580594829066774E-03   0.98525560976894893E+05The first number (87600) is the current timestep, whereas the other numbers relate to the global energy budget. Can you take two of the runs that you expected to be different, and look at the end of the atm.log for the last line that looks like this, in each run's log file? If those numbers are different, then your changes did take effect, but were too small for you to see on your plot.(We have a more sophisticated tool for comparing netCDF output, "cprnc", but this method is probably easier just to see if the output is the same.)I should point out that the system is really not designed for doing things this way; we actually make very few guarantees about the format of the restart files or what will happen if you change them with an external tool. That's part of why I suggested the SourceMods method.
 
Dear Mr. Santos,I apologize for not replying sooner. I was in a conference with no access to my code. I checked the lines as you instructed me and indeed they appear to be different by a small amount. So it seems that indeed the chages do take place. Since you informed me that the code is not made to operate in such a way, I will attempt to do it the way you suggest. Could you please provide me with some more information about how to do it with Sourcemodes? You mentioned writing a routine that produces a cooling tendency. Could you elaborate on it a little more? I am afraid my familiarity with CAM's infrastructure. Or could you perhaps point me in the direction of the manuals containing such information? Thank you very much for all your help so far.Beth
 

santos

Member
I believe that there's some documentation for this, but I'll have to check or remember where it is. In the meantime, I'll give the ten minute explanation.SourceMods is a directory in your case directory. If you put new or modified source code files in SourceMods/src.cam, they will be picked up during the CAM build. So if you have your own routine, you can put it there, as well as a modified version of physpkg to call it (original is in models/atm/cam/src/physics/cam/physpkg.F90).CAM's physics infrastructure is defined in physics_types (models/atm/cam/src/physics/cam/physics_types.F90). The basic formula for a physics parameterization in CAM is to write a routine that accepts a physics_state object (which has information about the state of a certain number, a "chunk", of columns), and provides a physics_ptend (which contain the tendencies that a parameterization wants to apply on a chunk). (There is also the physics buffer, which you probably do not need to get into in this simple case).One of the simpler examples of a routine that affects the physics is qbo_relax (models/atm/cam/src/physics/waccm/qbo.F90). I'm not going to get into too much detail about the code here, but if you look in the qbo module, you'll see a line that looks like this:
Code:
call physics_ptend_init(ptend, state%psetcols, 'qbo', lu=.true.)
That creates a "ptend" object, for the "qbo" process, that affects the zonal component of wind. Then qbo_relax sets ptend%u, which is the rate of change per second that will be applied. To change temperature, you want a heating rate, i.e. a tendency on dry static energy, so you would want to set "ls=.true." instead of lu, and then put the tendency in ptend%s.qbo_relax outputs ptend for physpkg, which applies it using a routine called physics_update:
Code:
call physics_update(state, ptend, ztodt, tend)
qbo_relax also accepts the state object (type "physics_state"). state%lat and state%lon are horizontal coordinates in radians, while components like state%pmid and state%zm provide vertical coordinates. You can get timing information from the routines in time_manager (models/atm/cam/src/utils/time_manager.F90).(I don't think that this sytem is too complex after a short time getting used to it, but it did seem much simpler before I started trying to explain it! Looking at some CAM routines might be more helpful, if you can filter out most of the details of the code and just focus on how interactions with state and ptend.)
 
Thank you very much for your detailed instructions Mr. Santos. I will try to see if I can make it work. I have just one more question. If someone wanted to change the albedo, would one need to follow the same procedure?
 

santos

Member
Hmm. It wouldn't be exactly the same; we get surface albedo from the coupler, so this is not a quantity generated directly within CAM itself. Depending on what you wanted to do and why, you might be able to intercept this quantity in the coupler interface, (in models/atm/cam/src/cpl_mct, or cpl_esmf). CAM actually accepts four such albedos, for shortwave and longwave, and for diffuse and "direct" (specular?) reflection.
 
Top