Where is the output?

[jihwang@colorado_edu]

Dear All, I am new to CESM, so please forgive me for the dumb question. I am running CESM1.2.2 on NERSC Edison. I followed the instruction in Quick Start, and tweaked it a little in order to get familiar with the model. I used the most common $COMPSET=B1850CN, $RES=f19_g16, $MACH=edison (I am running it on edison), and $CASE=$SCRATCH/testcase1. I created the new case. Then I modified, HIST_OPTION=nday and HIST_N=1 in "env_run.xml", as well as added, nhtfrq=-24 and mfilt=1 in "user_nl_cam", in order to see some output.Everything seemed fine. I did cesm_setup and $CASE.build, and finally $CASE.submit. I checked the queue and saw the model running. I could even see the log files in run/ directory. However, there is just NO *.nc output files anywhere. I checked run/ directory, the so-called short-term history file folder, and I could find NOTHING.I thought maybe the model run was too long before it reached the one-day mark to produce any history output, so I changed to nhtfrq=1 in "user_nl.cam" to see the output at every time step. I still got NO history output anywhere.Did I do anything wrong here? I pretty much followed the Quick Start on User's Guide. I tried the run at least a dozen times, but still got no results.Any help will be appreciated. Thanks a lot. Best Regards,Aaron

 

[jedwards]

Hi Aaron,This all looks correct - Did you check the coupler log and the atm log files?   If you are writting history you should see messages there when you open and close the files.   Is short term archiving enabled?  Check the stdout, stderr and cesm.log for other output related messages.   Is this a hybrid or branch run? 

 

[jihwang@colorado_edu]

Hi Jedwards, Thanks for your response.I checked the atm.log.*, cesm.log.*, and cpl.log.* files in the run/ directory. I didn't see any open/close new files, although there were some "Opened existing file" for the input. One thing that kept bothering me in the past few days is that I sometimes see an error message for reading in topography data. The error message is in cesm.log.*. I just checked again. All the cesm.log.* files that I produced in the past two days have the same error message:------------------------ Opened existing file /project/projectdirs/ccsm1/inputdata/atm/cam/topo/USGS-gtopo30_1.9x2.5_remap_c0 50602.nc      131072 NetCDF: Invalid dimension ID or nameforrtl: error (78): process killed (SIGTERM)---------------- That makes me wonder whether the model was running after the read-in error. The short term archving is enabled in "env_run.xml" as default.--------------    -------------------------------- By the way, do I need to set "DOUT_S_SAVE_ALL_ON_DISK" to "TRUE"? It is a "hybrid" run as it comes with the package for edison as default. Is that file  "/project/projectdirs/ccsm1/inputdata/atm/cam/topo/USGS-gtopo30_1.9x2.5_remap_c050602.nc" the origin of all the problem? Many Thanks. Best Regards,Aaron

 

[jedwards]

I have a job in the queue on edison to try to reproduce your issue.   I'll let you know the result when it runs.

 

[jihwang@colorado_edu]

Hi, Jedwards,Thanks for your help.I just found that there is actually one output file for CAM2 and one output file for CLM in the run/ directory. They are b40.1850.track1.2deg.003.cam.h0.0500-12.nc, and b40.1850.track1.2deg.003.clm2.h0.0500-12.nc. I guessed I was too tired yesterday to see anything.The sad thing is those two output files have the data on day 182500, which is exactly the same time as the time in the initial file, even though I changed my output frequency to 1 timestep or 1 hour.I am hoping the problem is as simple as a damaged topography file. Many Thanks.Best Regards,Aaron

 

[jihwang@colorado_edu]

Hello, CESM Software Engineer,Have you had the chance to see how it went on edison? How was it?Many Thanks.Best Regards,Aaron

 

[ckuo@lbl_gov]

I am commenting because I had the exact same problem.  Other compset/grid combinations worked, but this one did not, giving the same error.  Some build script may have been old, however.  After completely updating the whole cesm1_2_2 directory, it is now working.