Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

failed getting 'mosart.rh0.' file when continue_run=TRUE

A

Asiya

Asiya1a2
New Member
Hi all,

When I run an FHIST simulation with Resubmit > 0, it crashes with the error:

Reading restart dataset
(GETFIL): attempting to find local file
f.e22.geotrace.F1850.ne0np4.SAm.VR28.ne30x4.midHolo.006.mosart.rh0.0502-05-01-0
0000.ncA ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ0¢
A ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ0
¢A ^RüNÉ0¢A ^RüNÉ0
(GETFIL): failed getting file from full path:
./f.e22.geotrace.F1850.ne0np4.SAm.VR28.ne30x4.midHolo.006.mosart.rh0.0502-05-01
-00000.ncA ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ
0¢A ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ0¢A ^RüNÉ0¢A ^RüN
É0¢A ^RüNÉ0¢A ^RüNÉ0

Please find the log file here -- /glade/derecho/scratch/asiyab/f.e22.geotrace.F1850.ne0np4.SAm.VR28.ne30x4.midHolo.006/run/rof.log.7795009.desched1.250219-022049

The simulation runs fine when Resubmit = 0 or for the first month (n_stop=1), but it crashes in the second month of a branch/hybrid simulation when CONTINUE_RUN = TRUE. The mosart.rh0.* file exists in the run directory and is not corrupted. If I run a new case using the same file, it runs successfully for that month but crashes in the next.

I found this thread suggesting this might be a library issue. I tested running a single month after modifying PIO_TYPENAME for compclass="ROF" in env_run.xml to netcdf and it doubled the model cost. My variable-resolution simulation with water tags (/glade/u/home/asiyab/cesm2.2.0.geotrace) saw computational expense rise from ~83,978.82 to 164,422.25 pe-hrs/simulated year.

I’d greatly appreciate any suggestions on resolving this issue without a substantial increase in model cost.

Thanks!
 
Sort by date Sort by votes
nusbaume

nusbaume

Jesse Nusbaumer
CSEG and Liaisons
Staff member
Hi Asiya,

I am moving this post to the MOSART/RTM forum, as they have the ROF component experts who have the best chance at knowing how to remove those spurious extra characters in the "getfil" call without having to significantly increase the model cost.

Hope that helps, and have a great day!

Jesse
 
Vote Upvote 1 Downvote
O

oleson

Keith Oleson
CSEG and Liaisons
Staff member
Unfortunately, I don't have any suggestions for this. I tried a 2-month simulation at 0.9x1.25 with a restart at 1 month using this code and it worked fine:

/glade/work/marcusl/CESM_model_versions/cesm2.2.0.geotrace

Pinging @erik , @slevis , in case they have any suggestions.
 
Vote Upvote 1 Downvote
You must log in or register to reply here.
Top