Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

Cam stuck at the process "Creating new decomp"

bmduran

Brandon Duran
New Member
Hi,

I am trying to rerun a version of the CAM6 PPE on Derecho, incorporating a few changes to the model. To start, I have just been trying to get the correct CAM6 setup I need to build and run on Derecho. I recently was successful in building the model and submitting a case, but the case timed out after 12 hours. After looking at the various log files in my RUNDIR, I can't spot any errors, but I can't tell why the model is timing out. In the atm log file, the model seems to be stuck in the process "Creating new decomp". I found a prior post about this with no solution provided. I am only running for 1 month so the wallclock limit seems like it shouldn't be an issue! What might be causing this?

What version of the code are you using?
I am using the tag cam6_3_026, which is the version for the CAM6 PPE. I am nudging winds to MERRA-2, picking a random start date of January 2019. The general configuration I am using was previously run successfully on Cheyenne. Minor changes were made to a few files in the src/physics/cosp2/ directory. I set DEBUG=TRUE in hopes of getting more information on this issue.

Caseroot: /glade/work/bduran/PPE-try/PI_ppe_cases_no_nudge/PPE_ensemble_PI-scratch/PPE_ensemble_PI-scratch.001
Rundir: /glade/derecho/scratch/bduran/PPE_ensemble_PI-scratch.000/run001/
Cesmroot: /glade/derecho/scratch/bduran/ppe-tracked

The script I have used to generate the ensemble members to test is : /glade/u/home/bduran/ppe/my_PI_run_script.py

The atm log is attached below, as well as the version of the script used to generate the ensemble members.

I hope that this may just be a configuration / P-E layout issue.

Thanks!
 

Attachments

  • log.zip
    220.2 KB · Views: 0

peverley

Courtney Peverley
Moderator
Hi,

Apologies for the delay. Are you still having this problem?

If so, can you provide an update on anything new you have discovered to help with debugging?
If not, can you post your solution so others can see?

Thanks!
Courtney
 
Top