CAM-chem run error

[James King]

Hi all,

I'm trying to set up a run on Cheyenne with a custom compset: HIST_CAM60%CCTS2_CLM50%BGC-CROP_CICE%PRES_DOCN%DO1M_MOSART_CISM2%NOEVOLVE_SWAV.
Running create_newcase, case_setup, and case_build all work fine. The only change I made was to set JOB_WALLCLOCK_TIME to 00:20:00. However, when attempting to run the standard 5 day test, I run into an error:


2021-10-22 07:57:21: model execution error
ERROR: Command: 'mpiexec_mpt -p "%g:" -np 384 omplace -tm open64 /glade/scratch/jamesking/baseline/bld/cesm.exe >> cesm.log.$LID 2>&1 ' failed with error '' from dir '/glade/scratch/jamesking/baseline/run'
---------------------------------------------------
2021-10-22 07:57:21: case.run error
ERROR: RUN FAIL: Command 'mpiexec_mpt -p "%g:" -np 384 omplace -tm open64 /glade/scratch/jamesking/baseline/bld/cesm.exe >> cesm.log.$LID 2>&1 ' failed


In the cesm log file, the error is:

-1:MPT ERROR: MPI_COMM_WORLD rank 177 has terminated without calling MPI_Finalize()

It looks like the error arises in CAM, as the atm log file ends with:

open_trc_datafile:
/glade/p/cesmdata/cseg/inputdata/atm/cam/tracer_cnst/tracer_cnst_halons_3D_L70_
1849-2015_CMIP6ensAvg_c180927.nc
trcdata_init: file%has_ps = T
O3 is not an invariant
ERROR: tracer_cnst_init


Any ideas as to what might have broken CESM2 on this occasion? Log files in full are attached.

Cheers,

James

 

[919265375]

open_trc_datafile:
/glade/p/cesmdata/cseg/inputdata/atm/cam/tracer_cnst/tracer_cnst_halons_3D_L70_
1849-2015_CMIP6ensAvg_c180927.nc
trcdata_init: file%has_ps = T
O3 is not an invariant
ERROR: tracer_cnst_init
I also have this error. Have you solved this problem? Would you tell me how to solve this? Thank you very much.

 

[James King]

Hi! As you can see I posted this a few years ago so off the top of my head I'm afraid I can't remember how I fixed it. In general it seems this error is related to the initialisation of the input files for CAM-Chem, so make sure that your input files include the time periods over which you are running. You might also want to check 'CAM_NML_USE_CASE' to see if this is what you expect for your compset.

 

[919265375]

Thank you so much for your prompt response. I've checked my 'CAM_NML_USE_CASE' and changed it to what I expect. Now my case can continue to run!