iCESM 1.2: segmentation fault with F1850C5 compset

[mpb20@st-andrews_ac_uk]

Hi there,

I am trying to get iCESM1.2 up and running on a new HPC system called "hypatia". The machine config files are attached and my compiler is gnu/14.2.0.

I followed the instructions here and cloned the source code using the following command:

git clone GitHub - NCAR/iCESM1.2: Isotope-enabled CESM1.2

I am trying to run a tracer simulation with the F1850C5 compset:

./create_newcase -case lnd-trcr-f19-1850-fsst_cam5_1node_mpiexec_v6 -res f19_g16 -compset F1850C5 -compiler gnu -mach hypatia

I then change some xml files to (i) enable water tracing and (ii) setup as a hybrid run using restart files from here:

./xmlchange CAM_CONFIG_OPTS="-phys cam5 -water_tracer h2o_h216o_hdo_h218o -water_tag_num 1"
./xmlchange RUN_TYPE=hybrid
./xmlchange RUN_REFCASE=b.ie12.B1850C5CN.f19_g16.LME.002
./xmlchange RUN_STARTDATE=0001-01-01
./xmlchange START_TOD=0
./xmlchange RUN_REFDATE=1850-01-01
./xmlchange GET_REFCASE=FALSE

Before building, I make some changes to user_nl_cam, user_nl_cice, user_nl_rtm and user_nl_clm to enable water tracing (user_nl_* files attached) and copy atm_comp_mct.F90 to the SourceMods directory (as per the iCESM1.2 instructions). I then build the case and submit to the queue.

The error occurs during the run and shows up as a segmentation fault in cesm.log (all log files attached):

Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:

Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:

Interestingly, when I repeat all the steps above except copying atm_comp_mct.F90 to SourceMods, the case runs without error.

Any ideas on what might be the issue here? Has anyone else running iCESM1.2 encountered a similar issue? Given the case runs fine without including the modified atm_comp_mct.F90, it seems to be the water tracer functionality that is causing my problem.

Many thanks,

Mike

 

[mpb20@st-andrews_ac_uk]

P.S. According to the atm.log, the last action before the run crashed was to read-in ozone data:

READ_NEXT_TRCDATA ozone