Model run failing during base vs rest comparison, missing hist files

[jonwells04]

Hi Jim,

Our HPC changed linux versions and we've recompiled all our needed packages. We had CESM generally running last year but I haven't picked this up in a while. After a bunch of chasing down errors I'm stuck with an issue during base vs rest comparison at the end of model runs for both CESM2.2 and the most recent version of E3SM (confiurations.txt shows my config). I have "./create_test ERS.f19_g16_rx1.A.wind_gnu --wait" completing CREATE_NEWCASE, XML, SETUP, SHARELIB_BUILD, MODEL_BUILD, and RUN for both CESM and E3SM but failing due to missing hist files (shown in CESM TestStatus.txt attached).

I rebuilt cprnc with the following commands from the README with no change:
CIMEROOT=../.. ../configure --macros-format=Makefile --mpilib=mpi-serial
CIMEROOT=../.. source ./.env_mach_specific.sh && make

I don't really have an error to trace down besides the missing hist files. Both base and rest files exist in the $case/run/ folder. I'm guessing I've done something incorrect in the config or cprnc build. Any guidance on next steps would be greatly appreciated.

Thanks!
Jon

 

[jonwells04]

Whelp! I figured it out. cprnc was not defined properly in the config_machines.xml

I gave the path to the cprnc folder but not to the cprnc executable. Everything is now working. Thanks!

 

[jedwards]

The error is not that the history files were not found - that is expected. The error is that the cpl history comparison between the runs did not match.
The cprnc results can be found in file /scratch/jw2636/cesmoutput/ERS.f19_g16_rx1.A.wind_gnu.20220808_170825_b23nsf/run/ERS.f19_g16_rx1.A.wind_gnu.20220808_170825_b23nsf.cpl.hi.0001-01-12-00000.nc.base.cprnc.out

 

[jonwells04]

Thank you for the clarification!

Rebuilding cprnc and correctly calling the cprnc executable path in config_machines.xml fixed the issue.