MPI error when using sat_hist in CESM2.2.2 on derecho

Hi all,

When I used release-cesm2.2.2 on derecho and set sat_hist in usr_nl_cam to output results along flight track, MPI error occurred: "MPI error (MPI_File_set_view) : Other I/O error , error stack: ADIO_Set_view(48): **iobadoverlap filetype is not permitted to contain overlapping regions". But when I used release-cesm2.1.5 with the same sat_hist setting, the model ran and wrote output file successfully.
Any suggestions?

Weiyi

Thanks for writing Weiyi.

Could you share your Derecho case directory and the path to the sathist_track_infile you're using?

Does the model on release-cesm2.2.2 run when sathist is off?

hplin said:

Thanks for writing Weiyi.

Could you share your Derecho case directory and the path to the sathist_track_infile you're using?

Does the model on release-cesm2.2.2 run when sathist is off?

Click to expand...

Hi Haipeng,

The case directory was overwritten, but it was a standard experiment (FHIST, f09/ne30) with sathist_tack_infile (/glade/u/home/weiyiwang/aircraft_profilelist_20180724_20180913_wecan_mrg10_c20210812.nc).

The release-cesm2.2.2 did run without sathist file.

Thanks Weiyi, that is strange. I wonder if it's an issue with the parallel I/O. Could you try ./xmlchange PIO_TYPENAME=netcdf to see if it helps?

hplin said:

Thanks Weiyi, that is strange. I wonder if it's an issue with the parallel I/O. Could you try ./xmlchange PIO_TYPENAME=netcdf to see if it helps?

Click to expand...

It works, Haipeng. Thanks!

hplin said:

Thanks Weiyi, that is strange. I wonder if it's an issue with the parallel I/O. Could you try ./xmlchange PIO_TYPENAME=netcdf to see if it helps?

Click to expand...

Hi Haipeng, it works for one-month run using PIO_TYPENAME=netcdf, but the model had trouble in I/O and failed to advance to next month (/glade/derecho/scratch/weiyiwang/csmruns/FHIST.e22.nudge.f09/run/atm.log.9918720.desched1.250630-105632)...

Hi

Please try PIO version 1 with PIO_TYPENAME=pnetcdf. This combination works for me for sathist in cesm2.2.2 on derecho.

./xmlchange PIO_VERSION=1