Hi all,
We just ported CESM2.2.0 to our hpc system. I did a startup run with FHIST_BGC compset to test the model. The case was successfully compiled but failed with the following weird error in cesm log
---------------------------------------------------------------------------------------------------------------------
ENDRUN:
ERROR: ERROR in initInterpMindist.F90 at line 296
ERROR initInterp set_mindist: Cannot find any input points matching output poin
t:
subgrid level, index = pft 791949
lat, lon = -1.426676552148964E+229 , -1.426676552148964E+229
ltype: 1
ctype: 1
ptype: 0
---------------------------------------------------------------------------------------------------------------------
The lnd log tells me the run crashed during the "set_mindist" procedure
---------------------------------------------------------------------------------------------------------------------
(GETFIL): using
/storage/home/hcoda1/8/klu82/pace_storage/CESM_INPUTS/2.2.0/lnd/clm2/initdata_m
ap/clmi.BHIST.2000-01-01.0.9x1.25_gx1v7_simyr2000_c200728.nc
**** Mapping clm initial data from input
/storage/home/hcoda1/8/klu82/pace_storage/CESM_INPUTS/2.2.0/lnd/clm2/initdata_m
ap/clmi.BHIST.2000-01-01.0.9x1.25_gx1v7_simyr2000_c200728.nc to output
finidat_interp_dest.nc ****
input gridcells = 21013 output gridcells = 21013
input landuntis = 50862 output landunits = 62100
input columns = 141928 output columns = 590024
input pfts = 199889 output pfts = 884206
Glacier elevation classes same in input and output?: T
ipft_not_vegetated = 0
icol_vegetated_or_bare_soil = 1
ilun_vegetated_or_bare_soil = 1
ilun_crop = 2
ilun_landice_multiple_elevation_classes = 4
create_glacier_mec_landunits = true
finding minimum distance for pfts
calling set_subgrid_info for pft for input
calling set_subgrid_info for pft for output
calling set_mindist for pft
---------------------------------------------------------------------------------------------------------------------
I have checked that the input data "clmi.BHIST.2000-01-01.0.9x1.25_gx1v7_simyr2000_c200728.nc" exists and not corrupted.
I did a same run on cheyenne and didn't get any errors. When I compare the "finidat_interp_dest.nc" between the successful and failed runs, I realized that the model installed on our school HPC is not generating the correct "finidat_interp_dest.nc".
We are using the following modules:
intel/20.0.4
hdf5/1.10.8
mvapich2/2.3.6
perl/5.34.1
netcdf-fortran/4.5.4
netcdf-c/4.8.1
netcdf-cxx/4.2
We have checked that all the dependencies are correctly installed under intel/20.0.4 and the netcdf paths are setup correctly. Our job scheduler is slurm and the mpi lib is mvapich2. After consulting the HPC staff, we think this could be the issue with the compilers since the values of the latitude/longitude displayed in the log make no sense. We don't know if intel/20.0.4 is suitable for CESM2. We have ported CESM2 successfully on our system with intel/17 compiler and PBS scheduler. However, the system has been upgraded and we have to use a more recent compiler.
Thanks,
Melody
We just ported CESM2.2.0 to our hpc system. I did a startup run with FHIST_BGC compset to test the model. The case was successfully compiled but failed with the following weird error in cesm log
---------------------------------------------------------------------------------------------------------------------
ENDRUN:
ERROR: ERROR in initInterpMindist.F90 at line 296
ERROR initInterp set_mindist: Cannot find any input points matching output poin
t:
subgrid level, index = pft 791949
lat, lon = -1.426676552148964E+229 , -1.426676552148964E+229
ltype: 1
ctype: 1
ptype: 0
---------------------------------------------------------------------------------------------------------------------
The lnd log tells me the run crashed during the "set_mindist" procedure
---------------------------------------------------------------------------------------------------------------------
(GETFIL): using
/storage/home/hcoda1/8/klu82/pace_storage/CESM_INPUTS/2.2.0/lnd/clm2/initdata_m
ap/clmi.BHIST.2000-01-01.0.9x1.25_gx1v7_simyr2000_c200728.nc
**** Mapping clm initial data from input
/storage/home/hcoda1/8/klu82/pace_storage/CESM_INPUTS/2.2.0/lnd/clm2/initdata_m
ap/clmi.BHIST.2000-01-01.0.9x1.25_gx1v7_simyr2000_c200728.nc to output
finidat_interp_dest.nc ****
input gridcells = 21013 output gridcells = 21013
input landuntis = 50862 output landunits = 62100
input columns = 141928 output columns = 590024
input pfts = 199889 output pfts = 884206
Glacier elevation classes same in input and output?: T
ipft_not_vegetated = 0
icol_vegetated_or_bare_soil = 1
ilun_vegetated_or_bare_soil = 1
ilun_crop = 2
ilun_landice_multiple_elevation_classes = 4
create_glacier_mec_landunits = true
finding minimum distance for pfts
calling set_subgrid_info for pft for input
calling set_subgrid_info for pft for output
calling set_mindist for pft
---------------------------------------------------------------------------------------------------------------------
I have checked that the input data "clmi.BHIST.2000-01-01.0.9x1.25_gx1v7_simyr2000_c200728.nc" exists and not corrupted.
I did a same run on cheyenne and didn't get any errors. When I compare the "finidat_interp_dest.nc" between the successful and failed runs, I realized that the model installed on our school HPC is not generating the correct "finidat_interp_dest.nc".
We are using the following modules:
intel/20.0.4
hdf5/1.10.8
mvapich2/2.3.6
perl/5.34.1
netcdf-fortran/4.5.4
netcdf-c/4.8.1
netcdf-cxx/4.2
We have checked that all the dependencies are correctly installed under intel/20.0.4 and the netcdf paths are setup correctly. Our job scheduler is slurm and the mpi lib is mvapich2. After consulting the HPC staff, we think this could be the issue with the compilers since the values of the latitude/longitude displayed in the log make no sense. We don't know if intel/20.0.4 is suitable for CESM2. We have ported CESM2 successfully on our system with intel/17 compiler and PBS scheduler. However, the system has been upgraded and we have to use a more recent compiler.
Thanks,
Melody