Main menu

Navigation

CESM on Stampede2 (TACC)

1 post / 0 new
mdfowler@...
CESM on Stampede2 (TACC)

Hello all, 

 

I've been attempting to get CESM1.2.2 up and running on Stampede's KNL system, Stampede2, but we've run into a few issues. This seems to be architecture (or potentially compiler type and version) specific, as this model works on the NERSC Cori KNL system but not on Stampede. Unfortunately, we are not able to back-migrate compilers or try other versions as they are not available on this system.

 

 

We are running at a resolution of two degrees in the B_1850-2000_CN compset. The main error message being produced now is the following, which seems to stem from the ice_transport_remap.F90 file: 

forrtl: severe (154): array index out of bounds

Image              PC                Routine            Line        Source

cesm.exe           0000000001D7BFC9  Unknown               Unknown  Unknown

libpthread-2.17.s  00002AFFE1144370  Unknown               Unknown  Unknown

cesm.exe           0000000000E3815B  ice_transport_rem         678  ice_transport_remap.F90

cesm.exe           0000000000E205E8  ice_transport_dri         549  ice_transport_driver.F90

cesm.exe           0000000000DF90D6  ice_step_mod_mp_s         679  ice_step_mod.F90

cesm.exe           0000000000C987E3  ice_comp_mct_mp_i         631  ice_comp_mct.F90

cesm.exe           0000000000417E59  ccsm_comp_mod_mp_        3248  ccsm_comp_mod.F90

cesm.exe           0000000000439142  MAIN__                     91  ccsm_driver.F90

cesm.exe           000000000041467E  Unknown               Unknown  Unknown

libc-2.17.so       00002AFFE1674B35  __libc_start_main     Unknown  Unknown

cesm.exe           0000000000414569  Unknown               Unknown  Unknown

This message was produced after almost 1.5 years during a long simulation, but was again produced after just 15 days when we re-submitted the same run script. It seems that there's a root problem that we're unaware of, but that is leading to unpredictable failures. 

 

We are using the Intel compiler (version 17.0.4), with the following flags supplied to the model in the config_compilers.xml: 

<compiler COMPILER="intel">

  <!-- http://software.intel.com/en-us/articles/intel-composer-xe/ -->

  <ADD_CPPDEFS> -DFORTRANUNDERSCORE -DNO_R16 -DCRMACCEL -DSTRATOKILLER </ADD_CPPDEFS>

  <ADD_CFLAGS compile_threaded="true"> -openmp </ADD_CFLAGS>

  <ADD_FFLAGS compile_threaded="true"> -openmp </ADD_FFLAGS>

  <ADD_LDFLAGS compile_threaded="true"> -openmp </ADD_LDFLAGS>

  <FREEFLAGS> -free </FREEFLAGS>

  <FIXEDFLAGS> -fixed -132 </FIXEDFLAGS>

  <ADD_FFLAGS DEBUG="TRUE"> -g -CU -check pointers -fpe0 -ftz </ADD_FFLAGS>

  <ADD_FFLAGS DEBUG="FALSE"> -O2 </ADD_FFLAGS>

  <FFLAGS> -no-opt-dynamic-align -fp-model precise -convert big_endian -assume byterecl -ftz -traceback -assume realloc_lhs </FFLAGS>

  <CFLAGS> -O2 -fp-model precise </CFLAGS>

  <FFLAGS_NOOPT> -O0 </FFLAGS_NOOPT>

  <FC_AUTO_R8> -r8 </FC_AUTO_R8>

  <SFC> ifort </SFC>

  <SCC> icc </SCC>

  <SCXX> icpc </SCXX>

  <MPIFC> mpif90 </MPIFC>

  <MPICC> mpicc  </MPICC>

  <MPICXX> mpicxx </MPICXX>

  <CXX_LINKER>FORTRAN</CXX_LINKER>

  <CXX_LDFLAGS> -cxxlib </CXX_LDFLAGS>

  <SUPPORTS_CXX>TRUE</SUPPORTS_CXX>

</compiler>

 

Any insight would be much appreciated. 

Best,

Meg Fowler

Who's new

  • rory
  • dangcheng111@...
  • yuxisuo2020@...
  • bryn.ronalds@...
  • brookeadams@...