Out-of-box run fail

[hayesdev]

I'm trying an out-of-box CLM run, with the compset and resolution of

./create_newcase --case ~/OOB/clm_quickstart --mach lux-cpuq --res f09_g16_gl4 -compset I2000Clm50BgcCrop --run-unsupported

I'm using intel compiler (gnu compiler is old and would mess up other people's work to update) and port to a linux machine on my school's supercomputer.

I've done no alterations to the namelists or anything, just straight through setup, build, and submit.
When I submit, the run script fails and I can't figure out why. Not sure if this is an error with porting or something else going on. Any insight would be appreciated!

These are the versions of intel I'm using

ifort (IFORT) 19.0.5.281 20190815
icc (ICC) 19.0.5.281 20190815

and this is what shows up in my CaseStatus


When I check any of the logs, there's no mpirun error that I can find, but there are errors of
"ENDRUN:

ERROR: BandDiagonal ERROR: dgbsv returned error code

Intel MKL ERROR: Parameter 1 was incorrect on entry to DGBSV " throughout.


Not really sure where to go from here. I've attached all of the logs and config porting files. Thanks

Verison info:
------------------------------------------------------------------------
git describe:
cesm2.1.3-rc.01-0-g0596a97
------------------------------------------------------------------------

------------------------------------------------------------------------
git status:
# Not currently on any branch.
# Untracked files:
# (use "git add <file>..." to include in what will be committed)
#
# version_info.txt
nothing added to commit but untracked files present (use "git add" to track)
------------------------------------------------------------------------

------------------------------------------------------------------------
manage_externals status:
Processing externals description file : Externals.cfg
Processing externals description file : Externals_CLM.cfg
Processing externals description file : Externals_POP.cfg
Processing externals description file : Externals_CISM.cfg
Processing externals description file : Externals_CAM.cfg
Checking status of externals: clm, fates, ptclm, mosart, ww3, cime, cice, pop, cvmix, marbl, cism, source_cism, rtm, cam, clubb, carma, cosp2, chem_proc,
./cime
clean sandbox, on cime5.6.32
# HEAD detached at cime5.6.32
nothing to commit, working directory clean

./components/cam
clean sandbox, on cam_cesm2_1_rel_41
# HEAD detached at cam_cesm2_1_rel_41
nothing to commit, working directory clean

./components/cam/chem_proc
clean sandbox, on tools/proc_atm/chem_proc/release_tags/chem_proc5_0_03_rel

./components/cam/src/physics/carma/base
clean sandbox, on carma/release_tags/carma3_49_rel

./components/cam/src/physics/clubb
clean sandbox, on vendor_clubb_r8099_n03
# HEAD detached at vendor_clubb_r8099_n03
nothing to commit, working directory clean

./components/cam/src/physics/cosp2/src
clean sandbox, on CFMIP/COSPv2.0/tags/v2.1.4cesm/src

./components/cice
clean sandbox, on cice5_cesm2_1_1_20190321
# HEAD detached at cice5_cesm2_1_1_20190321
nothing to commit, working directory clean

./components/cism
clean sandbox, on cism-release-cesm2.1.2_02
# HEAD detached at cism-release-cesm2.1.2_02
nothing to commit, working directory clean

./components/cism/source_cism
clean sandbox, on release-cism2.1.03
# HEAD detached at release-cism2.1.03
nothing to commit, working directory clean

./components/clm
clean sandbox, on release-clm5.0.30
# HEAD detached at release-clm5.0.30
nothing to commit, working directory clean

./components/clm/src/fates
clean sandbox, on sci.1.30.0_api.8.0.0
# HEAD detached at sci.1.30.0_api.8.0.0
nothing to commit, working directory clean

./components/clm/tools/PTCLM
clean sandbox, on PTCLM2_20200121
# HEAD detached at PTCLM2_20200121
nothing to commit, working directory clean

./components/mosart
clean sandbox, on release-cesm2.0.04
# HEAD detached at release-cesm2.0.04
# Untracked files:
# (use "git add <file>..." to include in what will be committed)
#
# cime_config/__pycache__/
nothing added to commit but untracked files present (use "git add" to track)

./components/pop
clean sandbox, on pop2_cesm2_1_rel_n09
# HEAD detached at pop2_cesm2_1_rel_n09
# Untracked files:
# (use "git add <file>..." to include in what will be committed)
#
# externals/CVMix/
# externals/MARBL/
nothing added to commit but untracked files present (use "git add" to track)

./components/pop/externals/CVMix
clean sandbox, on v0.93-beta
# HEAD detached at v0.93-beta
nothing to commit, working directory clean

./components/pop/externals/MARBL
clean sandbox, on cesm2.1-n00
# HEAD detached at cesm2.1-n00
nothing to commit, working directory clean

./components/rtm
clean sandbox, on release-cesm2.0.04
# HEAD detached at release-cesm2.0.04
nothing to commit, working directory clean

./components/ww3
clean sandbox, on ww3_181001
# HEAD detached at ww3_cesm2_1_rel_01
nothing to commit, working directory clean

------------------------------------------------------------------------

 

[hayesdev]

I was able to resolve the problem by changing the config_compilers from
<append> -L/cm/shared/apps/intel/compilers_and_libraries/linux/mkl/lib/intel64/ -lmkl_intel_ilp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread -lm -ldl </append>
to
<append> -L/cm/shared/apps/intel/compilers_and_libraries/linux/mkl/lib/intel64/ -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread -lm -ldl </append>
using lp64 instead of ilp64

If anyone else runs into a similar issue, I hope this helps!

 

[erik]

Glad you were able to get this to work.