ESMF error when building CTSM with nuopc ("no specific subroutine for the generic 'esmf_xgridget')

[InneVanderkelen]

Dear CESM support,

I am trying to build the latest version of the CTSM master branch on our machine (piz daint; attached you can find the config_compilers.xml, config_machines.xml, and config_batch.xml files) using a gnu compiler and the nuopc driver, for a basic case (I2000Clm51Sp, f09_g17).

When building the following error occurs related to ESMF in the final cesm build step (cesm logfile attached):

Building cesm from /users/ivanderk/CCLM2/CTSM/components/cmeps/cime_config/buildexe with output to /scratch/snx3000/ivanderk/C
Error: There is no specific subroutine for the generic 'esmf_xgridget' at (1)

ERROR: BUILD FAIL: buildexe failed, cat /scratch/snx3000/ivanderk/CCLM2_cases/CTSMdev.gnu.I2000Clm51SpRs.f09_g17.glob.cclm2_20

I installed ESMF under the same compiler (esmf@8.1.1%gcc@9.3.0) without parallel netcdf and provide the following information before compiling ESMFMKFILE=/project/sm62/ivanderk/spack-install/cray-cnl7-haswell/gcc-9.3.0/esmf-8.1.1-34opubys2xhva2kwhqteuwif724qubhw/lib/esmf.mk

Could this error be due to due to the version of ESMF I am using?

Thank you in advance for the support,
Inne

 

[jedwards]

You need a much newer version of ESMF. The latest release is esmf 8.4.1.

 

[InneVanderkelen]

Thanks a lot, this helped me a few steps forward.

I installed esmf@8.4.1%gcc@9.3.0, and referenced to the new build in the ESMFMKFILE.
Now I get the following error in the cesm build log:

/usr/bin/ld: /project/s1207/ivanderk/spack-install/cray-cnl7-haswell/gcc-9.3.0/esmf-8.4.1-esftqomee2sllfsmjevw3f7cet6tbeb4/lib/libesmf.so: undefined reference to `PIOc_InitDecomp_ReadOnly'
ERROR: BUILD FAIL: buildexe failed, cat /scratch/snx3000/ivanderk/CCLM2_cases/CTSMdev.gnu.I2000Clm51SpRs.f09_g17.glob.cclm2_20230417-1702/bld/cesm.bldlog.230417-170238

As I read here that the PIOc_InitDecomp_ReadOnly function is added in the PIO version 2.5.8 (Release Version 2.5.8 · NCAR/ParallelIO (github.com)), I suppose it is a version issue related to the PIO library.

The following versions of the modules are currently included in the port of CTSM on my machine (pizdaint):
cray-netcdf-hdf5parallel/4.7.4.4
cray-hdf5-parallel/1.12.0.4
cray-parallel-netcdf/1.12.1.4

Is this the issue and do I need more recent versions? (And if so, which?)

Thank you in advance,
Inne

 

[jedwards]

When you installed ESMF 8.4.1 it used an internal copy of PIO 2.5.8 or newer, however ctsm is building and linking it's own copy
of an earlier version of PIO and then ESMF is using this library and generating a run time failure.

Here is a quick solution:

Update the ctsm Externals.cfg and change

[parallelio]
tag = pio2_5_7

to be:
[parallelio]
tag = pio2_5_10

then run ./manage_externals/checkout_externals parallelio
and rebuild your case.

 

[InneVanderkelen]

When you installed ESMF 8.4.1 it used an internal copy of PIO 2.5.8 or newer, however ctsm is building and linking it's own copy
of an earlier version of PIO and then ESMF is using this library and generating a run time failure.

Here is a quick solution:

Update the ctsm Externals.cfg and change

[parallelio]
tag = pio2_5_7

to be:
[parallelio]
tag = pio2_5_10

then run ./manage_externals/checkout_externals parallelio
and rebuild your case.

This solves the problem, thanks a lot!