Error trying to build cprnc

C

cristian-vasile.achim

Cristian Achim
New Member
Hello,

I am trying to compile the `cprnc` component but I am getting stuck at the configure step. this is what I am doing:

Code: 
$ export COMPILER=gnu
$ export MPILIB=openmpi
$ CIMEROOT=../.. ../configure --macros-format=Makefile --mpilib=openmpi
ERROR: module command /usr/share/lmod/lmod/libexec/lmod python load cmake perl/5.34.1 python-data parallel-netcdf/1.12.2 netcdf-c/4.8.1 netcdf-fortran/4.5.3 failed with message:
Lmod has detected the following error: These module(s) or extension(s) exist but
cannot be loaded as requested: "netcdf-c/4.8.1", "parallel-netcdf/1.12.2",
"netcdf-fortran/4.5.3"
   Try: "module spider netcdf-c/4.8.1 parallel-netcdf/1.12.2
netcdf-fortran/4.5.3" to see how to load the module(s).

I tried
Code: 
module spider netcdf-c/4.8.1
[\CODE]
 and I get this:
[CODE]
$ module spider  netcdf-c/4.8.1

------------------------------------------------------------------------------
  netcdf-c: netcdf-c/4.8.1
------------------------------------------------------------------------------

    You will need to load all module(s) on any one of the lines below before the "netcdf-c/4.8.1" module is available to load.

      aocc/3.2.0  mpich/4.0.1
      gcc/11.2.0  mpich/4.0.1
      gcc/11.2.0  openmpi/4.1.2
      gcc/11.2.0  openmpi/4.1.2-cuda
      gcc/9.4.0  openmpi/4.1.2
[\CODE]

The modules module gcc/11.2.0 and openmpi/4.1.2 are by default loaded on a new terminal.

Cristian
 
C

cristian-vasile.achim

Cristian Achim
New Member
jedwards said:
cprnc is a serial program - you can only use mpilib=mpi-serial to build it.
Click to expand...

Thank you for your reply.

I have in the config_compilers this entry:
Code: 
<!-- ******************* MAHTI ********************* -->
<compiler MACH="mahti" COMPILER="gnu">
 <CFLAGS>
    <base> -std=gnu99 </base>
    <append compile_threaded="true"> -fopenmp </append>
 </CFLAGS>
 <FFLAGS>
    <append compile_threaded="true">  -fopenmp </append>
</FFLAGS>
 <LDFLAGS>
    <append compile_threaded="true"> -fopenmp </append>
 </LDFLAGS>

 <NETCDF_PATH>/projappl/project_2008521/cristian/netcfd-gnu</NETCDF_PATH>
 <PNETCDF_PATH>/appl/spack/v017/install-tree/gcc-11.2.0/parallel-netcdf-1.12.2-f3ctm3</PNETCDF_PATH>
 <SLIBS>
   <append> -L$(NETCDF_PATH)/lib -lnetcdff -lnetcdf -Wl,-rpath=$(NETCDF_PATH)/lib -Wl,-rpath=$(PARALLEL_NETCDF_INSTALL_ROOT)/lib</append>
  </SLIBS>
  <ESMF_LIBDIR>/fmi/datasets/CESM-INPUT/esmf710r/lib/libO/Linux.intel.64.openmpi.default</ESMF_LIBDIR>
</compiler>

<!-- ***************** END *************************** -->
And in the config_machines another entry
Code: 
  <!--  MAHTI -->
    <machine MACH="mahti">
      <DESC>Atos BullSequana XH2000, 256 pes/node (two 128-core processors) and 256 GB of memory/node, batch system is SLURM</DESC>
      <OS>LINUX</OS>
      <COMPILERS>gnu</COMPILERS>
      <MPILIBS>openmpi</MPILIBS>
      <PROJECT>project_2008521</PROJECT>
      <CHARGE_ACCOUNT>project_2008521</CHARGE_ACCOUNT>
      <CIME_OUTPUT_ROOT>/scratch/project_2008521/$USER</CIME_OUTPUT_ROOT>
      <DIN_LOC_ROOT>/scratch/project_2008521/$USER/CESM-INPUT/inputdata</DIN_LOC_ROOT>
      <DIN_LOC_ROOT_CLMFORC>/scratch/project_2008521/$USER/CESM-INPUT/inputdata/atm/datm7</DIN_LOC_ROOT_CLMFORC>
      <DOUT_S_ROOT>/scratch/project_2008521/archive/$CASE</DOUT_S_ROOT>
      <BASELINE_ROOT>/projappl/project_2008521/CESM-INPUT/cime/cesm_baselines</BASELINE_ROOT>
      <CCSM_CPRNC>/projappl/project_2008521/cristian/CESM-INPUT/cime/tools/cprnc/cprnc</CCSM_CPRNC>
      <GMAKE_J>8</GMAKE_J>
      <BATCH_SYSTEM>slurm</BATCH_SYSTEM>
      <SUPPORTED_BY>cristian at csc, monika at fmi</SUPPORTED_BY>
      <MAX_TASKS_PER_NODE>256</MAX_TASKS_PER_NODE>
      <MAX_MPITASKS_PER_NODE>256</MAX_MPITASKS_PER_NODE>
      <PROJECT_REQUIRED>TRUE</PROJECT_REQUIRED>
      <mpirun mpilib="default">
          <executable>srun</executable>
          <arguments>
              <arg name="num_tasks" > -n {{ total_tasks }}</arg>
              <arg name="partition">-p medium</arg>
          </arguments>
      </mpirun>
      <module_system type="module" allow_error="true">
     <init_path lang="perl">/usr/share/lmod/lmod/init/perl</init_path>
     <init_path lang="python">/usr/share/lmod/lmod/init/env_modules_python.py</init_path>
     <init_path lang="csh">/usr/share/lmod/lmod/init/csh</init_path>
     <init_path lang="sh">/usr/share/lmod/lmod/init/sh</init_path>
     <cmd_path lang="perl">/usr/share/lmod/lmod/libexec/lmod perl</cmd_path>
     <cmd_path lang="python">/usr/share/lmod/lmod/libexec/lmod python</cmd_path>

     <cmd_path lang="csh">module</cmd_path>
     <cmd_path lang="sh">module</cmd_path>
     <modules>
        <command name="purge"/>
         <command name="load">cmake</command>
         <command name="load">perl/5.34.1</command>
         <command name="load">python-data</command>
         <command name="load">gcc/11.2.0</command>
         <command name="load">openmpi/4.1.2</command>
         <command name="load">parallel-netcdf/1.12.2</command>
         <command name="load">netcdf-c/4.8.1</command>
         <command name="load">netcdf-fortran/4.5.3</command>
     </modules>
     </module_system>

     <environment_variables>
      <env name="OMP_STACKSIZE">256M</env>
      <env name="MPI_TYPE_DEPTH">16</env>
      <env name="ESMFMKFILE">/fmi/datasets/CESM-INPUT/esmf710r/lib/libO/Linux.intel.64.openmpi.default/esmf.mk</env>
     </environment_variables>
     <resource_limits>
        <resource name="RLIMIT_STACK">-1</resource>
     </resource_limits>
</machine>
<!--  END MAHTI -->

With the above entries I did a ../configure --clean and rerun. Now I get different errors (but at least finds the netcfd library :)):
Code: 
$ export MPILIB=mpi-serial
[cristian@mahti-login12 cprnc]$ ../configure --clean
Removing file Macros.make
Removing file env_mach_specific.xml
Removing file .env_mach_specific.sh
Removing file .env_mach_specific.csh
Removing file Depends.gnu
[cristian@mahti-login12 cprnc]$ CIMEROOT=../.. ../configure --macros-format=Makefile --mpilib=mpi-serial
Setting resource.RLIMIT_STACK to -1 from (8388608, -1)
[cristian@mahti-login12 cprnc]$ CIMEROOT=../.. source ./.env_mach_specific.sh && make
The following modules were not unloaded:
  (Use "module --force purge" to unload all):

  1) csc-tools
gfortran  -o cprnc compare_vars_mod.o cprnc.o filestruct.o prec.o utils.o -L/projappl/project_2008521/cristian/netcfd-gnu/lib -lnetcdff   -L/projappl/project_2008521/cristian/netcfd-gnu/lib -lnetcdff -lnetcdf -Wl,-rpath=/projappl/project_2008521/cristian/netcfd-gnu/lib -Wl,-rpath=/appl/spack/v017/install-tree/gcc-11.2.0/parallel-netcdf-1.12.2-f3ctm3/lib
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: warning: libmfhdf.so.0, needed by /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so, not found (try using -rpath or -rpath-link)
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: warning: libdf.so.0, needed by /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so, not found (try using -rpath or -rpath-link)
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `SDend'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `SDselect'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `SDreaddata'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `SDdiminfo'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `H5Pset_dxpl_mpio'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `SDreadattr'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `SDgetdimid'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `SDendaccess'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `H5Pset_fapl_mpio'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `H5Pset_all_coll_metadata_ops'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `SDgetinfo'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `SDattrinfo'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `SDstart'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `SDgetchunkinfo'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `SDfileinfo'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `H5Pset_coll_metadata_write'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `SDgetfillvalue'
collect2: error: ld returned 1 exit status
make: *** [Makefile:77: cprnc] Error 1
 
J

jedwards

CSEG and Liaisons
Staff member
You've hard coded the path to a parallel build of netcdf and pnetcdf. You need a serial build of netcdf (and no pnetcdf) to
link with cprnc. So in config machine you should only load the openmpi/4.1.2 and parallel-netcdf/1.12.2 if openmpi is the mpi-library:

<modules mpilib="openmpi">
<command name="load">openmpi/4.1.2</command>
<command name="load">parallel-netcdf/1.12.2</command>
</modules>
Hopefully your module system is set up so that if openmpi is not loaded the serial version of netcdf is loaded.

Also in config_compilers.xml the following two lines should refer to env variables and not to hardcoded paths.

<NETCDF_PATH>/projappl/project_2008521/cristian/netcfd-gnu</NETCDF_PATH>
<PNETCDF_PATH>/appl/spack/v017/install-tree/gcc-11.2.0/parallel-netcdf-1.12.2-f3ctm3</PNETCDF_PATH>

do module show netcdf-c/4.8.1 to see the env variables set by the module.
 
C

cristian-vasile.achim

Cristian Achim
New Member
Thank very much for taking to help me.

Our machine is configured so that the netcdf modules (netcdf-fortran and netcdf-c) are dependent on gcc/11 and openmpi. This is why I created a folder projappl/project_2008521/cristian/netcfd-gnu and I linked the two include and lib folders into this folder .

This is what I did first:
Code: 
ln -s $NETCDF_FORTRAN_INSTALL_ROOT/lib/* /projappl/project_2008521/cristian/netcfd-gnu/
ln -s $NETCDF_C_INSTALL_ROOT/lib/* /projappl/project_2008521/cristian/netcfd-gnu/lib/
ln -s $NETCDF_C_INSTALL_ROOT/include/* /projappl/project_2008521/cristian/netcfd-gnu/include/
ln -s $NETCDF_FORTRAN_INSTALL_ROOT/include/* /projappl/project_2008521/cristian/netcfd-gnu/include/

This is the reason why the path <NETCDF_PATH> is hardcoded.

Cristian
 
C

cristian-vasile.achim

Cristian Achim
New Member
Follow up:


I just noticed that the variables NETCDF_PATH and PARALLEL_NETCDF_INSTALL_ROOT need to be set. I just did it, however after losing my connection to server I started to get the error
Code: 
$ ../configure  --clean
ERROR: Invalid compiler vendor given in COMPILER environment variable: gnu
[CODE]
Inspecting the Makefile and the Macros.make I noticed that I only have the option ifeq ($(COMPILER),intel) NO ifeq ($(COMPILER),gnu). 

Cristian
 
C

cristian-vasile.achim

Cristian Achim
New Member
Sorry about the too many messages I was in the wrong folder.

I fixed this and I tried to set the environment variables I do still get the same errors:


Code: 
gfortran  -o cprnc compare_vars_mod.o cprnc.o filestruct.o prec.o utils.o -L/projappl/project_2008521/cristian/netcfd-gnu/lib -lnetcdff   -L/projappl/project_2008521/cristian/netcfd-gnu/lib -lnetcdff -lnetcdf -Wl,-rpath=/projappl/project_2008521/cristian/netcfd-gnu/lib -Wl,-rpath=/appl/spack/v017/install-tree/gcc-11.2.0/parallel-netcdf-1.12.2-f3ctm3/lib
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: warning: libmfhdf.so.0, needed by /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so, not found (try using -rpath or -rpath-link)
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: warning: libdf.so.0, needed by /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so, not found (try using -rpath or -rpath-link)
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `SDend'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `SDselect'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `SDreaddata'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `SDdiminfo'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `H5Pset_dxpl_mpio'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `SDreadattr'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `SDgetdimid'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `SDendaccess'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `H5Pset_fapl_mpio'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `H5Pset_all_coll_metadata_ops'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `SDgetinfo'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `SDattrinfo'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `SDstart'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `SDgetchunkinfo'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `SDfileinfo'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `H5Pset_coll_metadata_write'
/appl/spack/v017/install-tree////////////////////////////////////gcc-8.5.0/binutils-2.37-ed6z3n/bin/ld: /projappl/project_2008521/cristian/netcfd-gnu/lib/libnetcdf.so: undefined reference to `SDgetfillvalue'
 
C

cristian-vasile.achim

Cristian Achim
New Member
For other people that might have to so something similar. It seems I created wrong the symbolic links to the netcdf. Once I used the paths of the netcdf-fortran module the compilation worked.

Cristian
 
You must log in or register to reply here.
Back
Top