Porting CIME to Greene NYU

[pperezhogin]

Hello,

I am working on porting CESM (and in particular CIME) to a cluster Greene in NYU.

I downloaded the code from github and checked out to version cesm2_3_beta17, and successfully ran ./manage_externals/checkout_externals.

Our machine (Greene) works with Intel compilers and slurm. Here is how we commonly set up compilers:
module purge
module load cmake/3.28.0 intel/19.1.2 openmpi/intel/4.0.5
module unload netcdf
module load netcdf-fortran/intel/4.5.3 netcdf-c/intel/4.7.4 hdf5/intel/1.12.0

As a result, the intel compilers and netcdf can be found here:
/share/apps/intel/19.1.2/bin/{icc,icpc, ifort}
/share/apps/openmpi/4.0.5/intel/bin/{mpicc, mpicxx, mpifort}
/share/apps/netcdf-c/4.7.4/intel/
/share/apps/netcdf-fortran/4.5.3/intel

I created config_machines.xml, config_batch.xml, greene.cmake and config_compilers in $HOME/.cime/, see attached files.

However, when I try to pass tests in CIME, the intel compilers and netcdf-c library are not used, resulting in Errors
./scripts_regression_tests.py test_sys_build_system.TestBuildSystem --machine greene --debug --no-teardown
as it follows from log file (error and log file are attached):
-- The Fortran compiler identification is GNU 11.3.1
-- The C compiler identification is GNU 11.3.1
-- Found NetCDF: -L/share/apps/netcdf-fortran/4.5.3/intel/lib -lnetcdff
-- Could NOT find NetCDF (missing: NetCDF_C_LIBRARIES)


As suggested by Jim Edwards, I checked .env_mach_specific.sh in the case directory, and it uploads modules which I expect.

So, how to set preferences for cmake to pick intel compilers instead of gcc, and how to include an additional netcdf path because we have two (/share/apps/netcdf-c/4.7.4/intel/, /share/apps/netcdf-fortran/4.5.3/intel)?

Thanks for any advice,
Pavel Perezhogin

 

[jedwards]

It's not that icc should be in the bin directory, it's that the icc directory should precede the /usr/bin in your path so that it is found
before gcc.

 

[pperezhogin]

Files I was supposed to attach in previous message are here.

 

[pperezhogin]

Thanks, Jim

I checked my $PATH. It seems that intel precedes the /usr/bin:
source .env_mach_specific.sh
echo $PATH
/share/apps/hdf5/1.12.0/intel/bin:/share/apps/intel/19.1.2/bin:/share/apps/netcdf-c/4.7.4/intel/bin:/share/apps/netcdf-fortran/4.5.3/intel/bin:/share/apps/openmpi/4.0.5/intel/bin:/share/apps/cmake/3.28.0/bin:/home/pp2681/.local/bin:/home/pp2681/bin:/share/apps/singularity/bin:/share/apps/local/bin:/scratch/pp2681/vscode_folder/.vscode-server/cli/servers/Stable-38c31bc77e0dd6ae88a4e9cc93428cc27a56ba40/server/bin/remote-cli:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/lpp/mmfs/bin:/opt/slurm/bin:/home/pp2681/MOM6-examples/FRE-NCtools/installed/bin

 

[pperezhogin]

Update.

Some limited success was achieved with including lines to config_machiens.xml:
<environment_variables>
<env name="CC">icc</env>
<env name="CXX">icpc</env>
<env name="FC">ifort</env>
</environment_variables>

Now intel compilers are seen by the cmake and I can compile the case mentioned above (cprnc executable). The next issue appeared with compiling gptl with error:
In file included from /scratch/pp2681/my_cesm_sandbox/cime/CIME/non_py/src/timing/gptl.c(9):
/usr/include/stdlib.h(565): error: attribute "__malloc__" does not take arguments
__attr_dealloc_free;

This likely related to the version of intel compiler and probably can be resolved only with NYU HPC team, which I contacted