problems in porting cesm1.02: csm_share build failed

[sebastian_gayler@uni-tuebingen_de]

Hello,

I am intending to run the CESM1.02 model in the CLM only mode on my linux pc (clm3.5 is running well). However, I have problems in building csm_share.
Can somebody help me? Many thanks in advance for your help.
Sebastian

Below some information about what I have done. I also attached mct.bldlog, pio.bldlog and the complete csm_share_bldlog

Machine=x86_64;
I use the intel composerXE-2011.1.107, openmpi-1.2.8 and netcdf-3.6.3;
openmpi and netcdf were both build with the same compiler.

First I created a new case following the instructions in the userguide:

./create_newcase -case Case.02 -res f19_g16 -compset X -mach generic_linux_intel -scratchroot /home/sebastian/prog/cesm1_0/cesm1_0_cases -din_loc_root_csmdata /home/sebastian/prog/cesm1_0/inputdata/ -max_tasks_per_node 1

Case.02 was succesfully configured and I made the following settings in the env_mach_specific file:

********************************************************************************
setenv INTEL_PATH /opt/intel/composerxe-2011.1.107
setenv MPICH_PATH /usr/local/openmpi-1.2.8
setenv PATH ${INTEL_PATH}/bin/intel64:${MPICH_PATH}/bin:${PATH}
setenv LD_LIBRARY_PATH ${INTEL_PATH}/compiler/lib/intel64:${LD_LIBRARY_PATH}
********************************************************************************


and in Macros.generic_linux_intel:

*******************************************************************
FC := ifort
CC := icc

#=====================================================
# GENERIC_USER

NETCDF_PATH := /usr/local/netcdf/netcdf-3.6.3_intel_composerXE-2011.1.107
INC_NETCDF := $(NETCDF_PATH)/include
LIB_NETCDF := $(NETCDF_PATH)/lib
MOD_NETCDF := $(NETCDF_PATH)/include
MPICH_PATH := /usr/local/openmpi-1.2.8
INC_MPI := $(MPICH_PATH)/include
LIB_MPI := $(MPICH_PATH)/lib
MPI_LIB_NAME := openmpi
PNETCDF_PATH :=
INC_PNETCDF :=
LIB_PNETCDF :=
LAPACK_LIBDIR :=
*********************************************************

With this settings, the build script crashes during building csm_share:

-------------------------------------------------------------------------
CCSM BUILDNML SCRIPT STARTING
- To prestage restarts, untar a restart.tar file into /home/sebastian/prog/cesm1_0_2/cesm1_0_2_cases/Case.02/run
- Create modelio namelist input files
CCSM BUILDNML SCRIPT HAS FINISHED SUCCESSFULLY
-------------------------------------------------------------------------
CCSM PRESTAGE SCRIPT STARTING
- CCSM input data directory, DIN_LOC_ROOT_CSMDATA, is /home/sebastian/prog/cesm1_0_2/inputdata
- Case input data directory, DIN_LOC_ROOT, is /home/sebastian/prog/cesm1_0_2/inputdata
- Checking the existence of input datasets in DIN_LOC_ROOT
CCSM PRESTAGE SCRIPT HAS FINISHED SUCCESSFULLY
-------------------------------------------------------------------------
CCSM BUILDEXE SCRIPT STARTING
- Build Libraries: mct pio csm_share
Mi 23. Feb 10:43:27 CET 2011 /home/sebastian/prog/cesm1_0_2/cesm1_0_2_cases/Case.02/mct/mct.bldlog.110223-104326
Mi 23. Feb 10:43:55 CET 2011 /home/sebastian/prog/cesm1_0_2/cesm1_0_2_cases/Case.02/pio/pio.bldlog.110223-104326
Mi 23. Feb 10:44:21 CET 2011 /home/sebastian/prog/cesm1_0_2/cesm1_0_2_cases/Case.02/csm_share/csm_share.bldlog.110223-104326
ERROR: csm_share.buildlib failed, see /home/sebastian/prog/cesm1_0_2/cesm1_0_2_cases/Case.02/csm_share/csm_share.bldlog.110223-104326
ERROR: cat /home/sebastian/prog/cesm1_0_2/cesm1_0_2_cases/Case.02/csm_share/csm_share.bldlog.110223-104326


First I get a lot of warning messages and later one the following error messages, e.g.:

csm_share.log:
...
icc -c -I. -I/usr/local/netcdf/netcdf-3.6.3_intel_composerXE-2011.1.107/include -I/usr/local/netcdf/netcdf-3.6.3_intel_composerXE-2011.1.107/include -I/usr/local/openmpi-1.2.8/include -I. -I/home/sebastian/prog/cesm1_0_2/cesm1_0_2_cases/Case.02/SourceMods/src.share -I/home/sebastian/prog/cesm1_0_2/models/drv/shr_mct -I/home/sebastian/prog/cesm1_0_2/models/drv/shr -I/home/sebastian/prog/cesm1_0_2/models/dead_share -I/home/sebastian/prog/cesm1_0_2/models/csm_share/shr -I/home/sebastian/prog/cesm1_0_2/models/utils/timing -I/home/sebastian/prog/cesm1_0_2/models/utils/esmf_wrf_timemgr -I/home/sebastian/prog/cesm1_0_2/cesm1_0_2_cases/Case.02/lib/include -DMCT_INTERFACE -DHAVE_MPI -DGLC_NEC_10 -DLINUX -DSEQ_ -DFORTRANUNDERSCORE -DNO_R16 -DNO_SHR_VMATH /home/sebastian/prog/cesm1_0_2/models/utils/timing/gptl_papi.c
ifort -c -I. -I/usr/local/netcdf/netcdf-3.6.3_intel_composerXE-2011.1.107/include -I/usr/local/netcdf/netcdf-3.6.3_intel_composerXE-2011.1.107/include -I/usr/local/openmpi-1.2.8/include -I. -I/home/sebastian/prog/cesm1_0_2/cesm1_0_2_cases/Case.02/SourceMods/src.share -I/home/sebastian/prog/cesm1_0_2/models/drv/shr_mct -I/home/sebastian/prog/cesm1_0_2/models/drv/shr -I/home/sebastian/prog/cesm1_0_2/models/dead_share -I/home/sebastian/prog/cesm1_0_2/models/csm_share/shr -I/home/sebastian/prog/cesm1_0_2/models/utils/timing -I/home/sebastian/prog/cesm1_0_2/models/utils/esmf_wrf_timemgr -I/home/sebastian/prog/cesm1_0_2/cesm1_0_2_cases/Case.02/lib/include -DMCT_INTERFACE -DHAVE_MPI -DGLC_NEC_10 -DLINUX -DSEQ_ -DFORTRANUNDERSCORE -DNO_R16 -DNO_SHR_VMATH -g -fp-model precise -convert big_endian -assume byterecl -ftz -traceback -O2 -FR /home/sebastian/prog/cesm1_0_2/models/utils/timing/perf_utils.F90
ifort -c -I. -I/usr/local/netcdf/netcdf-3.6.3_intel_composerXE-2011.1.107/include -I/usr/local/netcdf/netcdf-3.6.3_intel_composerXE-2011.1.107/include -I/usr/local/openmpi-1.2.8/include -I. -I/home/sebastian/prog/cesm1_0_2/cesm1_0_2_cases/Case.02/SourceMods/src.share -I/home/sebastian/prog/cesm1_0_2/models/drv/shr_mct -I/home/sebastian/prog/cesm1_0_2/models/drv/shr -I/home/sebastian/prog/cesm1_0_2/models/dead_share -I/home/sebastian/prog/cesm1_0_2/models/csm_share/shr -I/home/sebastian/prog/cesm1_0_2/models/utils/timing -I/home/sebastian/prog/cesm1_0_2/models/utils/esmf_wrf_timemgr -I/home/sebastian/prog/cesm1_0_2/cesm1_0_2_cases/Case.02/lib/include -DMCT_INTERFACE -DHAVE_MPI -DGLC_NEC_10 -DLINUX -DSEQ_ -DFORTRANUNDERSCORE -DNO_R16 -DNO_SHR_VMATH -g -fp-model precise -convert big_endian -assume byterecl -ftz -traceback -O2 -FR /home/sebastian/prog/cesm1_0_2/models/utils/timing/perf_mod.F90
ifort -c -I. -I/usr/local/netcdf/netcdf-3.6.3_intel_composerXE-2011.1.107/include -I/usr/local/netcdf/netcdf-3.6.3_intel_composerXE-2011.1.107/include -I/usr/local/openmpi-1.2.8/include -I. -I/home/sebastian/prog/cesm1_0_2/cesm1_0_2_cases/Case.02/SourceMods/src.share -I/home/sebastian/prog/cesm1_0_2/models/drv/shr_mct -I/home/sebastian/prog/cesm1_0_2/models/drv/shr -I/home/sebastian/prog/cesm1_0_2/models/dead_share -I/home/sebastian/prog/cesm1_0_2/models/csm_share/shr -I/home/sebastian/prog/cesm1_0_2/models/utils/timing -I/home/sebastian/prog/cesm1_0_2/models/utils/esmf_wrf_timemgr -I/home/sebastian/prog/cesm1_0_2/cesm1_0_2_cases/Case.02/lib/include -DMCT_INTERFACE -DHAVE_MPI -DGLC_NEC_10 -DLINUX -DSEQ_ -DFORTRANUNDERSCORE -DNO_R16 -DNO_SHR_VMATH -g -fp-model precise -convert big_endian -assume byterecl -ftz -traceback -O2 -FR /home/sebastian/prog/cesm1_0_2/models/csm_share/shr/shr_orb_mod.F90
ifort -c -I. -I/usr/local/netcdf/netcdf-3.6.3_intel_composerXE-2011.1.107/include -I/usr/local/netcdf/netcdf-3.6.3_intel_composerXE-2011.1.107/include -I/usr/local/openmpi-1.2.8/include -I. -I/home/sebastian/prog/cesm1_0_2/cesm1_0_2_cases/Case.02/SourceMods/src.share -I/home/sebastian/prog/cesm1_0_2/models/drv/shr_mct -I/home/sebastian/prog/cesm1_0_2/models/drv/shr -I/home/sebastian/prog/cesm1_0_2/models/dead_share -I/home/sebastian/prog/cesm1_0_2/models/csm_share/shr -I/home/sebastian/prog/cesm1_0_2/models/utils/timing -I/home/sebastian/prog/cesm1_0_2/models/utils/esmf_wrf_timemgr -I/home/sebastian/prog/cesm1_0_2/cesm1_0_2_cases/Case.02/lib/include -DMCT_INTERFACE -DHAVE_MPI -DGLC_NEC_10 -DLINUX -DSEQ_ -DFORTRANUNDERSCORE -DNO_R16 -DNO_SHR_VMATH -g -fp-model precise -convert big_endian -assume byterecl -ftz -traceback -O2 -FR /home/sebastian/prog/cesm1_0_2/models/drv/shr/seq_comm_mct.F90
ifort -c -I. -I/usr/local/netcdf/netcdf-3.6.3_intel_composerXE-2011.1.107/include -I/usr/local/netcdf/netcdf-3.6.3_intel_composerXE-2011.1.107/include -I/usr/local/openmpi-1.2.8/include -I. -I/home/sebastian/prog/cesm1_0_2/cesm1_0_2_cases/Case.02/SourceMods/src.share -I/home/sebastian/prog/cesm1_0_2/models/drv/shr_mct -I/home/sebastian/prog/cesm1_0_2/models/drv/shr -I/home/sebastian/prog/cesm1_0_2/models/dead_share -I/home/sebastian/prog/cesm1_0_2/models/csm_share/shr -I/home/sebastian/prog/cesm1_0_2/models/utils/timing -I/home/sebastian/prog/cesm1_0_2/models/utils/esmf_wrf_timemgr -I/home/sebastian/prog/cesm1_0_2/cesm1_0_2_cases/Case.02/lib/include -DMCT_INTERFACE -DHAVE_MPI -DGLC_NEC_10 -DLINUX -DSEQ_ -DFORTRANUNDERSCORE -DNO_R16 -DNO_SHR_VMATH -g -fp-model precise -convert big_endian -assume byterecl -ftz -traceback -O2 -FR /home/sebastian/prog/cesm1_0_2/models/drv/shr/seq_io_mod.F90
/home/sebastian/prog/cesm1_0_2/models/drv/shr/seq_io_mod.F90(88): error #6457: This derived type name has not been declared. [FILE_DESC_T]
type(file_desc_t) ,save :: cpl_io_file
--------^
/home/sebastian/prog/cesm1_0_2/models/drv/shr/seq_io_mod.F90(89): error #6457: This derived type name has not been declared. [IOSYSTEM_DESC_T]
type(iosystem_desc_t),save :: cpl_io_subsystem
--------^
...
------------------------^
/tmp/ifortDnbpmG.i90(1510): catastrophic error: Too many errors, exiting
compilation aborted for /home/sebastian/prog/cesm1_0_2/models/drv/shr/seq_io_mod.F90 (code 1)
gmake: *** [seq_io_mod.o] Fehler 1

 

[eaton]

The problem appears to be here:

/home/sebastian/prog/cesm1_0_2/models/drv/shr/seq_io_mod.F90(8:cool:: error #6457: This derived type name has not been declared. [FILE_DESC_T]
type(file_desc_t) ,save :: cpl_io_file
--------^
/home/sebastian/prog/cesm1_0_2/models/drv/shr/seq_io_mod.F90(89): error #6457: This derived type name has not been declared. [IOSYSTEM_DESC_T]
type(iosystem_desc_t),save :: cpl_io_subsystem
--------^

These type descriptors are coming from the PIO library. You'll find them in models/utils/pio/pio.F90. It isn't clear what ifort is complaining about since the PIO library built successfully. One thing to try would be to update to a later version of this library. The latest version (which is what we are currently using on the CESM development trunk) is here:

http://parallelio.googlecode.com/svn/trunk_tags/pio1_2_6/pio

 

[sebastian_gayler@uni-tuebingen_de]

Thank you for your quick reply.

I have downloaded the new pio. But now the same error occurs already during the build of pio.
Here are the first error messages from my pio.bldlog:

...
pio_msg_callbacks.F90(12): error #6457: This derived type name has not been declared. [FILE_DESC_T]
type(File_desc_t), pointer :: file
-------^
pio_msg_callbacks.F90(21): error #6404: This name does not have a type, and must have an explicit type. [FILE]
file=> lookupfile(fh)
--^
pio_msg_callbacks.F90(21): error #6795: The target must be of the same type and kind type parameters as the pointer. [FILE]
file=> lookupfile(fh)
--^
pio_msg_callbacks.F90(27): error #6285: There is no matching specific subroutine for this generic subroutine call. [PIO_CLOSEFILE]
call pio_closefile(file)
----------^

 

[eaton]

A similar problem has been reported here:

http://forum.cgd.ucar.edu/showthread.php?1913-Comipling-CESM-1_0_2

In that thread the compiler was intel-12.0.2. Is your compiler also a version 12.x (I can't tell from the description ComposerXE-2011.1.107).

 

[sebastian_gayler@uni-tuebingen_de]

Yes, the intel ComposerXE includes the intel c and fortran compilers version 12.0

 

[m_decker@unsw_edu_au]

Yes, the intel ComposerXE includes the intel c and fortran compilers version 12.0

I have had this same problem when using version 12 of the Intel compilers. Version 11.0 works well on the same machine that 12.0 fails to compile csm_share. On my machine 12.0 fails with the same error messages as posted above.

 

[eaton]

Thanks for confirming this.

I believe this is an intel compiler bug. I've looked at the code and can't find anything non-standard that might explain one compiler being successful and another failing.

I've seen this type of problem many times in the past, but usually with pgi which is one of our (at NCAR) production compilers (we use it mainly on cray-xt platforms). The problem is that an entity (in this case a type definition) which is available via use association is not being recognized by the compiler.

I'm afraid with our very limited resources that since the intel compiler is not one that we depend on, we don't have resources to pursue this bug with intel. If anyone else does please let us all know how it goes.

 

[m_decker@unsw_edu_au]

I have found a fix for compiling csm_shr with Intel version 12.0 that I thought might help others. In all of the files where there is the following (or similar) error
error #6457: This derived type name has not been declared. [FILE_DESC_T]

Explicitly tell the program to use the pio types by including

use pio_types

in the file. This most be done for several files but I don't remember the complete list. Doing so allows CLM to compile, however I haven't verified the result against the known model runs.

After this fix another error occurs while compiling CLM. The compiler feels that nan is either too small or too large. However this can be fixed by choosing a different value for nan.

 

[jpatton@iastate_edu]

I have found a fix for compiling csm_shr with Intel version 12.0 that I thought might help others. In all of the files where there is the following (or similar) error
error #6457: This derived type name has not been declared. [FILE_DESC_T]

Explicitly tell the program to use the pio types by including

use pio_types

in the file. This most be done for several files but I don't remember the complete list. Doing so allows CLM to compile, however I haven't verified the result against the known model runs.

Thanks mrd! For reference, for running I_2000_CN, I added "use pio_types" to the following files/lines, which allowed for compilation:

./drv/shr/seq_io_mod.F90:32:  use pio_types
./drv/shr/seq_io_mod.F90:121:   use pio_types
./drv/shr/seq_io_mod.F90:358:    use pio_types
./csm_share/shr/shr_pcdf_mod.F90:33:  use pio_types
./csm_share/shr/shr_dmodel_mod.F90:15:  use pio_types
./csm_share/shr/shr_strdata_mod.F90:41:   use pio_types

After this fix another error occurs while compiling CLM. The compiler feels that nan is either too small or too large. However this can be fixed by choosing a different value for nan.

I found this to be the largest number (well larger, I didn't try anything between O'33330...0 and O'44440...0) that worked:

./lnd/clm/src/main/nanMod.F90:30:  real(r8), parameter :: nan = O'0333300000000000000000'

I haven't yet had a chance to look at results, but this "works" (i.e. doesn't crash) for me.

Edit to add: Out of curiousity, if anyone knows, why are inf/nan defined using octal notation?