CESM Error on Bluegene

Hello,
I am porting of Bluegene/L with Linux front end OS. I am getting the lpnetcdf error, I tried removing lpnetcdf, but it gives lot of unresolved symbol error.

blrts_xlf90 -o /home/caos2/casgbala/dev_blugene/CESM102/cesm_intel/cesm1_0_2/scripts/casgbala/test1/run/ccsm.exe ccsm_driver.o map_atmatm_mct.o map_atmice_mct.o map_atmlnd_mct.o map_atmocn_mct.o map_glcglc_mct.o map_iceice_mct.o map_iceocn_mct.o map_lndlnd_mct.o map_ocnocn_mct.o map_rofocn_mct.o map_rofrof_mct.o map_snoglc_mct.o map_snosno_mct.o mrg_x2a_mct.o mrg_x2g_mct.o mrg_x2i_mct.o mrg_x2l_mct.o mrg_x2o_mct.o mrg_x2s_mct.o seq_avdata_mod.o seq_diag_mct.o seq_domain_mct.o seq_flux_mct.o seq_frac_mct.o seq_hist_mod.o seq_rearr_mod.o seq_rest_mod.o -L/home/caos2/casgbala/dev_blugene/CESM102/cesm_intel/cesm1_0_2/scripts/casgbala/test1/lib -latm -llnd -lice -locn -lglc -L/home/caos2/casgbala/dev_blugene/CESM102/cesm_intel/cesm1_0_2/scripts/casgbala/test1/lib -lcsm_share -lmct -lmpeu -lpio -L/opt/ibmmath/essl/4.2/lib -lmass -lmassv -L/contrib/memmon2 -L/home/caos2/casgbala/dev_blugene/netcdf_blugen/netcdf-3.6.3_installed/lib -lnetcdf -L/lib -lpnetcdf -L/bgl/BlueLight/ppcfloor/bglsys/lib -lmpich.rts -lmsglayer.rts -lrts.rts -ldevices.rts -Wl,--allow-multiple-definition
/bgl/BlueLight/ppcfloor/blrts-gnu/powerpc-bgl-blrts-gnu/bin/ld: cannot find -lpnetcdf
gmake: *** [/home/caos2/casgbala/dev_blugene/CESM102/cesm_intel/cesm1_0_2/scripts/casgbala/test1/run/ccsm.exe] Error 1

Is there anyway to remove the error..
Looking for help....

Thanks.
Dev
 

jedwards

CSEG and Liaisons
Staff member
Assuming that you started from the Macros.prototype_frost file you should remove the PNETCDF_PATH
PNETCDF_PATH := /contrib/bgl/parallel-netcdf-1.1.1
INC_PNETCDF := $(PNETCDF_PATH)/include
LIB_PNETCDF := $(PNETCDF_PATH)/lib

You'll then need to run the clean_build script and rebuild from scratch.

Having said all of that I really recommend installing and using pnetcdf for a better cesm experience on bgl.

- Jim
 

jedwards

CSEG and Liaisons
Staff member
Sorry - I realized that was incorrect.

The correct answer is to add the line
PIO_CONFIG_ARGS="--disable-pnetcdf"

near the top of the Macros file.
 
Hi Jim,

Thanks for the suggestion. I did add the line as suggested but getting the same error, there is no change.. Below is the Macros file:
#===============================================================================
# SVN $Id: Macros.cppdefs 23943 2010-06-21 20:47:36Z kauff $
# SVN $URL: https://svn-ccsm-models.cgd.ucar.ed...ts4_101018/ccsm_utils/Machines/Macros.cppdefs $
#===============================================================================
# set CPP options (must use this before any flags or cflags settings)
# Scripts build: this file is prepended to the $CASEROOT/Macros.machine file
# by create_newcase
# Cam build: this file is prepended to the $cam_bld/Macros file
# by cam's configure
#===============================================================================

PIO_CONFIG_ARGS= "--disable-pnetcdf"

CPPDEFS := $(USER_CPPDEFS)

# HIRES is currently only defined in env_build.xml
ifeq ($(HIRES),true)
CPPDEFS += -D_HIRES
endif

# USE_ESMF_LIB is currently only defined in env_build.xml
ifeq ($(USE_ESMF_LIB), TRUE)
CPPDEFS += -DUSE_ESMF_LIB
endif

# ESMF_INTERFACE is currently only defined in env_build.xml
ifeq ($(COMP_INTERFACE), ESMF)
CPPDEFS += -DESMF_INTERFACE
else
CPPDEFS += -DMCT_INTERFACE
endif

# USE_MPISERIAL is currently only defined in env_conf.xml
ifeq ($(USE_MPISERIAL),TRUE)
# default
else
CPPDEFS += -DHAVE_MPI
endif

# CCSM_BGC is currently only defined in env_conf.xml
ifneq ($(CCSM_BGC),)
ifneq ($(CCSM_BGC),none)
CPPDEFS += -D$(CCSM_BGC)
endif
endif

# CCSM_VOC is currently only defined in env_conf.xml
ifeq ($(CCSM_VOC), TRUE)
CPPDEFS += -DVOC
endif

# GLC_NEC is defined in env_conf.xml
ifneq ($(GLC_NEC),0)
CPPDEFS += -DGLC_NEC_$(GLC_NEC)
endif

#===============================================================================
# The following always need to be set
# SPMD Whether to build in SPMD mode or not. [values TRUE FALSE]
# SMP Set to TRUE to enable building in SMP mode (uses OpenMP).
# USER_CPPDEFS CPP definitions (non platform dependent)
#
# The following can be optionally set
# USER_LIB_MPI Allow user to override the default
# USER_LIB_NETCDF Allow user to override the default
# USER_LAPACK_LIBDIR Allow user to override the default
# USER_ESMF_LIBDIR Allos user to override the default
# USER_LINKER Allow user to override the default
#
# USER_INC_MPI Allow user to override the default
# USER MPI_LIB_NAME Allow user to override the default
# USER_INC_NETCDF Allow user to override the default
# USER_MOD_NETCDF Allow user to override the default
# USER_FC Allow user to override the default Fortran compiler specified in Makefile.
# USER_CC Allow user to override the default C compiler specified in Makefile (linux only).
# USER_CFLAGS Additional C compiler flags that the user wishes to set.
# USER_FFLAGS Additional Fortran compiler flags that the user wishes to set.
# USER_LDLAGS Additional load flags that the user wishes to set.
# F_OPTIMIZATION_OVERRIDE
# Use this to replace default Fortran compiler optimization.
#===============================================================================

# Note that CPPDEFS is set in Macros.cppdefs
CPPDEFS += -DAIX -DSEQ_$(FRAMEWORK) -DBGL -D_NOIO

ifeq ($(compile_threaded), true)
CPPDEFS += -DTHREADED_OMP
endif

cpre = $(null)-WF,-D$(null)
FPPFLAGS := $(patsubst -D%,$(cpre)%,$(CPPDEFS))

FC := blrts_xlf90
CC := blrts_xlc
NETCDF_PATH := /home/caos2/casgbala/dev_blugene/netcdf_blugen/netcdf-3.6.3_installed
INC_NETCDF := $(NETCDF_PATH)/include
LIB_NETCDF := $(NETCDF_PATH)/lib
MOD_NETCDF := $(NETCDF_PATH)/include
INC_MPI := /bgl/BlueLight/ppcfloor/bglsys/include
LIB_MPI :=
MPI_LIB_NAME :=
PNETCDF_PATH :=
INC_PNETCDF := $(PNETCDF_PATH)/include
LIB_PNETCDF := $(PNETCDF_PATH)/lib
LAPACK_LIBDIR :=

CFLAGS := $(CPPDEFS) -O3 -qstrict
FREEFLAGS := -qsuffix=f=f90:cpp=F90
FIXEDFLAGS := -qfixed=132
FFLAGS := $(FPPFLAGS) -qspillsize=2500 -g -qfullpath -qdebug=function_trace
FFLAGS_OPT := -O3 -qstrict -Q
LDFLAGS := -Wl,--allow-multiple-definition
AR := ar
MOD_SUFFIX := mod
CONFIG_SHELL :=

#===============================================================================
# Override with user settings
#===============================================================================

ifneq ($(strip $(USER_FC)),)
FC := $(USER_FC)
endif
ifneq ($(strip $(USER_CC)),)
CC := $(USER_CC)
endif
ifneq ($(strip $(USER_INC_NETCDF)),)
INC_NETCDF := $(USER_INC_NETCDF)
endif
ifneq ($(strip $(USER_MOD_NETCDF)),)
MOD_NETCDF := $(USER_MOD_NETCDF)
endif
ifneq ($(strip $(USER_INC_PNETCDF)),)
INC_PNETCDF := $(USER_INC_PNETCDF)
endif
ifneq ($(strip $(USER_INC_MPI)),)
INC_MPI := $(USER_INC_MPI)
endif

#===============================================================================
# Set include paths (needed after override for any model specific builds below)
#===============================================================================

INCLDIR := -I.
ifneq ($(strip $(INC_NETCDF)),)
INCLDIR += -I$(INC_NETCDF)
endif
ifneq ($(strip $(MOD_NETCDF)),)
INCLDIR += -I$(MOD_NETCDF)
endif
ifneq ($(strip $(INC_MPI)),)
INCLDIR += -I$(INC_MPI)
endif
ifneq ($(strip $(INC_PNETCDF)),)
INCLDIR += -I$(INC_PNETCDF)
endif

SLIBS := -L/opt/ibmmath/essl/4.2/lib -lmass -lmassv
SLIBS += -L/contrib/memmon2
MLIBS := -L/bgl/BlueLight/ppcfloor/bglsys/lib -lmpich.rts -lmsglayer.rts -lrts.rts -ldevices.rts

#===============================================================================
# Set model and other specific options
# NOTE - all CPPDEFS options must be defined before this point
#===============================================================================
ifeq ($(strip $(MODEL)),cam)
ifneq ($(F_OPTIMIZATION_OVERRIDE),$(null))
FFLAGS_OPT := $(F_OPTIMIZATION_OVERRIDE)
endif
endif

ifeq ($(DEBUG),TRUE)
FFLAGS += -qinitauto=FF911299 -qflttrap=ov:zero:inv:en
else
FFLAGS += $(FFLAGS_OPT)
endif

CONFIG_ARGS :=

ifeq ($(MODEL),mct)
#add arguments for mct configure here
CONFIG_ARGS += FC="$(FC)" F90="$(FC)" CC="$(CC)" INCLUDEPATH="-I$(INC_MPI)"
endif

ifeq ($(MODEL),pio)
ifneq ($(strip $(PIO_CONFIG_OPTS)),)
CONFIG_ARGS += $(PIO_CONFIG_OPTS)
endif
CONFIG_ARGS += FC="$(FC)" F90="$(FC)" CC="$(CC)" NETCDF_PATH=$(NETCDF_PATH) PNETCDF_PATH=$(PNETCDF_PATH) MPI_INC="-I$(INC_MPI)" --host=Linux
endif

# Append user defined compiler and load flags to Makefile defaults
CFLAGS += $(USER_CFLAGS)
FFLAGS += $(USER_FFLAGS)
LDFLAGS += $(USER_LDFLAGS)
 

eaton

CSEG and Liaisons
I'm not sure how this is all supposed to work, but what I see in the ccsm.bldlog file is a link command that contains -L/lib -lpnetcdf. This is apparently due to

PNETCDF_PATH :=
INC_PNETCDF := $(PNETCDF_PATH)/include
LIB_PNETCDF := $(PNETCDF_PATH)/lib

So it isn't sufficient to just remove the value of PNETCDF_PATH. I would try the following next:

PNETCDF_PATH :=
INC_PNETCDF :=
LIB_PNETCDF :=

Hopefully if LIB_PNETCDF is undefined then pnetcdf will not be included in the link command.
 

eaton

CSEG and Liaisons
I agree that there appears to be a problem with the netCDF library. It's strange that Fortran90 interfaces are present (e.g., nf90_get_var_4d_eightbytereal) but the Fortran77 interfaces are missing (e.g. nf_get_varm_double).

There also appear to be some unresolved system references, e.g., __func_trace_enter.

Your system administrator should be able to help with these issues.
 
Back
Top