Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

CESM Error on Bluegene

Hello,
I am porting of Bluegene/L with Linux front end OS. I am getting the lpnetcdf error, I tried removing lpnetcdf, but it gives lot of unresolved symbol error.

blrts_xlf90 -o /home/caos2/casgbala/dev_blugene/CESM102/cesm_intel/cesm1_0_2/scripts/casgbala/test1/run/ccsm.exe ccsm_driver.o map_atmatm_mct.o map_atmice_mct.o map_atmlnd_mct.o map_atmocn_mct.o map_glcglc_mct.o map_iceice_mct.o map_iceocn_mct.o map_lndlnd_mct.o map_ocnocn_mct.o map_rofocn_mct.o map_rofrof_mct.o map_snoglc_mct.o map_snosno_mct.o mrg_x2a_mct.o mrg_x2g_mct.o mrg_x2i_mct.o mrg_x2l_mct.o mrg_x2o_mct.o mrg_x2s_mct.o seq_avdata_mod.o seq_diag_mct.o seq_domain_mct.o seq_flux_mct.o seq_frac_mct.o seq_hist_mod.o seq_rearr_mod.o seq_rest_mod.o -L/home/caos2/casgbala/dev_blugene/CESM102/cesm_intel/cesm1_0_2/scripts/casgbala/test1/lib -latm -llnd -lice -locn -lglc -L/home/caos2/casgbala/dev_blugene/CESM102/cesm_intel/cesm1_0_2/scripts/casgbala/test1/lib -lcsm_share -lmct -lmpeu -lpio -L/opt/ibmmath/essl/4.2/lib -lmass -lmassv -L/contrib/memmon2 -L/home/caos2/casgbala/dev_blugene/netcdf_blugen/netcdf-3.6.3_installed/lib -lnetcdf -L/lib -lpnetcdf -L/bgl/BlueLight/ppcfloor/bglsys/lib -lmpich.rts -lmsglayer.rts -lrts.rts -ldevices.rts -Wl,--allow-multiple-definition
/bgl/BlueLight/ppcfloor/blrts-gnu/powerpc-bgl-blrts-gnu/bin/ld: cannot find -lpnetcdf
gmake: *** [/home/caos2/casgbala/dev_blugene/CESM102/cesm_intel/cesm1_0_2/scripts/casgbala/test1/run/ccsm.exe] Error 1

Is there anyway to remove the error..
Looking for help....

Thanks.
Dev
 

jedwards

CSEG and Liaisons
Staff member
Assuming that you started from the Macros.prototype_frost file you should remove the PNETCDF_PATH
PNETCDF_PATH := /contrib/bgl/parallel-netcdf-1.1.1
INC_PNETCDF := $(PNETCDF_PATH)/include
LIB_PNETCDF := $(PNETCDF_PATH)/lib

You'll then need to run the clean_build script and rebuild from scratch.

Having said all of that I really recommend installing and using pnetcdf for a better cesm experience on bgl.

- Jim
 

jedwards

CSEG and Liaisons
Staff member
Sorry - I realized that was incorrect.

The correct answer is to add the line
PIO_CONFIG_ARGS="--disable-pnetcdf"

near the top of the Macros file.
 
Hi Jim,

Thanks for the suggestion. I did add the line as suggested but getting the same error, there is no change.. Below is the Macros file:
#===============================================================================
# SVN $Id: Macros.cppdefs 23943 2010-06-21 20:47:36Z kauff $
# SVN $URL: https://svn-ccsm-models.cgd.ucar.ed...ts4_101018/ccsm_utils/Machines/Macros.cppdefs $
#===============================================================================
# set CPP options (must use this before any flags or cflags settings)
# Scripts build: this file is prepended to the $CASEROOT/Macros.machine file
# by create_newcase
# Cam build: this file is prepended to the $cam_bld/Macros file
# by cam's configure
#===============================================================================

PIO_CONFIG_ARGS= "--disable-pnetcdf"

CPPDEFS := $(USER_CPPDEFS)

# HIRES is currently only defined in env_build.xml
ifeq ($(HIRES),true)
CPPDEFS += -D_HIRES
endif

# USE_ESMF_LIB is currently only defined in env_build.xml
ifeq ($(USE_ESMF_LIB), TRUE)
CPPDEFS += -DUSE_ESMF_LIB
endif

# ESMF_INTERFACE is currently only defined in env_build.xml
ifeq ($(COMP_INTERFACE), ESMF)
CPPDEFS += -DESMF_INTERFACE
else
CPPDEFS += -DMCT_INTERFACE
endif

# USE_MPISERIAL is currently only defined in env_conf.xml
ifeq ($(USE_MPISERIAL),TRUE)
# default
else
CPPDEFS += -DHAVE_MPI
endif

# CCSM_BGC is currently only defined in env_conf.xml
ifneq ($(CCSM_BGC),)
ifneq ($(CCSM_BGC),none)
CPPDEFS += -D$(CCSM_BGC)
endif
endif

# CCSM_VOC is currently only defined in env_conf.xml
ifeq ($(CCSM_VOC), TRUE)
CPPDEFS += -DVOC
endif

# GLC_NEC is defined in env_conf.xml
ifneq ($(GLC_NEC),0)
CPPDEFS += -DGLC_NEC_$(GLC_NEC)
endif

#===============================================================================
# The following always need to be set
# SPMD Whether to build in SPMD mode or not. [values TRUE FALSE]
# SMP Set to TRUE to enable building in SMP mode (uses OpenMP).
# USER_CPPDEFS CPP definitions (non platform dependent)
#
# The following can be optionally set
# USER_LIB_MPI Allow user to override the default
# USER_LIB_NETCDF Allow user to override the default
# USER_LAPACK_LIBDIR Allow user to override the default
# USER_ESMF_LIBDIR Allos user to override the default
# USER_LINKER Allow user to override the default
#
# USER_INC_MPI Allow user to override the default
# USER MPI_LIB_NAME Allow user to override the default
# USER_INC_NETCDF Allow user to override the default
# USER_MOD_NETCDF Allow user to override the default
# USER_FC Allow user to override the default Fortran compiler specified in Makefile.
# USER_CC Allow user to override the default C compiler specified in Makefile (linux only).
# USER_CFLAGS Additional C compiler flags that the user wishes to set.
# USER_FFLAGS Additional Fortran compiler flags that the user wishes to set.
# USER_LDLAGS Additional load flags that the user wishes to set.
# F_OPTIMIZATION_OVERRIDE
# Use this to replace default Fortran compiler optimization.
#===============================================================================

# Note that CPPDEFS is set in Macros.cppdefs
CPPDEFS += -DAIX -DSEQ_$(FRAMEWORK) -DBGL -D_NOIO

ifeq ($(compile_threaded), true)
CPPDEFS += -DTHREADED_OMP
endif

cpre = $(null)-WF,-D$(null)
FPPFLAGS := $(patsubst -D%,$(cpre)%,$(CPPDEFS))

FC := blrts_xlf90
CC := blrts_xlc
NETCDF_PATH := /home/caos2/casgbala/dev_blugene/netcdf_blugen/netcdf-3.6.3_installed
INC_NETCDF := $(NETCDF_PATH)/include
LIB_NETCDF := $(NETCDF_PATH)/lib
MOD_NETCDF := $(NETCDF_PATH)/include
INC_MPI := /bgl/BlueLight/ppcfloor/bglsys/include
LIB_MPI :=
MPI_LIB_NAME :=
PNETCDF_PATH :=
INC_PNETCDF := $(PNETCDF_PATH)/include
LIB_PNETCDF := $(PNETCDF_PATH)/lib
LAPACK_LIBDIR :=

CFLAGS := $(CPPDEFS) -O3 -qstrict
FREEFLAGS := -qsuffix=f=f90:cpp=F90
FIXEDFLAGS := -qfixed=132
FFLAGS := $(FPPFLAGS) -qspillsize=2500 -g -qfullpath -qdebug=function_trace
FFLAGS_OPT := -O3 -qstrict -Q
LDFLAGS := -Wl,--allow-multiple-definition
AR := ar
MOD_SUFFIX := mod
CONFIG_SHELL :=

#===============================================================================
# Override with user settings
#===============================================================================

ifneq ($(strip $(USER_FC)),)
FC := $(USER_FC)
endif
ifneq ($(strip $(USER_CC)),)
CC := $(USER_CC)
endif
ifneq ($(strip $(USER_INC_NETCDF)),)
INC_NETCDF := $(USER_INC_NETCDF)
endif
ifneq ($(strip $(USER_MOD_NETCDF)),)
MOD_NETCDF := $(USER_MOD_NETCDF)
endif
ifneq ($(strip $(USER_INC_PNETCDF)),)
INC_PNETCDF := $(USER_INC_PNETCDF)
endif
ifneq ($(strip $(USER_INC_MPI)),)
INC_MPI := $(USER_INC_MPI)
endif

#===============================================================================
# Set include paths (needed after override for any model specific builds below)
#===============================================================================

INCLDIR := -I.
ifneq ($(strip $(INC_NETCDF)),)
INCLDIR += -I$(INC_NETCDF)
endif
ifneq ($(strip $(MOD_NETCDF)),)
INCLDIR += -I$(MOD_NETCDF)
endif
ifneq ($(strip $(INC_MPI)),)
INCLDIR += -I$(INC_MPI)
endif
ifneq ($(strip $(INC_PNETCDF)),)
INCLDIR += -I$(INC_PNETCDF)
endif

SLIBS := -L/opt/ibmmath/essl/4.2/lib -lmass -lmassv
SLIBS += -L/contrib/memmon2
MLIBS := -L/bgl/BlueLight/ppcfloor/bglsys/lib -lmpich.rts -lmsglayer.rts -lrts.rts -ldevices.rts

#===============================================================================
# Set model and other specific options
# NOTE - all CPPDEFS options must be defined before this point
#===============================================================================
ifeq ($(strip $(MODEL)),cam)
ifneq ($(F_OPTIMIZATION_OVERRIDE),$(null))
FFLAGS_OPT := $(F_OPTIMIZATION_OVERRIDE)
endif
endif

ifeq ($(DEBUG),TRUE)
FFLAGS += -qinitauto=FF911299 -qflttrap=ov:zero:inv:en
else
FFLAGS += $(FFLAGS_OPT)
endif

CONFIG_ARGS :=

ifeq ($(MODEL),mct)
#add arguments for mct configure here
CONFIG_ARGS += FC="$(FC)" F90="$(FC)" CC="$(CC)" INCLUDEPATH="-I$(INC_MPI)"
endif

ifeq ($(MODEL),pio)
ifneq ($(strip $(PIO_CONFIG_OPTS)),)
CONFIG_ARGS += $(PIO_CONFIG_OPTS)
endif
CONFIG_ARGS += FC="$(FC)" F90="$(FC)" CC="$(CC)" NETCDF_PATH=$(NETCDF_PATH) PNETCDF_PATH=$(PNETCDF_PATH) MPI_INC="-I$(INC_MPI)" --host=Linux
endif

# Append user defined compiler and load flags to Makefile defaults
CFLAGS += $(USER_CFLAGS)
FFLAGS += $(USER_FFLAGS)
LDFLAGS += $(USER_LDFLAGS)
 

eaton

CSEG and Liaisons
I'm not sure how this is all supposed to work, but what I see in the ccsm.bldlog file is a link command that contains -L/lib -lpnetcdf. This is apparently due to

PNETCDF_PATH :=
INC_PNETCDF := $(PNETCDF_PATH)/include
LIB_PNETCDF := $(PNETCDF_PATH)/lib

So it isn't sufficient to just remove the value of PNETCDF_PATH. I would try the following next:

PNETCDF_PATH :=
INC_PNETCDF :=
LIB_PNETCDF :=

Hopefully if LIB_PNETCDF is undefined then pnetcdf will not be included in the link command.
 

eaton

CSEG and Liaisons
I agree that there appears to be a problem with the netCDF library. It's strange that Fortran90 interfaces are present (e.g., nf90_get_var_4d_eightbytereal) but the Fortran77 interfaces are missing (e.g. nf_get_varm_double).

There also appear to be some unresolved system references, e.g., __func_trace_enter.

Your system administrator should be able to help with these issues.
 
Top