Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

forrtl: severe (174): SIGSEGV, segmentation fault occurred

M

moguo_sun@nasa_gov

Member
Dear Helper,          I got a problem when I run cam. Here is my configure $camcfg/configure -fc ifort -dyn fv -hgrid 10x15 -nospmd -nosmp -debug -test  and Here is the error message in cam.log:(seq_comm_setcomm)  initialize ID (  1 GLOBAL          ) pelist   =     0     0     1 ( npes =     1) ( nthreads =  1)
(seq_comm_setcomm)  initialize ID (  2 CPL             ) pelist   =     0     0     1 ( npes =     1) ( nthreads =  1)
(seq_comm_setcomm)  initialize ID ( 15 ATM             ) pelist   =     0     0     1 ( npes =     1) ( nthreads =  1)
(seq_comm_joincomm) initialize ID ( 16 CPLATM          ) join IDs =     2    15       ( npes =     1) ( nthreads =  1)
(seq_comm_jcommarr) initialize ID (  3 ALLATMID        ) join multiple comp IDs       ( npes =     1) ( nthreads =  1)
(seq_comm_joincomm) initialize ID (  9 CPLALLATMID     ) join IDs =     2     3       ( npes =     1) ( nthreads =  1)
(seq_comm_setcomm)  initialize ID ( 17 LND             ) pelist   =     0     0     1 ( npes =     1) ( nthreads =  1)
(seq_comm_joincomm) initialize ID ( 18 CPLLND          ) join IDs =     2    17       ( npes =     1) ( nthreads =  1)
(seq_comm_jcommarr) initialize ID (  4 ALLLNDID        ) join multiple comp IDs       ( npes =     1) ( nthreads =  1)
(seq_comm_joincomm) initialize ID ( 10 CPLALLLNDID     ) join IDs =     2     4       ( npes =     1) ( nthreads =  1)
(seq_comm_setcomm)  initialize ID ( 19 OCN             ) pelist   =     0     0     1 ( npes =     1) ( nthreads =  1)
(seq_comm_joincomm) initialize ID ( 20 CPLOCN          ) join IDs =     2    19       ( npes =     1) ( nthreads =  1)
(seq_comm_jcommarr) initialize ID (  5 ALLOCNID        ) join multiple comp IDs       ( npes =     1) ( nthreads =  1)
(seq_comm_joincomm) initialize ID ( 11 CPLALLOCNID     ) join IDs =     2     5       ( npes =     1) ( nthreads =  1)
(seq_comm_setcomm)  initialize ID ( 21 ICE             ) pelist   =     0     0     1 ( npes =     1) ( nthreads =  1)
(seq_comm_joincomm) initialize ID ( 22 CPLICE          ) join IDs =     2    21       ( npes =     1) ( nthreads =  1)
(seq_comm_jcommarr) initialize ID (  6 ALLICEID        ) join multiple comp IDs       ( npes =     1) ( nthreads =  1)
(seq_comm_joincomm) initialize ID ( 12 CPLALLICEID     ) join IDs =     2     6       ( npes =     1) ( nthreads =  1)
(seq_comm_setcomm)  initialize ID ( 23 GLC             ) pelist   =     0     0     1 ( npes =     1) ( nthreads =  1)
(seq_comm_joincomm) initialize ID ( 24 CPLGLC          ) join IDs =     2    23       ( npes =     1) ( nthreads =  1)
(seq_comm_jcommarr) initialize ID (  7 ALLGLCID        ) join multiple comp IDs       ( npes =     1) ( nthreads =  1)
(seq_comm_joincomm) initialize ID ( 13 CPLALLGLCID     ) join IDs =     2     7       ( npes =     1) ( nthreads =  1)
(seq_comm_setcomm)  initialize ID ( 25 ROF             ) pelist   =     0     0     1 ( npes =     1) ( nthreads =  1)
(seq_comm_joincomm) initialize ID ( 26 CPLROF          ) join IDs =     2    25       ( npes =     1) ( nthreads =  1)
(seq_comm_jcommarr) initialize ID (  8 ALLROFID        ) join multiple comp IDs       ( npes =     1) ( nthreads =  1)
(seq_comm_joincomm) initialize ID ( 14 CPLALLROFID     ) join IDs =     2     8       ( npes =     1) ( nthreads =  1)
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image              PC                Routine            Line        Source             
cam                00000000017F947B  Unknown               Unknown  Unknown
cam                00000000016D6CAD  Unknown               Unknown  Unknown
cam                00000000013ACE9B  Unknown               Unknown  Unknown
cam                00000000012BA4C6  Unknown               Unknown  Unknown
cam                000000000060C30A  Unknown               Unknown  Unknown
cam                000000000061E847  Unknown               Unknown  Unknown
cam                000000000040033C  Unknown               Unknown  Unknown
cam                00000000017D1534  Unknown               Unknown  Unknown
cam                0000000000400209  Unknown               Unknown  Unknown
---------------------------------------------          Appreciate if you can give any suggestions how to go forward. Thanks a lot. Moguo 
 
E

eaton

CSEG and Liaisons
What version of CESM are you working with?Could you also add the -v option to your configure command and post the output? 
 
M

moguo_sun@nasa_gov

Member
Hi, Eaton,      It cesm1_1_2.       I re-configured like this:$camcfg/configure -v -fc ifort -dyn fv -hgrid 10x15 -nospmd -nosmp -debug -test      this is output from the configure:-------Physics package: cam5
Chemistry package: trop_mam3
CAM configuration script directory: /homedir/msun/cam5/cesm1_1_2/models/atm/cam/bld
Configuration defaults file: config_files/defaults_fv.xml
CAM root directory: /homedir/msun/cam5/cesm1_1_2
CAM build directory: /homedir/msun/cam5/work/bld
CAM executable will be created in: /homedir/msun/cam5/work/bld
User source directories:
Configuration cache file: /homedir/msun/cam5/work/bld/config_cache.xml
Target OS: linux
SPMD parallelism: OFF
SMP parallelism: OFF
Component interface: mct
Dynamics package: fv
Number of user requested test tracers: 0
Age of air tracer package: OFF
Microphysics package: mg
Macrophysics package: park
Using uw PBL scheme.
Radiation package: rrtmg
Ocean package: docn
Land package: clm
Sea ice package: cice
Runoff package: rtm
Perturbation growth testing: OFF
CAM single column mode (SCAM): OFF
Produce IOP file for SCAM: OFF
Horizontal grid specifier: 10x15
Maximum number of columns in a chunk: 16
Number of vertical levels: 30
Advected constituents added by mg microphysics: 4
Advected constituents added by chemistry trop_mam3: 20
Total advected constituents: 25
Issuing command to the CICE configure utility:
  /homedir/msun/cam5/cesm1_1_2/models/ice/cice/bld/configure -hgrid 10x15 -cice_mode prescribed -ntr_aero 0 -ntasks 1 -nthreads 1 -cache config_cache_cice.xml -cachedir /homedir/msun/cam5/work/bld
CICE configure done.
Name of CAM executable: cam
Fortran compiler: ifort
Fortran compiler type: intel
C compiler: icc
Compiler debugging options: ON
Found netCDF include file in: /usr/local/include
Found netCDF library in: /usr/local/lib
Found netCDF module files in: /usr/local/include
CPP definitions set by configure: '  -DFORTRANUNDERSCORE -DCO2A -DMAXPATCH_PFT=numpft+1 -DLSMLAT=1 -DLSMLON=1 -DPLON=24 -DPLAT=19 -DNUM_COMP_INST_ATM=1 -DNUM_COMP_INST_LND=1 -DNUM_COMP_INST_OCN=1 -DNUM_COMP_INST_ICE=1 -DNUM_COMP_INST_GLC=1 -DNUM_CO
MP_INST_ROF=1 -DPLEV=30 -DPCNST=25 -DPCOLS=16 -DPTRM=1 -DPTRN=1 -DPTRK=1 -DSTAGGERED  -DCCSMCOUPLED -Dcoupled -Dncdf -DNCAT=1 -DNXGLOB=24 -DNYGLOB=19 -DNTR_AERO=0 -DBLCKX=24 -DBLCKY=19 -DMXBLCKS=1  -DMODAL_AERO -DMODAL_AERO_3MODE  -D_USEBOX -D_NETC
DF  '
checking for cc... cc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether cc accepts -g... yes
checking for cc option to accept ISO C89... none needed
checking how to run the C preprocessor... cc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking whether byte ordering is bigendian... no
checking whether we are using the GNU Fortran compiler... no
checking whether ifort accepts -g... yes
checking for Fortran flag to compile .F90 files... none
MPISERIAL ENABLED: BYPASSING MPI CHECK
FORT_SIZE IS PRESET TO real4double8
Checking Compiler Version
checking how to get the version output from ifort... -V
checking for Fortran flag to compile preprocessed .F files... none
checking how to define symbols for preprocessed Fortran... -D
checking how to get verbose linking output from ifort... -v
checking for Fortran libraries of ifort...  -L/usr/local/intel/lib/intel64 -L/SPG_ops/utils/x86_64/sles11/gcc-4.5.3-ceres/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.5.3 -L/SPG_ops/utils/x86_64/sles11/gcc-4.5.3-ceres/bin/../lib/gcc -L/SPG_ops/utils/x
86_64/sles11/gcc-4.5.3-ceres/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.5.3/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/SPG_ops/utils/x86_64/sles11/gcc-4.5.3-ceres/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.5.3/../../.. -L/lib64 -L/lib -
L/usr/lib64 -L/usr/lib -lifport -lifcore -limf -lsvml -lm -lipgo -lirc -lpthread -lirc_s -ldl
checking for dummy main to link with Fortran libraries... none
checking for Fortran name-mangling scheme... lower case, underscore, no extra underscore
Hostname=be313
Machine=x86_64
OS=Linux
Fortran Compiler is Intel ifort

Output Variables: {CC=cc} {CFLAGS=-g -O2} {FC=ifort} {FCFLAGS=-w -ftz} {PROGFCFLAGS=-assume byterecl}{CPPDEFS= -DSYSLINUX -DCPRINTEL} {OPT=-O2} {DEBUG=} {REAL8=-r8} {BIT64=} {ENDIAN=-convert big_endian} {MPIFC=ifort} {MPILIBS=-L/homedir/msun/cam5/w
ork/bld/mct/mpi-serial -lmpi-serial} {MPIHEADER=-I/homedir/msun/cam5/work/bld/mct/mpi-serial} {INCLUDEFLAG=-I} {INCLUDEPATH=} {AR=ar cq} {BABELROOT=} {COMPILER_ROOT=} {PYTHON=} {PYTHONOPTS=} {FORT_SIZE=real4double8} {prefix=NONE} {SRCDIR=} {FC_DEFI
NE=-D}

configure: creating ./config.status
config.status: creating Makefile.conf
config.status: creating config.h
config.status: config.h is unchanged
Please check the Makefile.conf
Have a nice day!
MCT configure is done.
creating /homedir/msun/cam5/work/bld/Filepath
creating /homedir/msun/cam5/work/bld/Makefile
creating /homedir/msun/cam5/work/bld/config.h
creating /homedir/msun/cam5/work/bld/config_cache.xml
Looking for a valid GNU make... using gmake
Testing for Fortran 90 compatible compiler... **** PASS ****
Issued the command:
gmake -f /homedir/msun/cam5/work/bld/Makefile test_fc 2>&1

The output was:
cat: Srcfiles: No such file or directory
/homedir/msun/cam5/work/bld/Makefile:942: /homedir/msun/cam5/work/bld/configure-tests/Depends: No such file or directory
/homedir/msun/cam5/cesm1_1_2/models/atm/cam/bld/mkSrcfiles > /homedir/msun/cam5/work/bld/configure-tests/Srcfiles
/homedir/msun/cam5/cesm1_1_2/models/atm/cam/bld/mkDepends Filepath Srcfiles > /homedir/msun/cam5/work/bld/configure-tests/Depends
ifort -c  -I. -I/homedir/msun/cam5/work/bld/configure-tests -I/usr/local/include  -I/usr/local/include -I/homedir/msun/cam5/work/bld/mct/mct -I/homedir/msun/cam5/work/bld/mct/mpeu -I/homedir/msun/cam5/work/bld/mct/mpi-serial -DNO_SHR_VMATH -DSEQ_MC
T -DNO_C_SIZEOF -DFORTRANUNDERSCORE -DCO2A -DMAXPATCH_PFT=numpft+1 -DLSMLAT=1 -DLSMLON=1 -DPLON=24 -DPLAT=19 -DNUM_COMP_INST_ATM=1 -DNUM_COMP_INST_LND=1 -DNUM_COMP_INST_OCN=1 -DNUM_COMP_INST_ICE=1 -DNUM_COMP_INST_GLC=1 -DNUM_COMP_INST_ROF=1 -DPLEV=
30 -DPCNST=25 -DPCOLS=16 -DPTRM=1 -DPTRN=1 -DPTRK=1 -DSTAGGERED  -DCCSMCOUPLED -Dcoupled -Dncdf -DNCAT=1 -DNXGLOB=24 -DNYGLOB=19 -DNTR_AERO=0 -DBLCKX=24 -DBLCKY=19 -DMXBLCKS=1  -DMODAL_AERO -DMODAL_AERO_3MODE  -D_USEBOX -D_NETCDF   -DLINUX -DCPRINT
EL -static -ftz -convert big_endian -fp-model precise  -check all -check noarg_temp_created -fpe0 -g -traceback -DHIDE_MPI -D_MPISERIAL -DNO_MPIMOD -DNO_MPI2 -DNO_SIZEOF  test_fc.F90
ifort -o test_fc -I. -I/homedir/msun/cam5/work/bld/configure-tests -I/usr/local/include  -I/usr/local/include -I/homedir/msun/cam5/work/bld/mct/mct -I/homedir/msun/cam5/work/bld/mct/mpeu -I/homedir/msun/cam5/work/bld/mct/mpi-serial -DNO_SHR_VMATH -
DSEQ_MCT -DNO_C_SIZEOF -DFORTRANUNDERSCORE -DCO2A -DMAXPATCH_PFT=numpft+1 -DLSMLAT=1 -DLSMLON=1 -DPLON=24 -DPLAT=19 -DNUM_COMP_INST_ATM=1 -DNUM_COMP_INST_LND=1 -DNUM_COMP_INST_OCN=1 -DNUM_COMP_INST_ICE=1 -DNUM_COMP_INST_GLC=1 -DNUM_COMP_INST_ROF=1
-DPLEV=30 -DPCNST=25 -DPCOLS=16 -DPTRM=1 -DPTRN=1 -DPTRK=1 -DSTAGGERED  -DCCSMCOUPLED -Dcoupled -Dncdf -DNCAT=1 -DNXGLOB=24 -DNYGLOB=19 -DNTR_AERO=0 -DBLCKX=24 -DBLCKY=19 -DMXBLCKS=1  -DMODAL_AERO -DMODAL_AERO_3MODE  -D_USEBOX -D_NETCDF   -DLINUX -
DCPRINTEL -static -ftz -convert big_endian -fp-model precise  -check all -check noarg_temp_created -fpe0 -g -traceback -DHIDE_MPI -D_MPISERIAL -DNO_MPIMOD -DNO_MPI2 -DNO_SIZEOF  test_fc.o

**** PASS ****
Issued the command:
./test_fc

The output was:
 hello world

using ifort
Test linking to NetCDF library... **** PASS ****
Issued the command:
gmake -f /homedir/msun/cam5/work/bld/Makefile test_nc 2>&1

The output was:
cat: Srcfiles: No such file or directory
/homedir/msun/cam5/work/bld/Makefile:942: /homedir/msun/cam5/work/bld/configure-tests/Depends: No such file or directory
/homedir/msun/cam5/cesm1_1_2/models/atm/cam/bld/mkSrcfiles > /homedir/msun/cam5/work/bld/configure-tests/Srcfiles
/homedir/msun/cam5/cesm1_1_2/models/atm/cam/bld/mkDepends Filepath Srcfiles > /homedir/msun/cam5/work/bld/configure-tests/Depends
ifort -c  -I. -I/homedir/msun/cam5/work/bld/configure-tests -I/usr/local/include  -I/usr/local/include -I/homedir/msun/cam5/work/bld/mct/mct -I/homedir/msun/cam5/work/bld/mct/mpeu -I/homedir/msun/cam5/work/bld/mct/mpi-serial -DNO_SHR_VMATH -DSEQ_MC
T -DNO_C_SIZEOF -DFORTRANUNDERSCORE -DCO2A -DMAXPATCH_PFT=numpft+1 -DLSMLAT=1 -DLSMLON=1 -DPLON=24 -DPLAT=19 -DNUM_COMP_INST_ATM=1 -DNUM_COMP_INST_LND=1 -DNUM_COMP_INST_OCN=1 -DNUM_COMP_INST_ICE=1 -DNUM_COMP_INST_GLC=1 -DNUM_COMP_INST_ROF=1 -DPLEV=
30 -DPCNST=25 -DPCOLS=16 -DPTRM=1 -DPTRN=1 -DPTRK=1 -DSTAGGERED  -DCCSMCOUPLED -Dcoupled -Dncdf -DNCAT=1 -DNXGLOB=24 -DNYGLOB=19 -DNTR_AERO=0 -DBLCKX=24 -DBLCKY=19 -DMXBLCKS=1  -DMODAL_AERO -DMODAL_AERO_3MODE  -D_USEBOX -D_NETCDF   -DLINUX -DCPRINT
EL -static -ftz -convert big_endian -fp-model precise  -check all -check noarg_temp_created -fpe0 -g -traceback -DHIDE_MPI -D_MPISERIAL -DNO_MPIMOD -DNO_MPI2 -DNO_SIZEOF  test_nc.F90
ifort -o test_nc -I. -I/homedir/msun/cam5/work/bld/configure-tests -I/usr/local/include  -I/usr/local/include -I/homedir/msun/cam5/work/bld/mct/mct -I/homedir/msun/cam5/work/bld/mct/mpeu -I/homedir/msun/cam5/work/bld/mct/mpi-serial -DNO_SHR_VMATH -
DSEQ_MCT -DNO_C_SIZEOF -DFORTRANUNDERSCORE -DCO2A -DMAXPATCH_PFT=numpft+1 -DLSMLAT=1 -DLSMLON=1 -DPLON=24 -DPLAT=19 -DNUM_COMP_INST_ATM=1 -DNUM_COMP_INST_LND=1 -DNUM_COMP_INST_OCN=1 -DNUM_COMP_INST_ICE=1 -DNUM_COMP_INST_GLC=1 -DNUM_COMP_INST_ROF=1
-DPLEV=30 -DPCNST=25 -DPCOLS=16 -DPTRM=1 -DPTRN=1 -DPTRK=1 -DSTAGGERED  -DCCSMCOUPLED -Dcoupled -Dncdf -DNCAT=1 -DNXGLOB=24 -DNYGLOB=19 -DNTR_AERO=0 -DBLCKX=24 -DBLCKY=19 -DMXBLCKS=1  -DMODAL_AERO -DMODAL_AERO_3MODE  -D_USEBOX -D_NETCDF   -DLINUX -
DCPRINTEL -static -ftz -convert big_endian -fp-model precise  -check all -check noarg_temp_created -fpe0 -g -traceback -DHIDE_MPI -D_MPISERIAL -DNO_MPIMOD -DNO_MPI2 -DNO_SIZEOF  test_nc.o -L/usr/local/lib -lnetcdf -Wl,-rpath=/usr/local/lib

**** PASS ****
Issued the command:
./test_nc

The output was:
 created foo.nc

ok
CAM configure done.
-------after that, I run cam5 and it gives following:(seq_comm_setcomm)  initialize ID (  1 GLOBAL          ) pelist   =     0     0     1 ( npes =     1) ( nthreads =  1)
(seq_comm_setcomm)  initialize ID (  2 CPL             ) pelist   =     0     0     1 ( npes =     1) ( nthreads =  1)
(seq_comm_setcomm)  initialize ID ( 15 ATM             ) pelist   =     0     0     1 ( npes =     1) ( nthreads =  1)
(seq_comm_joincomm) initialize ID ( 16 CPLATM          ) join IDs =     2    15       ( npes =     1) ( nthreads =  1)
(seq_comm_jcommarr) initialize ID (  3 ALLATMID        ) join multiple comp IDs       ( npes =     1) ( nthreads =  1)
(seq_comm_joincomm) initialize ID (  9 CPLALLATMID     ) join IDs =     2     3       ( npes =     1) ( nthreads =  1)
(seq_comm_setcomm)  initialize ID ( 17 LND             ) pelist   =     0     0     1 ( npes =     1) ( nthreads =  1)
(seq_comm_joincomm) initialize ID ( 18 CPLLND          ) join IDs =     2    17       ( npes =     1) ( nthreads =  1)
(seq_comm_jcommarr) initialize ID (  4 ALLLNDID        ) join multiple comp IDs       ( npes =     1) ( nthreads =  1)
(seq_comm_joincomm) initialize ID ( 10 CPLALLLNDID     ) join IDs =     2     4       ( npes =     1) ( nthreads =  1)
(seq_comm_setcomm)  initialize ID ( 19 OCN             ) pelist   =     0     0     1 ( npes =     1) ( nthreads =  1)
(seq_comm_joincomm) initialize ID ( 20 CPLOCN          ) join IDs =     2    19       ( npes =     1) ( nthreads =  1)
(seq_comm_jcommarr) initialize ID (  5 ALLOCNID        ) join multiple comp IDs       ( npes =     1) ( nthreads =  1)
(seq_comm_joincomm) initialize ID ( 11 CPLALLOCNID     ) join IDs =     2     5       ( npes =     1) ( nthreads =  1)
(seq_comm_setcomm)  initialize ID ( 21 ICE             ) pelist   =     0     0     1 ( npes =     1) ( nthreads =  1)
(seq_comm_joincomm) initialize ID ( 22 CPLICE          ) join IDs =     2    21       ( npes =     1) ( nthreads =  1)
(seq_comm_jcommarr) initialize ID (  6 ALLICEID        ) join multiple comp IDs       ( npes =     1) ( nthreads =  1)
(seq_comm_joincomm) initialize ID ( 12 CPLALLICEID     ) join IDs =     2     6       ( npes =     1) ( nthreads =  1)
(seq_comm_setcomm)  initialize ID ( 23 GLC             ) pelist   =     0     0     1 ( npes =     1) ( nthreads =  1)
(seq_comm_joincomm) initialize ID ( 24 CPLGLC          ) join IDs =     2    23       ( npes =     1) ( nthreads =  1)
(seq_comm_jcommarr) initialize ID (  7 ALLGLCID        ) join multiple comp IDs       ( npes =     1) ( nthreads =  1)
(seq_comm_joincomm) initialize ID ( 13 CPLALLGLCID     ) join IDs =     2     7       ( npes =     1) ( nthreads =  1)
(seq_comm_setcomm)  initialize ID ( 25 ROF             ) pelist   =     0     0     1 ( npes =     1) ( nthreads =  1)
(seq_comm_joincomm) initialize ID ( 26 CPLROF          ) join IDs =     2    25       ( npes =     1) ( nthreads =  1)
(seq_comm_jcommarr) initialize ID (  8 ALLROFID        ) join multiple comp IDs       ( npes =     1) ( nthreads =  1)
(seq_comm_joincomm) initialize ID ( 14 CPLALLROFID     ) join IDs =     2     8       ( npes =     1) ( nthreads =  1)
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image              PC                Routine            Line        Source            
cam                0000000004DBC0FB  Unknown               Unknown  Unknown
cam                0000000004C99BBD  Unknown               Unknown  Unknown
cam                00000000043670DC  shr_mpi_mod_mp_sh        1770  shr_mpi_mod.F90
cam                0000000003FDD46B  seq_comm_mct_mp_s         807  seq_comm_mct.F90
cam                0000000000E6516F  ccsm_comp_mod_mp_         522  ccsm_comp_mod.F90
cam                0000000000ED9BB9  MAIN__                     47  ccsm_driver.F90
cam                000000000040033C  Unknown               Unknown  Unknown
cam                0000000004D941B4  Unknown               Unknown  Unknown
cam                0000000000400209  Unknown               Unknown  Unknown
---------------------       
 
M

moguo_sun@nasa_gov

Member
Continue from last reply,I want to confess that I commented out this line in cam5/cesm1_1_2/models/atm/cam/src/utils/spmd_utils.F90line 1661:  integer :: mpi_status_ignore     ! Needs to be defined in mpi-serial

In order to pass compilation. I can't pass withoiut commenting that out since I can't find where else that variable has been defined earlier. thank you for your valuable suggestions. Moguo   
 
E

eaton

CSEG and Liaisons
My guess is that the problem does involve mpi.What is the path of the ifort command that you are using to compile?  I'm suspicious that the ifort command you are invoking is a wrapper script which is adding paths for mpi include and library under the covers.  To have a successful serial build you'll need to be able to invoke the compiler directly.Another possible way to see whether the cam executable has been linked to an external mpi library is to issue the command% ldd cam | grep mpi 
 
M

moguo_sun@nasa_gov

Member
Hi, Eaton,
       the /usr/local/bin/ifort compiler in fact points to /SPG_ops/utils/x86_64/sles11/intel/bin/intel64/ifortthe ifort is a wrapper as you said, and here is the content. But how can I invoke the compiler itself on the command line? ------------------------------#!/bin/bash

if [ -z "${INTEL_LICENSE_FILE}" ]
then
 INTEL_LICENSE_FILE="/usr/local/intel/licenses:/opt/intel/licenses:${HOME}/intel/licenses:/Users/Shared/Library/Application Support/Intel/Licenses";
else
 INTEL_LICENSE_FILE="${INTEL_LICENSE_FILE}:/usr/local/intel/licenses:/opt/intel/licenses:${HOME}/intel/licenses:/Users/Shared/Library/Application Support/Intel/Licenses";
fi
export INTEL_LICENSE_FILE;

if [ "`uname`" == "Darwin" ]; then
   # DYLD_LIBRARY_PATH is used on MAC OS*
   if [ -z "${DYLD_LIBRARY_PATH}" ]
   then
    DYLD_LIBRARY_PATH="/usr/local/intel/lib:/usr/local/intel/lib/intel64";
   else
    DYLD_LIBRARY_PATH="/usr/local/intel/lib:/usr/local/intel/lib/intel64:${DYLD_LIBRARY_PATH}";
   fi
   export DYLD_LIBRARY_PATH;

   if [ -z "${NLSPATH}" ]
   then
    NLSPATH="/usr/local/intel/lib/locale/en_US/%N";
   else
    NLSPATH="/usr/local/intel/lib/locale/en_US/%N:${NLSPATH}";
   fi
   export NLSPATH;
else
   if [ -z "${LD_LIBRARY_PATH}" ]
   then
    LD_LIBRARY_PATH="/usr/local/intel/lib/intel64";
   else
    LD_LIBRARY_PATH="/usr/local/intel/lib/intel64:${LD_LIBRARY_PATH}";
   fi
   export LD_LIBRARY_PATH;
  if [ -z "${NLSPATH}" ]
  then
     NLSPATH="/usr/local/intel/lib/intel64/locale/%L/%N";
  else
     NLSPATH="/usr/local/intel/lib/intel64/locale/%L/%N:${NLSPATH}";
  fi
  export NLSPATH;
fi

if [ -z "${PATH}" ]
then
 PATH="/usr/local/intel/bin/intel64";
else
 PATH="/usr/local/intel/bin/intel64:${PATH}";
fi
export PATH;

export -n IA32ROOT; unset IA32ROOT;

if [ $# != 0 ]
then
 exec -a "/usr/local/intel/bin/intel64/ifort" /usr/local/intel/bin/intel64/ifortbin "$@";
else
 exec -a "/usr/local/intel/bin/intel64/ifort" /usr/local/intel/bin/intel64/ifortbin;
fi
--------
 
E

eaton

CSEG and Liaisons
The compiler wrapper is just providing locations of libraries that the compiler needs to do its job.  It isn't providing any information about MPI, so my speculation is not holding up.  I have no idea why you encountered the compiler error that mpi_status_ignore was redefined.  The strategy I would pursue is to resolve the problem with mpi_status_ignore.  Return its definition to spmd_utils.F90 and rebuild from scratch.  If you encounter the same error message please compress and attach the entire output from gmake.  
 
M

moguo_sun@nasa_gov

Member
I did that exactly. I found out that I included path /usr/local/include/, in which it has mpif.h in it. That is why mpi_status_ignore is considered redefined in spmd_utils.F90.Later I replace include 'mpif.h' with include "/homedir/msun/cam5/work/bld/mct/mpi-serial/mpif.h"    in spmd_ultis.F90and it works. I compiled successfully and then try to rerun. Unfortunately, I got the same error messages. Any futher insights? thanks for your continuing help.    
 
E

eaton

CSEG and Liaisons
I'm concerned that this is not a complete solution because there are many other modules that use mpif.h and having different modules access different versions of mpif.h seems like a potential problem.I assume that /usr/local/include and /usr/local/lib are the locations of your netcdf files.  I would try removing the /usr/local paths by moving the netcdf files to a different location and trying to build without /usr/local in the search paths. 
 
M

moguo_sun@nasa_gov

Member
Thanks a lot. I did as you suggested and the CAM5 runs better but still has problem.here is the end of the output file (on screen): trcdata_init: data type: CYCLICAL file: oxid_1.9x2.5_L26_1850-2005_c091123.nc
 (GETFIL): attempting to find local file oxid_1.9x2.5_L26_1850-2005_c091123.nc
 (GETFIL): using
 /homedir/msun/cam5/data/atm/cam/chem/trop_mozart_aero/oxid/oxid_1.9x2.5_L26_185
 0-2005_c091123.nc
 Opened existing file
 /homedir/msun/cam5/data/atm/cam/chem/trop_mozart_aero/oxid/oxid_1.9x2.5_L26_185
 0-2005_c091123.nc           7
 open_trc_datafile:
 /homedir/msun/cam5/data/atm/cam/chem/trop_mozart_aero/oxid/oxid_1.9x2.5_L26_185
 0-2005_c091123.nc
 ERROR timeaddmonths():  MM out of range
 set_time_float_from_date: error return from ESMF_TimeSet for set_time_float_fro
 m_date
 ENDRUN:CHKRC
abort:
-------------Thanks. 
 
M

moguo_sun@nasa_gov

Member
Eaton,       I checked /homedir/msun/cam5/data/atm/cam/chem/trop_mozart_aero/oxid/oxid_1.9x2.5_L26_185 and found out the data is not complete. I will re-download it and run again. Thanks. 
 
You must log in or register to reply here.
Top