ERROR !!!:forrtl: severe (174): SIGSEGV, segmentation fault occurred

[lxiaojuan8778@163_com]

Hi All,
I have built cam3 executable using "ifort" and "cc" in my intel
cluster.I have a problem in running a control run.

error:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Attempting to read PFT physiological data .....
(GETFIL): attempting to find local file pft-physiology
(GETFIL): using /disk1/ljp/liuyz/inputdata/lnd/clm2/pftdata/pft-physiology
(OPNFIL): Successfully opened file
/disk1/ljp/liuyz/inputdata/lnd/clm2/pftdata/pft-physiology on unit=
7
Successfully read PFT physiological data

Attempting to read surface boundary data .....
(GETFIL): attempting to find local file clms_64x128_USGS_c030605.nc
(GETFIL): using
/disk1/ljp/liuyz/inputdata/lnd/clm2/srfdata/cam/clms_64x128_USGS_c030605.nc
Successfully read surface boundary data

Surface Grid Characteristics
longitude points = 128
latitude points = 64
minimum number of longitude points per latitude = 128
maximum number of longitude points per latitude = 128
total number of gridcells = 3886
total number of landunits = 4897
total number of columns = 4897
total number of pfts = 11260
Decomposition Characteristics
clumps per process = 1

forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
cam 00000000004D94D7 Unknown Unknown Unknown
cam 00000000005C2B6D Unknown Unknown Unknown
cam 0000000000470376 Unknown Unknown Unknown
cam 00000000005BF69E Unknown Unknown Unknown
cam 0000000000476A7F Unknown Unknown Unknown
cam 0000000000404E9C Unknown Unknown Unknown
libc.so.6 0000003D55C1D974 Unknown Unknown Unknown
cam 0000000000404DA9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
How can I solve this problem?

Thanks in advance

 

[eaton]

I can't see what's wrong from the log output. My recommendation is to try the new CAM4 version which was just released (as part of CCSM4). We provided some clues for using ifort with CAM3, but we were not regularly testing using ifort. CAM4 has been regularly tested using ifort, so I think your chances of using it successfully are much improved.

 

[lxiaojuan8778@163_com]

I can't see what's wrong from the log output. My recommendation is to try the new CAM4 version which was just released (as part of CCSM4). We provided some clues for using ifort with CAM3, but we were not regularly testing using ifort. CAM4 has been regularly tested using ifort, so I think your chances of using it successfully are much improved.

I found what was wrong ,and I have changed someting in Makefile .it works.

 

[eaton]

Did you need to edit the Makefile directly to implement the changes you needed? If so and you'd like to post the changes then I'll see whether it's possible to modify the configure script so that the specific change you needed could be accomplished by specifying an appropriate argument. My goal with configure is that it provide the necessary flexibility so that hand editing of Makefiles is unnecessary.

 

[lxiaojuan8778@163_com]

Did you need to edit the Makefile directly to implement the changes you needed? If so and you'd like to post the changes then I'll see whether it's possible to modify the configure script so that the specific change you needed could be accomplished by specifying an appropriate argument. My goal with configure is that it provide the necessary flexibility so that hand editing of Makefiles is unnecessary.

I found there is only ifort but not mpiifort,so I change someting in
cam1/..../bld/Makefile
#lcx ifeq ($(findstring ifort,$(FC)),ifort)
ifeq ($(findstring mpiifort,$(FC)),mpiifort)
ESMF_ARCH := linux_intel
mod_path := -I$(ESMF_MOD)/$(ESMF_ARCH) -I$(MOD_NETCDF)
FFLAGS := $(cpp_path) $(mod_path) $(CPPDEF) -132 -r8 -autodouble -ftz -g
SPEC_FFLAGS := $(FFLAGS)
LDFLAGS :=
FREEFLAGS := -FR
ifeq ($(DEBUG),TRUE)
FFLAGS += -CB
else
# Check for override of default Fortran compiler optimizations
ifeq ($(F_OPTIMIZATION_OVERRIDE),$(null))
#lcx FORTRAN_OPTIMIZATION := -O2
FORTRAN_OPTIMIZATION := -O3
endif
FFLAGS += $(FORTRAN_OPTIMIZATION)

Did this change cause difference from multiple identical runs or will vectorization cause this error??I have no idea.

thank you !!!

 

[eaton]

Thanks for the feedback. The use of compiler wrappers such as mpiifort is an issue that I plan to address with configure.

Increasing the optimization is always something that must be done very carefully. The change from -O2 to -O3 certainly has the potential to cause irreproducible results. The best way to check is simply to turn the optimization back down. You may even want to try turning the optimization off entirely.