Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

POP.F terminated by signal 9

M

mengpai@gmail_com

New Member
Hi there,
I am building the ccsm3 on IBM AMD cluster. The compliation passes through lnd, cpl, ice, but crased at pop with a message: TERMINATED by signal 9. I was wondering if anybody have solved the same problem before, or any suggestions to the solution. You will be really appreciated.
Thank you.
-Odden

Due to the text limitation, I cut the lines for the part of successful compiling files.

My environment is:
Machine: IBM AMD Opteron 1350 cluster with Linux
mpi: mvapich 0.9.9
pgi: 7.0-4
Batch: PBS
netCDF: netcdf 3.6.2
Network: Infiniband
Rrunning message:
## Warning: SCRATCH not defined in system environment. Set SCRATCH to be /home/usr4918
-------------------------------------------------------------------------
Preparing T31_gx3v5 component models for execution
-------------------------------------------------------------------------
- Create execution directories for atm,cpl,lnd,ice,ocn
- If a restart run then copy restart files into executable directory
ccsm_getrestart: get /home/usr4918 restarts from /home/usr4918/archive/TER.01a.T31_gx3v5.B.glenn.131725/restart
- Check validity of configuration
- Determine if build must happen (env variable BLDTYPE)
- Build flag (BLDTYPE) is TRUE
- Build Libraries: esmf, mph, mct
Fri Feb 15 16:54:20 EST 2008 esmf.buildlib.080215-165420
Fri Feb 15 16:54:21 EST 2008 mph.buildlib.080215-165420
Fri Feb 15 16:54:21 EST 2008 mct.buildlib.080215-165420
- Create model directories for each platform
- Determine if models must be rebuilt
- Build model executables, create namelist files, prestage input data
Fri Feb 15 16:54:23 EST 2008 /home/usr4918/cpl/cpl.log.080215-165420
Fri Feb 15 16:54:23 EST 2008 /home/usr4918/cpl/cpl.buildexe.080215-165420
Fri Feb 15 16:54:24 EST 2008 /home/usr4918/ice/ice.log.080215-165420
Fri Feb 15 16:54:24 EST 2008 /home/usr4918/ice/ice.buildexe.080215-165420
Fri Feb 15 16:54:25 EST 2008 /home/usr4918/lnd/lnd.log.080215-165420
Fri Feb 15 16:54:25 EST 2008 /home/usr4918/lnd/lnd.buildexe.080215-165420
Fri Feb 15 16:54:26 EST 2008 /home/usr4918/ocn/ocn.log.080215-165420
Fri Feb 15 16:54:26 EST 2008 /home/usr4918/ocn/ocn.buildexe.080215-165420
ERROR: pop.buildexe.csh failed, see ocn.buildexe.080215-165420
ERROR: cat /home/usr4918/ocn/ocn.buildexe.080215-165420

ocn.buildexe.080215-165420 shows:
Tue Feb 19 13:22:10 EST 2008 /tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/ocn/ocn.buildexe.080219-132022
## Warning: SCRATCH not defined in system environment. Set SCRATCH to be /home/usr4918
-------------------------------------------------------------------------
a. Build an executable in /tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/ocn/obj
-------------------------------------------------------------------------
a.1 Calculate processor tiling based on 24
-----------------------------------------------------------------
a2. creating the internal directory structure
-----------------------------------------------------------------
a3. create model_size.F in /tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/ocn/obj/source, first computing NT
-----------------------------------------------------------------
ocn.iage.setup.csh : setting nt
-----------------------------------------------------------------
copy the necessary files into $source_dir
-----------------------------------------------------------------
cp: No match.
cp: No match.
diff: /tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/ocn/obj/ocnres.old: No such file or directory
-----------------------------------------------------------------
a4. compile and copy the executable into /tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/ocn directory
-----------------------------------------------------------------
Building dependency for xdisplay.d
Building dependency for vmix_rich.d
...(successfully passes through)
Building dependency for baroclinic.d
Building dependency for advection.d
Building dependency for POP.d
mpif90 -I. -I/usr/local/netcdf-3.6.2/include -I/usr/include -I/tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/lib/include -I/usr/local/mpi/mvapich-0.9.8-pgi/include -I/usr/local/pgi/linux86-64/7.0-4/include -I/tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/ocn/obj/source -c -r8 -i4 -Kieee -Mdalign -Mextend -Msave -tp amd64 -O3 -byteswapio -DLINUX -DPGF90 -DNO_SHR_VMATH -DLINUX -DPOSIX -Dimpvmix -Dcoupled -DNPROC_X=4 -DNPROC_Y=6 /tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/ocn/obj/source/kinds_mod.F
mpif90 -I. -I/usr/local/netcdf-3.6.2/include -I/usr/include -I/tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/lib/include -I/usr/local/mpi/mvapich-0.9.8-pgi/include -I/usr/local/pgi/linux86-64/7.0-4/include -I/tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/ocn/obj/source -c -r8 -i4 -Kieee -Mdalign -Mextend -Msave -tp amd64 -O3 -byteswapio -DLINUX -DPGF90 -DNO_SHR_VMATH -DLINUX -DPOSIX -Dimpvmix -Dcoupled -DNPROC_X=4 -DNPROC_Y=6 /tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/ocn/obj/source/model_size.F
...(successfully passes through)
mpif90 -I. -I/usr/local/netcdf-3.6.2/include -I/usr/include -I/tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/lib/include -I/usr/local/mpi/mvapich-0.9.8-pgi/include -I/usr/local/pgi/linux86-64/7.0-4/include -I/tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/ocn/obj/source -c -r8 -i4 -Kieee -Mdalign -Mextend -Msave -tp amd64 -O3 -byteswapio -DLINUX -DPGF90 -DNO_SHR_VMATH -DLINUX -DPOSIX -Dimpvmix -Dcoupled -DNPROC_X=4 -DNPROC_Y=6 /tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/ocn/obj/source/forcing.F
mpif90 -I. -I/usr/local/netcdf-3.6.2/include -I/usr/include -I/tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/lib/include -I/usr/local/mpi/mvapich-0.9.8-pgi/include -I/usr/local/pgi/linux86-64/7.0-4/include -I/tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/ocn/obj/source -c -r8 -i4 -Kieee -Mdalign -Mextend -Msave -tp amd64 -O3 -byteswapio -DLINUX -DPGF90 -DNO_SHR_VMATH -DLINUX -DPOSIX -Dimpvmix -Dcoupled -DNPROC_X=4 -DNPROC_Y=6 /tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/ocn/obj/source/diagnostics.F
mpif90 -I. -I/usr/local/netcdf-3.6.2/include -I/usr/include -I/tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/lib/include -I/usr/local/mpi/mvapich-0.9.8-pgi/include -I/usr/local/pgi/linux86-64/7.0-4/include -I/tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/ocn/obj/source -c -r8 -i4 -Kieee -Mdalign -Mextend -Msave -tp amd64 -O3 -Mfree -DLINUX -DPGF90 -DNO_SHR_VMATH -DLINUX -DPOSIX -Dimpvmix -Dcoupled -DNPROC_X=4 -DNPROC_Y=6 /tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/ocn/obj/source/shr_vmath_mod.F90
...(successfully passes through)
mpif90 -I. -I/usr/local/netcdf-3.6.2/include -I/usr/include -I/tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/lib/include -I/usr/local/mpi/mvapich-0.9.8-pgi/include -I/usr/local/pgi/linux86-64/7.0-4/include -I/tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/ocn/obj/source -c -r8 -i4 -Kieee -Mdalign -Mextend -Msave -tp amd64 -O3 -byteswapio -DLINUX -DPGF90 -DNO_SHR_VMATH -DLINUX -DPOSIX -Dimpvmix -Dcoupled -DNPROC_X=4 -DNPROC_Y=6 /tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/ocn/obj/source/initial.F
mpif90 -I. -I/usr/local/netcdf-3.6.2/include -I/usr/include -I/tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/lib/include -I/usr/local/mpi/mvapich-0.9.8-pgi/include -I/usr/local/pgi/linux86-64/7.0-4/include -I/tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/ocn/obj/source -c -r8 -i4 -Kieee -Mdalign -Mextend -Msave -tp amd64 -O3 -byteswapio -DLINUX -DPGF90 -DNO_SHR_VMATH -DLINUX -DPOSIX -Dimpvmix -Dcoupled -DNPROC_X=4 -DNPROC_Y=6 /tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/ocn/obj/source/POP.F
pgf90-Fatal-/usr/local/pgi/linux86-64/7.0-4/bin/pgf901 TERMINATED by signal 9
Arguments to /usr/local/pgi/linux86-64/7.0-4/bin/pgf901
/usr/local/pgi/linux86-64/7.0-4/bin/pgf901 /tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/ocn/obj/source/POP.F -opt 3 -terse 1 -inform warn -nohpf -nostatic -x 19 0x400000 -quad -x 59 4 -x 59 4 -x 15 2 -x 49 0x400004 -x 51 0x20 -x 57 0x4c -x 58 0x10000 -x 124 0x1000 -x 57 0xfb0000 -x 58 0x78031040 -x 70 0x6c00 -x 47 0x400000 -x 48 4608 -x 49 0x100 -x 120 0x200 -stdinc /usr/local/pgi/linux86-64/7.0-4/include-gcc41:/usr/local/pgi/linux86-64/7.0-4/include:/usr/local/include:/usr/lib/gcc/x86_64-redhat-linux/4.1.1/include:/usr/lib/gcc/x86_64-redhat-linux/4.1.1/include:/usr/include -def unix -def __unix -def __unix__ -def linux -def __linux -def __linux__ -def __NO_MATH_INLINES -def __x86_64__ -def __LONG_MAX__=9223372036854775807L -def '__SIZE_TYPE__=unsigned long int' -def '__PTRDIFF_TYPE__=long int' -def __THROW= -def __extension__= -def __amd64__ -def __SSE__ -def __MMX__ -def __SSE2__ -idir . -idir /usr/local/netcdf-3.6.2/include -idir /usr/include -idir /tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/lib/include -idir /usr/local/mpi/mvapich-0.9.8-pgi/include -idir /usr/local/pgi/linux86-64/7.0-4/include -idir /tmp/usr4918/TER.01a.T31_gx3v5.B.glenn.131725/ocn/obj/source -idir /usr/local/mpi/mvapich-0.9.9-pgi-fixed-shmem/include/f90base -def LINUX -def PGF90 -def NO_SHR_VMATH -def LINUX -def POSIX -def impvmix -def coupled -def NPROC_X=4 -def NPROC_Y=6 -preprocess -nofreeform -i4 -extend -x 125 2 -vect 48 -x 124 0x8 -save -output /tmp/pgf90H8WdJeuM2hA6.ilm
gmake: *** [POP.o] Error 127
 
You must log in or register to reply here.
Top