janezhang8587@yeah_net
Member
When I sumit the $CASE.$MACH.run script, it prints:
.........................................(main) -------------------------------------------------------------------------(main) read input namelist file(main) -------------------------------------------------------------------------(shr_msg_chStdIn) read cpl_stdio.nml, unit 5 connected to cpl.stdin(cpl_control_readNList) ------------------------------------------------------------(cpl_control_readNList) Namelist values BEFORE reading file... &INPARM CASE_NAME = unset , CASE_DESC = unset Fatal error in PMPI_Allgather: Invalid buffer pointer, error stack:PMPI_Allgather(958): MPI_Allgather(sbuf=0x21c26e0, scount=1, MPI_INTEGER, rbuf=0x21c26e0, rcount=1, MPI_INTEGER, comm=0x84000004) failedPMPI_Allgather(931): Buffers must not be aliased , START_TYPE = initial , START_DATE = 10101, START_PFILE = unset (shr_msg_chStdIn) read ice_stdio.nml, unit 5 connected to ice.stdin , START_BFILE = unset , REST_OPTION = monthly , REST_N = 3, REST_DATE = -999, STOP_OPTION = monthly , STOP_N = 3, STOP_DATE = -999, HIST_OPTION = monthly , HIST_N = 3, HIST_DATE = -999, HIST_64BIT = F, AVHIST_OPTION = none , AVHIST_N = 1, AVHIST_DATE = -999, DIAG_OPTION = monthly , DIAG_N = 3, DIAG_DATE = -999, AVDIAG_OPTION = yearly , AVDIAG_N = 3, AVDIAG_DATE = -999, MAP_A2OF_FN = unknown , MAP_A2OS_FN = unknown , MAP_O2AF_FN = unknown , MAP_R2O_FN = unknown , ORB_YEAR = -999, ORB_ECCEN = -999.0000000000000 , ORB_MVELP = -999.0000000000000 , ORB_OBLIQ = -999.0000000000000 , FLX_ALBAV = F, FLX_EPBAL = off , INFO_DBUG = 1, INFO_BCHECK = 0, DECOMP_AL = 1, DECOMP_OI = 1, DECOMP_R = 1, BFBFLAG = F /(cpl_control_readNList) ------------------------------------------------------------(cpl_control_readNList) ------------------------------------------------------------(cpl_control_readNList) Namelist values AFTER reading file... &INPARM CASE_NAME = testB6 , CASE_DESC = testB6 testB6 , START_TYPE = initial , START_DATE = 10101, START_PFILE = rpointer.cpl , START_BFILE = null , REST_OPTION = ndays , REST_N = 5, REST_DATE = -999, STOP_OPTION = ndays , STOP_N = 5, STOP_DATE = -999, HIST_OPTION = never , HIST_N = -999, HIST_DATE = -999, HIST_64BIT = F, AVHIST_OPTION = never , AVHIST_N = -999, AVHIST_DATE = -999, DIAG_OPTION = ndays , DIAG_N = 10, DIAG_DATE = -999, AVDIAG_OPTION = yearly , AVDIAG_N = 3, AVDIAG_DATE = -999, MAP_A2OF_FN = map_T31_to_gx3v5_aave_da_040122.nc , MAP_A2OS_FN = map_T31_to_gx3v5_bilin_da_040122.nc , MAP_O2AF_FN = map_gx3v5_to_T31_aave_da_040122.nc , MAP_R2O_FN = map_r05_to_gx3v5_e2000r500_040209.nc , ORB_YEAR = 1990, ORB_ECCEN = -999.0000000000000 , ORB_MVELP = -999.0000000000000 , ORB_OBLIQ = -999.0000000000000 , FLX_ALBAV = F, FLX_EPBAL = off , INFO_DBUG = 1, INFO_BCHECK = 0, DECOMP_AL = 1, DECOMP_OI = 1, DECOMP_R = 1, BFBFLAG = F /(cpl_control_readNList) ------------------------------------------------------------(cpl_control_readNList) orbit based on orb_year = 1990(shr_orb_params) Calculate characteristics of the orbit:(shr_orb_params) CVS revision: $Revision: 1.2 $(shr_orb_params) CVS Tag : $Name: ccsm3_0_rel04 $(shr_orb_params) Calculate orbit for year: 1990(shr_orb_params) ------ Computed Orbital Parameters ------(shr_orb_params) Eccentricity = 1.670772E-02(shr_orb_params) Obliquity (deg) = 2.344107E+01(shr_orb_params) Obliquity (rad) = 4.091238E-01(shr_orb_params) Long of perh(deg) = 1.027242E+02(shr_orb_params) Long of perh(rad) = 4.934468E+00(shr_orb_params) Long at v.e.(rad) = -3.250364E-02(shr_orb_params) -----------------------------------------(main) -------------------------------------------------------------------------(main) get simulation start date(main) -------------------------------------------------------------------------(restart_readDate) restart type = initial => start date specified by input namelist(main) simulation start date is 00010101(main) -------------------------------------------------------------------------(main) contract init: establishes domains & routers (excluding lnd)(main) -------------------------------------------------------------------------Fatal error in PMPI_Allgather: Other MPI error, error stack:PMPI_Allgather(958).......: MPI_Allgather(sbuf=0x1a4f578, scount=1, MPI_INTEGER, rbuf=0x1a4f570, rcount=1, MPI_INTEGER, comm=0x84000004) failedMPIR_Allgather_impl(805)..: MPIR_Allgather(766).......: MPIR_Allgather_intra(524).: dequeue_and_set_error(596): Communication error with rank 10(cpl_contract_init) cpl-recv-atm(shr_sys_abort) ERROR: ice: Namelist read error in ice_init.F(shr_sys_abort) WARNING: calling shr_mpi_abort() and stoppingFatal error in PMPI_Allgather: Other MPI error, error stack:PMPI_Allgather(958).......: MPI_Allgather(sbuf=0x22466c0, scount=1, MPI_INTEGER, rbuf=0x22466b0, rcount=1, MPI_INTEGER, comm=0x84000004) failedMPIR_Allgather_impl(805)..: MPIR_Allgather(766).......: MPIR_Allgather_intra(501).: dequeue_and_set_error(596): Communication error with rank 10(shr_sys_abort) ERROR: ice: Namelist read error in ice_init.F(shr_sys_abort) WARNING: calling shr_mpi_abort() and stopping (shr_mpi_abort):ice: Namelist read error in ice_init.F(shr_sys_abort) ERROR: ice: Namelist read error in ice_init.F(shr_sys_abort) WARNING: calling shr_mpi_abort() and stopping(shr_sys_abort) ERROR: ice: Namelist read error in ice_init.F(shr_sys_abort) WARNING: calling shr_mpi_abort() and stoppingFatal error in PMPI_Barrier: Other MPI error, error stack:PMPI_Barrier(425)...............: MPI_Barrier(comm=0x84000004) failedMPIR_Barrier_impl(306)..........: MPIR_Bcast_impl(1321)...........: MPIR_Bcast_intra(1155)..........: MPIR_Bcast_binomial(213)........: Failure during collectiveMPIR_Barrier_impl(292)..........: MPIR_Barrier_or_coll_fn(121)....: MPIR_Barrier_intra(83)..........: MPIDI_CH3U_Recvq_FDU_or_AEP(380): Communication error with rank 18MPIR_Barrier_intra(83)..........: dequeue_and_set_error(596)......: Communication error with rank 20 (shr_mpi_abort):ice: Namelist read error in ice_init.F (shr_mpi_abort):ice: Namelist read error in ice_init.F (shr_mpi_abort):ice: Namelist read error in ice_init.FFatal error in PMPI_Barrier: Other MPI error, error stack:PMPI_Barrier(425)...............: MPI_Barrier(comm=0x84000004) failedMPIR_Barrier_impl(306)..........: MPIR_Bcast_impl(1321)...........: MPIR_Bcast_intra(1155)..........: MPIR_Bcast_binomial(213)........: Failure during collectiveMPIR_Barrier_impl(292)..........: MPIR_Barrier_or_coll_fn(121)....: MPIR_Barrier_intra(83)..........: MPIDI_CH3U_Recvq_FDU_or_AEP(380): Communication error with rank 20 --------------------------------------------------------------------------------------I do not how to work it out. After running the build script, it echos " CCSM BUILD HAS FINISHED SUCCESSFULLY ". But I check the file $EXE/cpl/cpl.stdin, it is an empty file, I have no idea what's wrong about it. the ice.stdin (in $EXEROOT/ice/) is also empty. In addition, there is no a file named ocn.stdin in $EXEROOT/ocn. Jian
.........................................(main) -------------------------------------------------------------------------(main) read input namelist file(main) -------------------------------------------------------------------------(shr_msg_chStdIn) read cpl_stdio.nml, unit 5 connected to cpl.stdin(cpl_control_readNList) ------------------------------------------------------------(cpl_control_readNList) Namelist values BEFORE reading file... &INPARM CASE_NAME = unset , CASE_DESC = unset Fatal error in PMPI_Allgather: Invalid buffer pointer, error stack:PMPI_Allgather(958): MPI_Allgather(sbuf=0x21c26e0, scount=1, MPI_INTEGER, rbuf=0x21c26e0, rcount=1, MPI_INTEGER, comm=0x84000004) failedPMPI_Allgather(931): Buffers must not be aliased , START_TYPE = initial , START_DATE = 10101, START_PFILE = unset (shr_msg_chStdIn) read ice_stdio.nml, unit 5 connected to ice.stdin , START_BFILE = unset , REST_OPTION = monthly , REST_N = 3, REST_DATE = -999, STOP_OPTION = monthly , STOP_N = 3, STOP_DATE = -999, HIST_OPTION = monthly , HIST_N = 3, HIST_DATE = -999, HIST_64BIT = F, AVHIST_OPTION = none , AVHIST_N = 1, AVHIST_DATE = -999, DIAG_OPTION = monthly , DIAG_N = 3, DIAG_DATE = -999, AVDIAG_OPTION = yearly , AVDIAG_N = 3, AVDIAG_DATE = -999, MAP_A2OF_FN = unknown , MAP_A2OS_FN = unknown , MAP_O2AF_FN = unknown , MAP_R2O_FN = unknown , ORB_YEAR = -999, ORB_ECCEN = -999.0000000000000 , ORB_MVELP = -999.0000000000000 , ORB_OBLIQ = -999.0000000000000 , FLX_ALBAV = F, FLX_EPBAL = off , INFO_DBUG = 1, INFO_BCHECK = 0, DECOMP_AL = 1, DECOMP_OI = 1, DECOMP_R = 1, BFBFLAG = F /(cpl_control_readNList) ------------------------------------------------------------(cpl_control_readNList) ------------------------------------------------------------(cpl_control_readNList) Namelist values AFTER reading file... &INPARM CASE_NAME = testB6 , CASE_DESC = testB6 testB6 , START_TYPE = initial , START_DATE = 10101, START_PFILE = rpointer.cpl , START_BFILE = null , REST_OPTION = ndays , REST_N = 5, REST_DATE = -999, STOP_OPTION = ndays , STOP_N = 5, STOP_DATE = -999, HIST_OPTION = never , HIST_N = -999, HIST_DATE = -999, HIST_64BIT = F, AVHIST_OPTION = never , AVHIST_N = -999, AVHIST_DATE = -999, DIAG_OPTION = ndays , DIAG_N = 10, DIAG_DATE = -999, AVDIAG_OPTION = yearly , AVDIAG_N = 3, AVDIAG_DATE = -999, MAP_A2OF_FN = map_T31_to_gx3v5_aave_da_040122.nc , MAP_A2OS_FN = map_T31_to_gx3v5_bilin_da_040122.nc , MAP_O2AF_FN = map_gx3v5_to_T31_aave_da_040122.nc , MAP_R2O_FN = map_r05_to_gx3v5_e2000r500_040209.nc , ORB_YEAR = 1990, ORB_ECCEN = -999.0000000000000 , ORB_MVELP = -999.0000000000000 , ORB_OBLIQ = -999.0000000000000 , FLX_ALBAV = F, FLX_EPBAL = off , INFO_DBUG = 1, INFO_BCHECK = 0, DECOMP_AL = 1, DECOMP_OI = 1, DECOMP_R = 1, BFBFLAG = F /(cpl_control_readNList) ------------------------------------------------------------(cpl_control_readNList) orbit based on orb_year = 1990(shr_orb_params) Calculate characteristics of the orbit:(shr_orb_params) CVS revision: $Revision: 1.2 $(shr_orb_params) CVS Tag : $Name: ccsm3_0_rel04 $(shr_orb_params) Calculate orbit for year: 1990(shr_orb_params) ------ Computed Orbital Parameters ------(shr_orb_params) Eccentricity = 1.670772E-02(shr_orb_params) Obliquity (deg) = 2.344107E+01(shr_orb_params) Obliquity (rad) = 4.091238E-01(shr_orb_params) Long of perh(deg) = 1.027242E+02(shr_orb_params) Long of perh(rad) = 4.934468E+00(shr_orb_params) Long at v.e.(rad) = -3.250364E-02(shr_orb_params) -----------------------------------------(main) -------------------------------------------------------------------------(main) get simulation start date(main) -------------------------------------------------------------------------(restart_readDate) restart type = initial => start date specified by input namelist(main) simulation start date is 00010101(main) -------------------------------------------------------------------------(main) contract init: establishes domains & routers (excluding lnd)(main) -------------------------------------------------------------------------Fatal error in PMPI_Allgather: Other MPI error, error stack:PMPI_Allgather(958).......: MPI_Allgather(sbuf=0x1a4f578, scount=1, MPI_INTEGER, rbuf=0x1a4f570, rcount=1, MPI_INTEGER, comm=0x84000004) failedMPIR_Allgather_impl(805)..: MPIR_Allgather(766).......: MPIR_Allgather_intra(524).: dequeue_and_set_error(596): Communication error with rank 10(cpl_contract_init) cpl-recv-atm(shr_sys_abort) ERROR: ice: Namelist read error in ice_init.F(shr_sys_abort) WARNING: calling shr_mpi_abort() and stoppingFatal error in PMPI_Allgather: Other MPI error, error stack:PMPI_Allgather(958).......: MPI_Allgather(sbuf=0x22466c0, scount=1, MPI_INTEGER, rbuf=0x22466b0, rcount=1, MPI_INTEGER, comm=0x84000004) failedMPIR_Allgather_impl(805)..: MPIR_Allgather(766).......: MPIR_Allgather_intra(501).: dequeue_and_set_error(596): Communication error with rank 10(shr_sys_abort) ERROR: ice: Namelist read error in ice_init.F(shr_sys_abort) WARNING: calling shr_mpi_abort() and stopping (shr_mpi_abort):ice: Namelist read error in ice_init.F(shr_sys_abort) ERROR: ice: Namelist read error in ice_init.F(shr_sys_abort) WARNING: calling shr_mpi_abort() and stopping(shr_sys_abort) ERROR: ice: Namelist read error in ice_init.F(shr_sys_abort) WARNING: calling shr_mpi_abort() and stoppingFatal error in PMPI_Barrier: Other MPI error, error stack:PMPI_Barrier(425)...............: MPI_Barrier(comm=0x84000004) failedMPIR_Barrier_impl(306)..........: MPIR_Bcast_impl(1321)...........: MPIR_Bcast_intra(1155)..........: MPIR_Bcast_binomial(213)........: Failure during collectiveMPIR_Barrier_impl(292)..........: MPIR_Barrier_or_coll_fn(121)....: MPIR_Barrier_intra(83)..........: MPIDI_CH3U_Recvq_FDU_or_AEP(380): Communication error with rank 18MPIR_Barrier_intra(83)..........: dequeue_and_set_error(596)......: Communication error with rank 20 (shr_mpi_abort):ice: Namelist read error in ice_init.F (shr_mpi_abort):ice: Namelist read error in ice_init.F (shr_mpi_abort):ice: Namelist read error in ice_init.FFatal error in PMPI_Barrier: Other MPI error, error stack:PMPI_Barrier(425)...............: MPI_Barrier(comm=0x84000004) failedMPIR_Barrier_impl(306)..........: MPIR_Bcast_impl(1321)...........: MPIR_Bcast_intra(1155)..........: MPIR_Bcast_binomial(213)........: Failure during collectiveMPIR_Barrier_impl(292)..........: MPIR_Barrier_or_coll_fn(121)....: MPIR_Barrier_intra(83)..........: MPIDI_CH3U_Recvq_FDU_or_AEP(380): Communication error with rank 20 --------------------------------------------------------------------------------------I do not how to work it out. After running the build script, it echos " CCSM BUILD HAS FINISHED SUCCESSFULLY ". But I check the file $EXE/cpl/cpl.stdin, it is an empty file, I have no idea what's wrong about it. the ice.stdin (in $EXEROOT/ice/) is also empty. In addition, there is no a file named ocn.stdin in $EXEROOT/ocn. Jian