some question in cesm.exe

[lianghb]

Hi,
While running the CTSM, the model setup and built has been successful. But when I executing the cesm.exe, it shows a error in pioc_support.c at line 2832. I have tried to recompile the parallelio with intel MPI, but the error was shown again. I don't know how to deal with it. I have attached the log file and the code of pioc_support.c at line 2832.

log file:
Abort with message Unknown error in file operation in file ../CTSM/ctsm_MYBRANCH/libraries/parallelio/src/clib/pioc_support.c at line 2832
Abort with message Unknown error in file operation in file ../CTSM/ctsm_MYBRANCH/libraries/parallelio/src/clib/pioc_support.c at line 2832
../CTSM/scratch/mycase72/bld/cesm.exe() [0x1139909]
../CTSM/scratch/mycase72/bld/cesm.exe() [0x1137890]
../CTSM/scratch/mycase72/bld/cesm.exe() [0x113781c]
../CTSM/scratch/mycase72/bld/cesm.exe() [0x1138232]
../CTSM/scratch/mycase72/bld/cesm.exe(PIOc_openfile+0x16) [0x1133436]
../CTSM/scratch/mycase72/bld/cesm.exe() [0x10f16b1]
../CTSM/scratch/mycase72/bld/cesm.exe() [0xf4ccc0]
../CTSM/scratch/mycase72/bld/cesm.exe() [0xf4b68b]
../CTSM/scratch/mycase72/bld/cesm.exe() [0xf3b6fe]
../CTSM/scratch/mycase72/bld/cesm.exe() [0xf35f78]
Abort(-1) on node 0 (rank 0 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, -1) - process 0

pioc_support.c:
/* Broadcast and check the return code. */
if (ios->ioroot == ios->union_rank)
PLOG((2, "Bcasting error code ierr %d ios->ioroot %d ios->my_comm %d",
ierr, ios->ioroot, ios->my_comm));
if ((mpierr = MPI_Bcast(&ierr, 1, MPI_INT, ios->ioroot, ios->my_comm)))
return check_mpi(NULL, file, mpierr, __FILE__, __LINE__);
PLOG((2, "Bcast openfile_retry error code ierr = %d", ierr));
/* If there was an error, free allocated memory and deal with the error. */
if (ierr)
{
free(file);
2832 return check_netcdf2(ios, NULL, ierr, __FILE__, __LINE__);
}
/* Broadcast writability to all tasks. */
if ((mpierr = MPI_Bcast(&file->writable, 1, MPI_INT, ios->ioroot, ios->my_comm)))
return check_mpi(NULL, file, mpierr, __FILE__, __LINE__);

 

[oleson]

No log file attached.

 

[lianghb]

Thanks for your reminder.
I resubmit the log file.

 

[lianghb]

(shr_orb_params) ------ Computed Orbital Parameters ------
(shr_orb_params) Eccentricity = 1.670366E-02
(shr_orb_params) Obliquity (deg) = 2.343977E+01
(shr_orb_params) Obliquity (rad) = 4.091011E-01
(shr_orb_params) Long of perh(deg) = 1.028955E+02
(shr_orb_params) Long of perh(rad) = 4.937458E+00
(shr_orb_params) Long at v.e.(rad) = -3.247250E-02
(shr_orb_params) -----------------------------------------
Abort with message Unknown error in file operation in file /stu01/lianghb21/CTSM/ctsm_MYBRANCH_old/libraries/parallelio/src/clib/pioc_support.c at line 2832
Abort with message Unknown error in file operation in file /stu01/lianghb21/CTSM/ctsm_MYBRANCH_old/libraries/parallelio/src/clib/pioc_support.c at line 2832
Obtained 10 stack frames.
/stu01/lianghb21/CTSM/scratch/mycase72/bld/cesm.exe() [0x1139909]
/stu01/lianghb21/CTSM/scratch/mycase72/bld/cesm.exe() [0x1137890]
/stu01/lianghb21/CTSM/scratch/mycase72/bld/cesm.exe() [0x113781c]
/stu01/lianghb21/CTSM/scratch/mycase72/bld/cesm.exe() [0x1138232]
/stu01/lianghb21/CTSM/scratch/mycase72/bld/cesm.exe(PIOc_openfile+0x16) [0x1133436]
/stu01/lianghb21/CTSM/scratch/mycase72/bld/cesm.exe() [0x10f16b1]
/stu01/lianghb21/CTSM/scratch/mycase72/bld/cesm.exe() [0xf4ccc0]
/stu01/lianghb21/CTSM/scratch/mycase72/bld/cesm.exe() [0xf4b68b]
/stu01/lianghb21/CTSM/scratch/mycase72/bld/cesm.exe() [0xf3b6fe]
/stu01/lianghb21/CTSM/scratch/mycase72/bld/cesm.exe() [0xf35f78]
Abort(-1) on node 0 (rank 0 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, -1) - process 0
Obtained 10 stack frames.
/stu01/lianghb21/CTSM/scratch/mycase72/bld/cesm.exe() [0x1139909]
/stu01/lianghb21/CTSM/scratch/mycase72/bld/cesm.exe() [0x1137890]
/stu01/lianghb21/CTSM/scratch/mycase72/bld/cesm.exe() [0x113781c]
/stu01/lianghb21/CTSM/scratch/mycase72/bld/cesm.exe() [0x1138232]
/stu01/lianghb21/CTSM/scratch/mycase72/bld/cesm.exe(PIOc_openfile+0x16) [0x1133436]
/stu01/lianghb21/CTSM/scratch/mycase72/bld/cesm.exe() [0x10f16b1]
/stu01/lianghb21/CTSM/scratch/mycase72/bld/cesm.exe() [0xf4ccc0]
/stu01/lianghb21/CTSM/scratch/mycase72/bld/cesm.exe() [0xf4b68b]
/stu01/lianghb21/CTSM/scratch/mycase72/bld/cesm.exe() [0xf3b6fe]
/stu01/lianghb21/CTSM/scratch/mycase72/bld/cesm.exe() [0xf35f78]
Abort(-1) on node 1 (rank 1 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, -1) - process 1

 

[oleson]

Thanks for providing the log file (although which log file is this and is this the entire file?). Unfortunately, it doesn't shed any light on what the problem might be, at least from my perspective. I suggest you look at all of the other log files also to see if there is any helpful information there.
If you haven't run the model before on your machine, then I suggest you follow the porting instructions here:

7. Porting and validating CIME on a new platform — CIME master documentation

and post any questions regarding this to the Infrastructure Forum:

Infrastructure (CIME, porting, machines, scripts)

CIME, case control system, build and configure, machines and supporting scripts.

bb.cgd.ucar.edu

If you have successfully run the model before on your machine, then please provide the information noted here and we will see if we can help further:

Information to include in help requests

When you are posting to the forums because you want help with an issue, please include as much of the following information as possible. Please read the following before submitting a help request: Check the CESM support policy, and particularly the list of currently-supported versions at the...

bb.cgd.ucar.edu

 

[lianghb]

Thanks for your answer!
I will try as your say.