luoyangcheng96@pku_edu_cn
New Member
Hi,I need to start running CAM5 (F_2000_CAM5 in CESM1.2.2.1) from November 1st. I modified the initial data (ncdata in run/atm.in) and set the RUN_STARTDATE variable in env_run.xml to be 0002-11-01. I ran the model for 3 days for a test. However, the model crashed at the end of the third day, reporting the following lines in cesm.log: 128: box_rearrange::compute_dest:: ERROR: no destination found for compdof=128: 1072693248128: box_rearrange::compute_dest:: INFO: gsize= 392252128: box_rearrange::compute_dest:: INFO: nioproc 4 ioproc 1 128: ioindex -1128: box_rearrange::compute_dest:: INFO io 1 start=128: 1 count= 98063128: box_rearrange::compute_dest:: INFO io 2 start=128: 98064 count= 98063128: box_rearrange::compute_dest:: INFO io 3 start=128: 196127 count= 98063128: box_rearrange::compute_dest:: INFO io 4 start=128: 294190 count= 98063128: box_rearrange::compute_dest:: INFO io 1 lb= 0 128: ub= 98063128: box_rearrange::compute_dest:: INFO io 2 lb= 98063 128: ub= 196126128: box_rearrange::compute_dest:: INFO io 3 lb= 196126 128: ub= 294189128: box_rearrange::compute_dest:: INFO io 4 lb= 294189 128: ub= 392252128: box_rearrange::compute_dest:: INFO dof 2273 index=128: 1072693247 gcoord= 1072693247128: pio_support::pio_die:: myrank= -1 : ERROR: box_rearrange.F90.in:128: 878 : quitting128:MPT ERROR: Rank 128(g:128) is aborting with error code 1.128:Process ID: 43967, Host: r3i5n13, Program: /glade2/scratch2/luoyang/h_f02_prism4/bld/cesm.exe128:MPT Version: SGI MPT 2.15 09/03/16 04:15:54128:128:MPT: --------stack traceback-------128:MPT: Attaching to program: /proc/43967/exe, process 43967128:MPT: Try: zypper install -C "debuginfo(build-id)=3d290be00d48b823d3b71df2249e80d881bc473d"128:MPT: (no debugging symbols found)...done.128:MPT: Try: zypper install -C "debuginfo(build-id)=5409c48fdb15e90649c1407e444fbe31d6dc8ec1"128:MPT: (no debugging symbols found)...done.128:MPT: [Thread debugging using libthread_db enabled]128:MPT: Using host libthread_db library "/glade/u/apps/ch/os/lib64/libthread_db.so.1".128:MPT: Try: zypper install -C "debuginfo(build-id)=e97cfdb062d6f0c41073f2109a7605d0ae991c03"128:MPT: (no debugging symbols found)...done.128:MPT: Try: zypper install -C "debuginfo(build-id)=f43d7754940a14ffe3d9bd8fc9472ffbbfead544"128:MPT: (no debugging symbols found)...done.128:MPT: Try: zypper install -C "debuginfo(build-id)=0ea764119690f32c98faae9a63a73f35ed8b1099"128:MPT: (no debugging symbols found)...done.128:MPT: Try: zypper install -C "debuginfo(build-id)=15916519d9dbaea26ec88427460b4cedb9c0a6ab"128:MPT: (no debugging symbols found)...done.128:MPT: Try: zypper install -C "debuginfo(build-id)=79264652a62453da222372a430cd9351d4bbcbde"128:MPT: (no debugging symbols found)...done.128:MPT: Try: zypper install -C "debuginfo(build-id)=68682e9ac223d269cbecb94315fcec5e16b32bfb"128:MPT: (no debugging symbols found)...done.128:MPT: 0x00002aaaab7d641c in waitpid () from /glade/u/apps/ch/os/lib64/libpthread.so.0128:MPT: Missing separate debuginfos, use: zypper install glibc-debuginfo-2.19-35.1.x86_64128:MPT: (gdb) #0 0x00002aaaab7d641c in waitpid ()128:MPT: from /glade/u/apps/ch/os/lib64/libpthread.so.0128:MPT: #1 0x00002aaaabefe79c in mpi_sgi_system (command=) at sig.c:98128:MPT: #2 MPI_SGI_stacktraceback (header=) at sig.c:339128:MPT: #3 0x00002aaaabe58fae in print_traceback (ecode=1) at abort.c:227128:MPT: #4 0x00002aaaabe591e1 in PMPI_Abort (comm=, errorcode=1)128:MPT: at abort.c:66128:MPT: #5 0x00002aaaabe5923a in pmpi_abort__ ()128:MPT: from /glade/u/apps/opt/mpt/2.15-sgi715a158/lib/libmpi.so128:MPT: #6 0x000000000190ccff in pio_support_mp_piodie_ ()128:MPT: #7 0x0000000001a27651 in box_rearrange_mp_compute_dest_ ()128:MPT: #8 0x0000000001a24c5b in box_rearrange_mp_box_rearrange_create_ ()128:MPT: #9 0x000000000190454c in piolib_mod_mp_pio_initdecomp_dof_i8_ ()128:MPT: #10 0x0000000001904e79 in piolib_mod_mp_pio_initdecomp_dof_i4_ ()128:MPT: #11 0x0000000001520b4e in ncdio_pio_mp_ncd_getiodesc_ ()128:MPT: #12 0x000000000151d54d in ncdio_pio_mp_ncd_io_real_var1_ ()128:MPT: #13 0x0000000001559343 in subgridrestmod_mp_subgridrest_ ()128:MPT: #14 0x0000000001546b62 in restfilemod_mp_restfile_write_ ()128:MPT: #15 0x0000000001405cdf in clm_driver_mp_clm_drv_ ()128:MPT: #16 0x00000000013e0e0e in lnd_comp_mct_mp_lnd_run_mct_ ()128:MPT: #17 0x000000000040bcb8 in ccsm_comp_mod_mp_ccsm_run_ ()128:MPT: #18 0x000000000042a898 in MAIN__ ()128:MPT: #19 0x000000000040861e in main ()128:MPT: (gdb) A debugging session is active.128:MPT: 128:MPT:Inferior 1 [process 43967] will be detached.128:MPT: 128:MPT: Quit anyway? (y or n) [answered Y; input not from terminal]128:MPT: Detaching from program: /proc/43967/exe, process 43967128:128:MPT: -----stack traceback ends------1:MPT ERROR: MPI_COMM_WORLD rank 128 has terminated without calling MPI_Finalize()-1:aborting job I checked the output files of the three days, and they seem valid.My question is, is my method correct? Can I start the run from Nov 1st instead of the beginning of a year? What do the reported problems above mean and how can I solve them?Many thanks to any help. Yangcheng