heyaqian1987@gmail_com
Member
Hi everyone,I am trying to run a CESM case with the compset IHistClm50BgcCrop. I run it from 1953 to 2012 as start up run and save the restart file every month. Here is my run script:cd /glade/p/cesm/releases/cesm2_0/cime/scripts./create_newcase --case ~/cases/IHISTluhspin2 --res f09_g17 --compset IHistClm50BgcCrop --project UDRT0013 --run-unsupportedcd ~/cases/IHISTluhspin2/./xmlchange INFO_DBUG=3,STOP_N=5,STOP_OPTION=nyears,RESUBMIT=5 ./xmlchange RUN_STARTDATE=1953-01-01./xmlchange DOUT_S_SAVE_INTERIM_RESTART_FILES=TRUE,REST_OPTION=nmonths,REST_N=1./case.setupedit/check user_nl_clmfsurdat = '/glade/p/cesmdata/cseg/inputdata/lnd/clm2/surfdata_map/surfdata_0.9x1.25_78pfts_CMIP6_simyr1850_c170824.nc'do_harvest = .true.do_transient_crops = .true.do_transient_pfts = .true.flanduse_timeseries = '/glade/p/cesmdata/cseg/inputdata/lnd/clm2/surfdata_map/landuse.timeseries_0.9x1.25_hist_78pfts_CMIP6_simyr1850-2015_c170824.nc'./preview_namelistsqcmd -- ./case.build ./case.submit The experiment is running successfully on first five years. Then I got error in the CaseStatus as follow:ERROR: RUN FAIL: Command 'mpiexec_mpt -np 1836 omplace -tm open64 /glade/scratch/yahe/IHISTluhspin2/bld/cesm.exe >> cesm.log.$LID 2>&1 ' failedSee log file for details: /glade/scratch/yahe/IHISTluhspin2/run/cesm.log.7438754.chadmin1.ib0.cheyenne.ucar.edu.190805-215625 I went to cesm log file, it shows: Invalid PIO rearranger comm max pend req (comp2io), 0 Resetting PIO rearranger comm max pend req (comp2io) to 64 PIO rearranger options:.................. memory_write: model date = 19601231 75600 memory = 128750.71 MB (highwater) 122.74 MB (usage) (pe= 0 comps= ATM ESP) size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 size=33644476 rss=48390 share=9187 text=6997 datastack=0 MPT: shepherd terminated: r7i5n15.ib0.cheyenne.ucar.edu - job aborting Can anyone help to fix this problem? Thanks in advance