Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

Run fail cesm.exe

Dear all,I am running CESM2.0 on Canada super computer Niagara. I created a case named fxsd with compset FXSD, run unsupported. When I run the command case.submit,there is an error as:ERROR: RUN FAIL: Command 'mpirun -np 80 /scratch/y/yochen/xiaoshi/cesm/output/fxsd/bld/cesm.exe  In the CESM.log file,there are many records as:==== backtrace ==== 0 0x0000000000a4bacd edyn_mpi_mp_mp_gatherlons_f3d_()  /gpfs/fs0/scratch/y/yochen/xiaoshi/cesm/my_cesm_sandbox/components/cam/src/ionosphere/waccmx/edyn_mpi.F90:1716 1 0x0000000000a5b6cf edyn_mpi_mp_switch_model_format_()  /gpfs/fs0/scratch/y/yochen/xiaoshi/cesm/my_cesm_sandbox/components/cam/src/ionosphere/waccmx/edyn_mpi.F90:2016 2 0x0000000000a1bcb7 dpie_coupling_mp_d_pie_coupling_()  /gpfs/fs0/scratch/y/yochen/xiaoshi/cesm/my_cesm_sandbox/components/cam/src/ionosphere/waccmx/dpie_coupling.F90:554 3 0x00000000005c147e ionosphere_interface_mp_ionosphere_run2_()  /gpfs/fs0/scratch/y/yochen/xiaoshi/cesm/my_cesm_sandbox/components/cam/src/ionosphere/waccmx/ionosphere_interface.F90:768 4 0x00000000004ecf3e cam_comp_mp_cam_run2_()  /gpfs/fs0/scratch/y/yochen/xiaoshi/cesm/my_cesm_sandbox/components/cam/src/control/cam_comp.F90:310 5 0x00000000004deae9 atm_comp_mct_mp_atm_run_mct_()  /gpfs/fs0/scratch/y/yochen/xiaoshi/cesm/my_cesm_sandbox/components/cam/src/cpl/atm_comp_mct.F90:433 6 0x000000000043245a component_mod_mp_component_run_()  /gpfs/fs0/scratch/y/yochen/xiaoshi/cesm/my_cesm_sandbox/cime/src/drivers/mct/main/component_mod.F90:728 7 0x0000000000419460 cime_comp_mod_mp_cime_run_()  /gpfs/fs0/scratch/y/yochen/xiaoshi/cesm/my_cesm_sandbox/cime/src/drivers/mct/main/cime_comp_mod.F90:3383 8 0x000000000043217f MAIN__()  /gpfs/fs0/scratch/y/yochen/xiaoshi/cesm/my_cesm_sandbox/cime/src/drivers/mct/main/cime_driver.F90:103 9 0x0000000000415b2e main()  ???:010 0x0000000000021c05 __libc_start_main()  ???:011 0x0000000000415a29 _start()  ???:0===================Primary job  terminated normally, but 1 process returneda non-zero exit code. Per user-direction, the job has been aborted.-------------------------------------------------------mpirun noticed that process rank 64 with PID 173000 on node nia1298 exited on signal 11 (Segmentation fault).--------------------------------------------------------------------------
There are many core.* and PET*.ESMF_LogFile files in Output/CASE/run directory. The content of PET*.ESMF_LogFile are similar, as: "20181130 193746.440 INFO             PET50 Running with ESMF Version 7.1.0r"
I attached the log  files. Could you please help me to check it?  Many thanks.  
 

jedwards

CSEG and Liaisons
Staff member
No log files attached.   Have you completed the porting process for your system including running the UF_CAM_ECT tests described here.   
 
I am sorry for missing log files. I attached them now.I just run UF_CAM_ECT tests using the command as:./../tools/statistical_ensemble_test/ensemble.sh --case ./f2000.000 --mach cheyenne --compset F2000climo --res f19_f19_mg17 --compiler intel --ensemble 350 --ufThere is an error as:ERROR: module command /glade/u/apps/ch/opt/lmod/7.5.3/lmod/lmod/libexec/lmod python purge  failed with message:/bin/sh: /glade/u/apps/ch/opt/lmod/7.5.3/lmod/lmod/libexec/lmod: No such file or directoryError running case_setup, probably a bad NP value.   I don't have any directory as "/glade/u/……".  "ensemble.sh" seems read some config files which are not suitable for my mashine. 
 

jedwards

CSEG and Liaisons
Staff member
cheyenne is provided as an example - you should replace that with the name of your own machine.
 
Top