Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

Issue in submitting the case

C

csp

CSP
Member
Dear experts,
I managed to create, set and build the case successfully in my machine using the following codes respectively:
Code: 
./create_newcase --case aqua_case --compset QSC6 --res f09_f09_mg17
Code: 
./case.setup
Code: 
./case.build
But while submitting the case using:
Code: 
./case.submit
I am getting the following output in my screen and nothing runs:

Code: 
Creating component namelists
  2022-09-20 12:18:38 atm 
   Calling /home/csp/my_cesm_sandbox/components/cam//cime_config/buildnml
CAM namelist copy: file1 /home/csp/my_cesm_sandbox/cime/scripts/aqua_case/Buildconf/camconf/atm_in file2 /home/csp/OUTPUT/CESM/scratch/aqua_case/run/atm_in 
  2022-09-20 12:18:38 lnd 
   Calling /home/csp/my_cesm_sandbox/cime/src/components/stub_comps_mct/slnd/cime_config/buildnml
  2022-09-20 12:18:38 ice 
   Calling /home/csp/my_cesm_sandbox/cime/src/components/stub_comps_mct/sice/cime_config/buildnml
  2022-09-20 12:18:38 ocn 
   Calling /home/csp/my_cesm_sandbox/cime/src/components/data_comps_mct/docn/cime_config/buildnml
  2022-09-20 12:18:38 rof 
   Calling /home/csp/my_cesm_sandbox/cime/src/components/stub_comps_mct/srof/cime_config/buildnml
  2022-09-20 12:18:38 glc 
   Calling /home/csp/my_cesm_sandbox/cime/src/components/stub_comps_mct/sglc/cime_config/buildnml
  2022-09-20 12:18:38 wav 
   Calling /home/csp/my_cesm_sandbox/cime/src/components/stub_comps_mct/swav/cime_config/buildnml
  2022-09-20 12:18:38 iac 
   Running /home/csp/my_cesm_sandbox/cime/src/components/stub_comps_mct/siac/cime_config/buildnml 
  2022-09-20 12:18:38 esp 
   Calling /home/csp/my_cesm_sandbox/cime/src/components/stub_comps_mct/sesp/cime_config/buildnml
  2022-09-20 12:18:38 cpl 
   Calling /home/csp/my_cesm_sandbox/cime/src/drivers/mct/cime_config/buildnml
Checking that inputdata is available as part of case submission
Loading input file list: 'Buildconf/cpl.input_data_list'
Loading input file list: 'Buildconf/docn.input_data_list'
Loading input file list: 'Buildconf/cam.input_data_list'
Check case OK
submit_jobs case.run
Submit job case.run
Starting job script case.run
Generating namelists for /home/csp/my_cesm_sandbox/cime/scripts/aqua_case
Creating component namelists
  2022-09-20 12:18:39 atm 
   Calling /home/csp/my_cesm_sandbox/components/cam//cime_config/buildnml
CAM namelist copy: file1 /home/csp/my_cesm_sandbox/cime/scripts/aqua_case/Buildconf/camconf/atm_in file2 /home/csp/OUTPUT/CESM/scratch/aqua_case/run/atm_in 
  2022-09-20 12:18:39 lnd 
   Calling /home/csp/my_cesm_sandbox/cime/src/components/stub_comps_mct/slnd/cime_config/buildnml
  2022-09-20 12:18:39 ice 
   Calling /home/csp/my_cesm_sandbox/cime/src/components/stub_comps_mct/sice/cime_config/buildnml
  2022-09-20 12:18:39 ocn 
   Calling /home/csp/my_cesm_sandbox/cime/src/components/data_comps_mct/docn/cime_config/buildnml
  2022-09-20 12:18:39 rof 
   Calling /home/csp/my_cesm_sandbox/cime/src/components/stub_comps_mct/srof/cime_config/buildnml
  2022-09-20 12:18:39 glc 
   Calling /home/csp/my_cesm_sandbox/cime/src/components/stub_comps_mct/sglc/cime_config/buildnml
  2022-09-20 12:18:39 wav 
   Calling /home/csp/my_cesm_sandbox/cime/src/components/stub_comps_mct/swav/cime_config/buildnml
  2022-09-20 12:18:39 iac 
   Running /home/csp/my_cesm_sandbox/cime/src/components/stub_comps_mct/siac/cime_config/buildnml 
  2022-09-20 12:18:39 esp 
   Calling /home/csp/my_cesm_sandbox/cime/src/components/stub_comps_mct/sesp/cime_config/buildnml
  2022-09-20 12:18:39 cpl 
   Calling /home/csp/my_cesm_sandbox/cime/src/drivers/mct/cime_config/buildnml
-------------------------------------------------------------------------
 - Prestage required restarts into /home/csp/OUTPUT/CESM/scratch/aqua_case/run
 - Case input data directory (DIN_LOC_ROOT) is /home/csp/OUTPUT/CESM/cesm-inputdata 
 - Checking for required input datasets in DIN_LOC_ROOT
-------------------------------------------------------------------------
run command is mpiexec_mpt  -np 16 --prepend-rank /home/csp/OUTPUT/CESM/scratch/aqua_case/bld/cesm.exe   >> cesm.log.$LID 2>&1  
Exception from case_run: ERROR: RUN FAIL: Command 'mpiexec_mpt  -np 16 --prepend-rank /home/csp/OUTPUT/CESM/scratch/aqua_case/bld/cesm.exe   >> cesm.log.$LID 2>&1 ' failed
See log file for details: /home/csp/OUTPUT/CESM/scratch/aqua_case/run/cesm.log.220920-121839
Submit job case.st_archive
Starting job script case.st_archive
st_archive starting
moving /home/csp/OUTPUT/CESM/scratch/aqua_case/run/cesm.log.220920-121839 to /home/csp/OUTPUT/CESM/scratch/archive/aqua_case/logs/cesm.log.220920-121839
Cannot find a aqua_case.cpl*.r.*.nc file in directory /home/csp/OUTPUT/CESM/scratch/aqua_case/run 
Archiving history files for cam (atm)
Archiving history files for docn (ocn)
Archiving history files for drv (cpl)
Archiving history files for dart (esp)
st_archive completed
Submitted job case.run with id None
Submitted job case.st_archive with id None
I am unable to figure out the issue, any comment/suggestion would be kind help, thanks in advance.
 
K

katec

CSEG and Liaisons
Staff member
This line:
moving /home/csp/OUTPUT/CESM/scratch/aqua_case/run/cesm.log.220920-121839 to /home/csp/OUTPUT/CESM/scratch/archive/aqua_case/logs/cesm.log.220920-121839
Click to expand...

Suggests that the model did run and produced a cesm.log file. Please take a look at that file for the error that caused the model to fail to complete.

Also this line:
Cannot find a aqua_case.cpl*.r.*.nc file in directory /home/csp/OUTPUT/CESM/scratch/aqua_case/run
Click to expand...

Suggests that you are missing the coupler restart file from your previous or reference case.
 
C

csp

CSP
Member
Thanks a lot @katec for your kind help and for your time. I looked at the log file and found the following:
Code: 
/bin/sh: 1: mpiexec_mpt: not found
but when I checked in my pc for mpiexec using
Code: 
mpiexec --version
I got the output:
Code: 
mpiexec (OpenRTE) 4.0.3
seems like mpiexec is installed, not sure about the error.
 
K

katec

CSEG and Liaisons
Staff member
csp said:
Thanks a lot @katec for your kind help and for your time. I looked at the log file and found the following:
Code: 
/bin/sh: 1: mpiexec_mpt: not found
but when I checked in my pc for mpiexec using
Code: 
mpiexec --version
I got the output:
Code: 
mpiexec (OpenRTE) 4.0.3
seems like mpiexec is installed, not sure about the error.
Click to expand...
So the error "mpiexec_mpt: not found" looks like the model is trying to run with the command "mpiexec_mpt" which is different than "mpiexec" because it has an extra "_mpt" at the end. If you want the model to run using "mpiexec" (no "_mpt") then you will need to update the "<executable>" flag in your config_machines.xml file.
 
You must log in or register to reply here.
Top