Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

Error while case.submit: mpirun conmmand not found

C

changmao

Yufei Wang
New Member
Dear CESM forum,

I got an error: mpirun command could not be found when ./case.submit (res f19_g17 --compset B1850). The existing related posts seemed to be little help for me.

Details are as follows:
------------------------------------------------------------------------
- Prestage required restarts into /data1/elzd_2023_00031/cesm/scratch/b.day2.1/run
- Case input data directory (DIN_LOC_ROOT) is /data1/elzd_2023_00031/cesm/inputdata
- Checking for required input datasets in DIN_LOC_ROOT
-------------------------------------------------------------------------
run command is mpirun -n 384 /data1/elzd_2023_00031/cesm/scratch/b.day2.1/bld/cesm.exe >> cesm.log.$LID 2>&1
Exception from case_run: ERROR: RUN FAIL: Command 'mpirun -n 384 /data1/elzd_2023_00031/cesm/scratch/b.day2.1/bld/cesm.exe >> cesm.log.$LID 2>&1 ' failed
See log file for details: /data1/elzd_2023_00031/cesm/scratch/b.day2.1/run/cesm.log.231128-115340
Submit job case.st_archive
Starting job script case.st_archive
st_archive starting
moving /data1/elzd_2023_00031/cesm/scratch/b.day2.1/run/cesm.log.231128-115340 to /data1/elzd_2023_00031/cesm/scratch/archive/b.day2.1/logs/cesm.log.231128-115340
Cannot find a b.day2.1.cpl*.r.*.nc file in directory /data1/elzd_2023_00031/cesm/scratch/b.day2.1/run
Archiving history files for cam (atm)
Archiving history files for clm (lnd)
Archiving history files for cice (ice)
Archiving history files for pop (ocn)
Archiving history files for mosart (rof)
Archiving history files for cism (glc)
Archiving history files for ww3 (wav)
Archiving history files for drv (cpl)
Archiving history files for dart (esp)
st_archive completed
Submitted job case.run with id None
Submitted job case.st_archive with id None
Click to expand...

[elzd_2023_00031@login01 b.day2.1]$ cat /data1/elzd_2023_00031/cesm/scratch/archive/b.day2.1/logs/cesm.log.23112
/bin/sh: mpirun: command not found
Click to expand...

It was also mentioned that the. cpl *. r. *. nc file was missing, but I cannot understand what this specifically refered to. The checksum was already completed when building the case.

Run fhello_ World_ Mpi F90, the result showed as follows:
$ vi /data1/elzd_2023_00031/fhello_world_mpi.F90
$ mpif90 /data1/elzd_2023_00031/fhello_world_mpi.F90 -o hello_world
$ mpirun -n 2 ./hello_world

Process 1 says "Hello, world!" login01

HELLO_MPI - Master process:
FORTRAN90/MPI version

An MPI test program.

The number of processes is 2


Process 0 says "Hello, world!" login01
Click to expand...

Another question: The compiler is intelmpi. In compilers.xml,<mpicc><mpicxx><mpifc>can only use the path of MPICH. If replaced by mpi/intelmpi/bin, the error will be "cannot open source file <mpi.h>". I am quite confused about this.

See the attach files for configuration files.

Looking forward to any help!
 
J

jedwards

CSEG and Liaisons
Staff member
The path to mpirun or mpiexec is set in config_machines.xml not in config_compilers.xml perhaps you should look there for your mistake.
 
C

changmao

Yufei Wang
New Member
jedwards said:
The path to mpirun or mpiexec is set in config_machines.xml not in config_compilers.xml perhaps you should look there for your mistake.
Click to expand...
Hi, jedwards. Thank you very much for your reply. The problem of mpirun mentioned before has been solved. However, the problem of .cn file still exists: Cannot find a b.day2.1.cpl*.r.*.nc file in directory /data1/elzd_2023_00031/cesm/scratch/b.day2.1/run。The inputdata has been checked before the case build. I really don't know what file is missing.
I have also run a simple case and show the results here (./create_newcase --case runtestX --res f19_g16 --compset X):
-------------------------------------------------------------------------
- Prestage required restarts into /data1/elzd_2023_00031/cesm/scratch/runtestX/run
- Case input data directory (DIN_LOC_ROOT) is /data1/elzd_2023_00031/cesm/inputdata
- Checking for required input datasets in DIN_LOC_ROOT
-------------------------------------------------------------------------
run command is mpirun -n 64 /data1/elzd_2023_00031/cesm/scratch/runtestX/bld/cesm.exe >> cesm.log.$LID 2>&1
Exception from case_run: ERROR: RUN FAIL: Command 'mpirun -n 64 /data1/elzd_2023_00031/cesm/scratch/runtestX/bld/cesm.exe >> cesm.log.$LID 2>&1 ' failed
See log file for details: /data1/elzd_2023_00031/cesm/scratch/runtestX/run/cesm.log.231205-163015
Submit job case.st_archive
Starting job script case.st_archive
st_archive starting
moving /data1/elzd_2023_00031/cesm/scratch/runtestX/run/cesm.log.231205-163015 to /data1/elzd_2023_00031/cesm/scratch/archive/runtestX/logs/cesm.log.231205-163015
moving /data1/elzd_2023_00031/cesm/scratch/runtestX/run/lnd.log.231205-163015 to /data1/elzd_2023_00031/cesm/scratch/archive/runtestX/logs/lnd.log.231205-163015
moving /data1/elzd_2023_00031/cesm/scratch/runtestX/run/ocn.log.231205-163015 to /data1/elzd_2023_00031/cesm/scratch/archive/runtestX/logs/ocn.log.231205-163015
moving /data1/elzd_2023_00031/cesm/scratch/runtestX/run/ice.log.231205-163015 to /data1/elzd_2023_00031/cesm/scratch/archive/runtestX/logs/ice.log.231205-163015
moving /data1/elzd_2023_00031/cesm/scratch/runtestX/run/wav.log.231205-163015 to /data1/elzd_2023_00031/cesm/scratch/archive/runtestX/logs/wav.log.231205-163015
moving /data1/elzd_2023_00031/cesm/scratch/runtestX/run/cpl.log.231205-163015 to /data1/elzd_2023_00031/cesm/scratch/archive/runtestX/logs/cpl.log.231205-163015
moving /data1/elzd_2023_00031/cesm/scratch/runtestX/run/atm.log.231205-163015 to /data1/elzd_2023_00031/cesm/scratch/archive/runtestX/logs/atm.log.231205-163015
moving /data1/elzd_2023_00031/cesm/scratch/runtestX/run/rof.log.231205-163015 to /data1/elzd_2023_00031/cesm/scratch/archive/runtestX/logs/rof.log.231205-163015
moving /data1/elzd_2023_00031/cesm/scratch/runtestX/run/glc.log.231205-163015 to /data1/elzd_2023_00031/cesm/scratch/archive/runtestX/logs/glc.log.231205-163015
Cannot find a runtestX.cpl*.r.*.nc file in directory /data1/elzd_2023_00031/cesm/scratch/runtestX/run
Archiving history files for drv (cpl)
Archiving history files for dart (esp)
st_archive completed
Submitted job case.run with id None
Submitted job case.st_archive with id None
Click to expand...
 

Attachments

  • csm_share.bldlog.230727-104515.txt
    83.1 KB · Views: 2
J

jedwards

CSEG and Liaisons
Staff member
What version of cesm are you trying to build? cesm2.3 requires ESMF to be built and installed first.
 
J

jedwards

CSEG and Liaisons
Staff member
You need to define ESMFMKFILE in the environment - if you are using a module it should do that for you.
If not you need to set it in the environment variables section of config_machines.xml
 
You must log in or register to reply here.
Top