Hi everyone,
I have been perplexed for some time by the varied ways in which people configure PE layouts (on different machines) in their scripts, as evidenced by the four examples I have provided below.
How can I determine the most appropriate PE settings for my simulation? Should I take into account the specific hardware of the machine (e.g., Perlmutter, Derecho, etc.)? Additionally, how crucial are factors like walltime and the time span of my case, especially if there's a risk of the run not completing within the allocated walltime? Should I also consider the status of components, whether they are active, stub, or inactive?
Moreover, why do some individuals specify settings like ROOTPE_X, PIO_STRIDE, and MAX_MPITASKS_PER_NODE, while others omit these? What criteria should guide the inclusion or exclusion of these settings?
Thank you very much.
Example 1
#./xmlchange NTASKS_ATM=900,NTHRDS_ATM=2,ROOTPE_ATM=0
#./xmlchange NTASKS_CPL=900,NTHRDS_CPL=1,ROOTPE_CPL=0
#./xmlchange NTASKS_LND=504,NTHRDS_LND=2,ROOTPE_LND=0
#./xmlchange NTASKS_ROF=504,NTHRDS_ROF=2,ROOTPE_ROF=0
#./xmlchange NTASKS_ICE=396,NTHRDS_ICE=2,ROOTPE_ICE=504
#./xmlchange NTASKS_OCN=180,NTHRDS_OCN=2,ROOTPE_OCN=900
#./xmlchange NTASKS_WAV=1,NTHRDS_WAV=1,ROOTPE_WAV=0
#./xmlchange NTASKS_GLC=1,NTHRDS_GLC=1,ROOTPE_GLC=0
Example 2
#./xmlchange NTASKS=1440,NTHRDS=1
Example 3
./xmlchange PIO_STRIDE=1
./xmlchange NTASKS=128
Example 4
np=2816
stridc=8
ntatm=1
nt=1
ntcpl=1
./xmlchange --file env_mach_pes.xml MAX_MPITASKS_PER_NODE=128
./xmlchange --file env_mach_pes.xml NTASKS_ATM="$np"
./xmlchange --file env_mach_pes.xml NTASKS_LND="$np"
./xmlchange --file env_mach_pes.xml NTASKS_ICE="$np"
./xmlchange --file env_mach_pes.xml NTASKS_OCN="$np"
./xmlchange --file env_mach_pes.xml NTASKS_CPL="$np"
./xmlchange --file env_mach_pes.xml NTASKS_ROF="$np"
./xmlchange --file env_mach_pes.xml NTASKS_GLC="32"
./xmlchange --file env_mach_pes.xml NTASKS_WAV="32"
I have been perplexed for some time by the varied ways in which people configure PE layouts (on different machines) in their scripts, as evidenced by the four examples I have provided below.
How can I determine the most appropriate PE settings for my simulation? Should I take into account the specific hardware of the machine (e.g., Perlmutter, Derecho, etc.)? Additionally, how crucial are factors like walltime and the time span of my case, especially if there's a risk of the run not completing within the allocated walltime? Should I also consider the status of components, whether they are active, stub, or inactive?
Moreover, why do some individuals specify settings like ROOTPE_X, PIO_STRIDE, and MAX_MPITASKS_PER_NODE, while others omit these? What criteria should guide the inclusion or exclusion of these settings?
Thank you very much.
Example 1
#./xmlchange NTASKS_ATM=900,NTHRDS_ATM=2,ROOTPE_ATM=0
#./xmlchange NTASKS_CPL=900,NTHRDS_CPL=1,ROOTPE_CPL=0
#./xmlchange NTASKS_LND=504,NTHRDS_LND=2,ROOTPE_LND=0
#./xmlchange NTASKS_ROF=504,NTHRDS_ROF=2,ROOTPE_ROF=0
#./xmlchange NTASKS_ICE=396,NTHRDS_ICE=2,ROOTPE_ICE=504
#./xmlchange NTASKS_OCN=180,NTHRDS_OCN=2,ROOTPE_OCN=900
#./xmlchange NTASKS_WAV=1,NTHRDS_WAV=1,ROOTPE_WAV=0
#./xmlchange NTASKS_GLC=1,NTHRDS_GLC=1,ROOTPE_GLC=0
Example 2
#./xmlchange NTASKS=1440,NTHRDS=1
Example 3
./xmlchange PIO_STRIDE=1
./xmlchange NTASKS=128
Example 4
np=2816
stridc=8
ntatm=1
nt=1
ntcpl=1
./xmlchange --file env_mach_pes.xml MAX_MPITASKS_PER_NODE=128
./xmlchange --file env_mach_pes.xml NTASKS_ATM="$np"
./xmlchange --file env_mach_pes.xml NTASKS_LND="$np"
./xmlchange --file env_mach_pes.xml NTASKS_ICE="$np"
./xmlchange --file env_mach_pes.xml NTASKS_OCN="$np"
./xmlchange --file env_mach_pes.xml NTASKS_CPL="$np"
./xmlchange --file env_mach_pes.xml NTASKS_ROF="$np"
./xmlchange --file env_mach_pes.xml NTASKS_GLC="32"
./xmlchange --file env_mach_pes.xml NTASKS_WAV="32"