how to set -pes when executing the script cice.setup

[CST1996]

Dear support of CICE consortium,
When I run CICE-master and try creating a case, I note that by using -pes of cice.setup, I can define the tasks and threads.
So I wonder whether tasks and threads will affect the speed of calculation, if the answer is yes, how to set it properly?
I note that the default value is 4x1, and I will get an error message if setting the value unproperly.
And I also note a related parameter, 'nprocs', the number of processors. And what is the relationship between them?
It will be helpful if computing time can be saved by setting the -pes parameter properly.
Best Wishes.
Look forward to your reply.
Sincerely,
Cao

 

[dbailey]

The syntax of mxn indicates MPI processes versus threads. If you don't have OpenMP threading on your machine, then you should always have n=1 here. In the namelist, nprocs is the number of MPI tasks or m here. When we use threading, then max_blocks is sometimes equal to the number of threads. However, we can always do multiple blocks per processor even without threads. When you set the -pes option in cice.setup, this will set everything correctly in cice.settings and ice_in.

 

[tcraig]

The code generally scales (runs faster on more processors) up to a limit depending on the configuration. Choosing the number of pes depends very much on the amount of resources available (the hardware and resource sharing), the configuration, and the machine performance.

 

[CST1996]

The syntax of mxn indicates MPI processes versus threads. If you don't have OpenMP threading on your machine, then you should always have n=1 here. In the namelist, nprocs is the number of MPI tasks or m here. When we use threading, then max_blocks is sometimes equal to the number of threads. However, we can always do multiple blocks per processor even without threads. When you set the -pes option in cice.setup, this will set everything correctly in cice.settings and ice_in.

Thanks for your guide. We have OpenMPI on the machine. And I have got a general idea of how to set up -PES with your help.
Best wishes.
Sincerely,
Cao

 

[CST1996]

The code generally scales (runs faster on more processors) up to a limit depending on the configuration. Choosing the number of pes depends very much on the amount of resources available (the hardware and resource sharing), the configuration, and the machine performance.

Thanks. Maybe I can do some experiments to decide to use how many cores and threads to compute.
Best Wishes.
Sincerely,
Cao

 

[CST1996]

The syntax of mxn indicates MPI processes versus threads. If you don't have OpenMP threading on your machine, then you should always have n=1 here. In the namelist, nprocs is the number of MPI tasks or m here. When we use threading, then max_blocks is sometimes equal to the number of threads. However, we can always do multiple blocks per processor even without threads. When you set the -pes option in cice.setup, this will set everything correctly in cice.settings and ice_in.

I have another question. For the -pes option description of cice.setup, what does the syntax in the bracket mean? That's [x blocksize_x x blocksize_y [x maxblocks]]. If I set -pes as mxn, is m equal to blocksize_x, and n equal to blocksize_y?

 

[dbailey]

Not quite. The values for blocksize_x and blocksize_y are the x and y sizes of the computational blocks as follows:

blocksize_x = nx_gobal / m
blocksize_y = ny_global / n

 

[tcraig]

Just getting back to this thread. The general -pes option is

--pes, -p : tasks x threads [x blocksize_x x blocksize_y [x maxblocks]] (default is 4x1)

tasks and threads define the hardware resources we want to use and how to allocate them between MPI and OpenMP. blocksize_x, blocksize_y, and maxblocks are additional option settings that can be defined. They define the size of the 2-dimensional block in CICE and the maximum number of blocks expected per MPI task. In general, you can have more than one 2-dimensional block per MPI task. These options help define the CICE decomposition with the namelist variables distribution_type and others. There is additional information about how these work in the user guide,

CICE Documentation — CICE documentation

In particular,

3.1. Implementation — CICE documentation

and

3.2. Running CICE — CICE documentation

If there are additional questions, please ask.