Default PIO settings for large runs

[worleyph@gmail_com]

Perhaps this is more of an educational issue, but ... for large processor count runs (for high resolution cases) the default PIO settings in env_run.xml, e.g.

id="PIO_STRIDE" value="-1"

id="PIO_ROOT" value="1"

id="PIO_NUMTASKS" value="-1"


can end up requesting more PIO tasks than are currently supported in PIO. The error messages are "voluminous", but Jayesh Krishna (at Argonne) plowed through these and diagnosed the issue as

"After looking into the code, the part of the code that fails is where PIO tries to map the global coordinate into an IO process box. The code that fails has an implicit assumption that the number of IO processes that actually perform IO is

 

[jedwards]

The number 1000 was choosen pretty much at random and could be increased - however it does not have a one-to-one correspondence to the number of IO tasks and we have many successful cases with > 1000 IO tasks.   If you do hit this error changing the PIO_STRIDE or PIO_NUMTASKS may or may not change the characteristics in a way that solves the problem.   The good news is that this algorythm no longer exists in the development version of PIO.

 

[jedwards]

The number 1000 was choosen pretty much at random and could be increased - however it does not have a one-to-one correspondence to the number of IO tasks and we have many successful cases with > 1000 IO tasks.   If you do hit this error changing the PIO_STRIDE or PIO_NUMTASKS may or may not change the characteristics in a way that solves the problem.   The good news is that this algorythm no longer exists in the development version of PIO.

 

[jedwards]

The number 1000 was choosen pretty much at random and could be increased - however it does not have a one-to-one correspondence to the number of IO tasks and we have many successful cases with > 1000 IO tasks.   If you do hit this error changing the PIO_STRIDE or PIO_NUMTASKS may or may not change the characteristics in a way that solves the problem.   The good news is that this algorythm no longer exists in the development version of PIO.