Main menu

Navigation

the internal domain decomposition error for the land model

10 posts / 0 new
Last post
xgao304@...
the internal domain decomposition error for the land model

I am running the cesm2 with intel compiler and mpi and netcdf on our own machine. I am able to run the model with 4 nodes (128 processors), but with 8 nodes I will get the error related to the internal domain decomposition for the land model. Here is the information for my case:


./create_newcase --case ../../../cases/test2000 --compset I2000Clm50SpGs --res hcru_hcru --machine svante --compiler intel --run-unsupported

I attach the cesm log file for error message.

Any help is appreciated.

Xiang

jedwards

NO attachement found.  What did you do to switch from 4 to 8 nodes?

xgao304@...

create a totally new case when I switch from 4 to 8. See the attachment.

xgao304@...

try again.

xgao304@...

I am wondering if you have any idea about this problem based on the attached log file. Any update is really appreciated.

jedwards

I have been unable to reproduce your problem locally.   What version of netcdf are you using?   

Try changing the PIO_STRIDE value:

./xmlchange PIO_STRIDE=32 (16, 8, ...) 

xgao304@...

Here is the information:

intel/2017.0.1

netcdf/4

openmpi/2.0.2

I will give it a try with PIO_STRIDE and let you know.

Thanks.

 

 

xgao304@...

I just found out that my default PIO_STRIDE is set as "32" as follows:

    PIO_STRIDE: ['CPL:32', 'ATM:32', 'LND:32', 'ICE:32', 'OCN:32', 'ROF:32', 'GLC:32', 'WAV:32', 'ESP:32']

Thanks,

Xiang

jedwards

Experiment with changing the pio stride to different values, since you were successful with 4 nodes try PIO_STRIDE=64

The netcdf version has three digits, you can get it with nc-config --version

xgao304@...

netcdf 4.4.1

I will try with different PIO_STRIDE values.

 

Log in or register to post comments

Who's new

  • alessandro.delo...
  • zweina@...
  • yuan.liang@...
  • lian.xue@...
  • 353482168@...