ENDRUN:SPMDINIT_DYN: less than 3 latitudes per subdomain

[emonier@mit_edu]

Hi,

I am a new CAM user. I have compiled CAM3.1 with pgf90 (pgi/9.0-4), netcdf/3.6.2 and openmpi/1.3.3a to use on Linux x86_64 cluster, with spmd enabled. I have successfully ran T21,T31,T42,fv2x2.5,fv4x5 on 1 node with 8 cores. However, when running fv10x15, it fails and returns the following error message:

ENDRUN:SPMDINIT_DYN: less than 3 latitudes per subdomain

Similarly, when running fv4x5 on 2 nodes with 16 cores, I get the same error message.

My understanding is that the setup sends latitudinal bands to each core that have less than 3 latitude grid points, which fails when it comes to calculating finite volumes. I have seen in literature that people have run fv1.9x2.5 with up several 100 cores. What do I need to modify to do this? Is it possible to change the tiling so it tiles not only in latitude but in longitude as well?
Any help is welcome. Thanks

 

[hannay]

CAM requires a minimum of 3 latitudes per MPI task. To run the model with more nodes, you need to use the namelist variable npr_yz that controls the decomposition.

The npr_yz namelist variable specifies the y,z and x,y domain decomposition.
If npr_yz is not set in the namelist then you are the default value which is:
npr_yz = n,1,1,n
where n is the number of MPI processes.

 

[emonier@mit_edu]

Thank you for the prompt answer. I was able to make it run that way.

Is it possible to also decompose (in the vertical) the spectral runs?

I am able to run spectral eulerian with a maximum number of cores equal to the number of latitudinal points of the model domain. I understand that due to the spectral calculations, it wouldn't be possible to decompose in longitude, but how about in the vertical?

I could not find much help on the parallel run configurations in the CAM user's guide. So any help or link toward a document that could shed some light on the process would be much appreciated.