CESM on NCSA Ember, segfault

[solsen]

Hi,

Has anyone successfully run CESM on Ember at NCSA?
Ember is an SGI Altix UV. CESM seems to compile and run OK but whenever CAM tries to write a restart file it causes a segfault. Other modules are able to write restart files and CAM can write history files. I'm using intel 11.1.073 and sgi mpt-2.01 with compset B and res f45_g37.

Suggestions welcome,

Seth

 

[eaton]

This generally means that you've hit a memory limit. A bit surprising at such a low resolution. Try running with more tasks assigned to the atm. This means customizing the PE layout, as per instructions in the users guide.

 

[solsen]

It seems that on the SGI Altix UV the mpi library (sgi-mpt) defaults to setting up communication through memory maps as an optimization for the NUMA architecture. This doesn't interact well with CESM. After turning this off with environment variables, e.g., MPI_DEFAULT_SINGLE_COPY_OFF, MPI_MEMMAP_OFF, etc. it seems to be working.