Optimal batch setting for Cam on Bluefire?

[tjive@udel_edu]

Hi,
I am very very new to the whole batch submission process. I have sucessfully run a 1 day test of CAM4 couples with CLM and a data ocean. I used the settings that were included in the run-ibm.csh script....but I am wondering if I can get better performance. I read in the forum that some people were getting 10 modeled years in 1 real-time day.


My current batch setting are:

#BSUB -a poe # use LSF openmp elim
#BSUB -x # exclusive use of node (not_shared)
#BSUB -n 16 # bluefire setting
#BSUB -R "span[ptile=16]" # bluefire setting
#BSUB -o out.%J # output filename
#BSUB -e out.%J # error filename
#BSUB -q regular # queue
#BSUB -W 6:00 # wall clock limit (default, max for regular batch job)
#BSUB -J 1daytest #job name
#BSUB -P ###### #account number


##extract number of tasks from batch environment
set ntasks = `echo $LSB_HOSTS | wc -w`

## should be set equal to (CPUs-per-node / tasks_per_node)
setenv OMP_NUM_THREADS 4

## suggestions from Jim Edwards 07/08
setenv XLSMPOPTS "stack=256000000"
setenv OMP_DYNAMIC false
setenv AIXTHREAD_SCOPE S
setenv MALLOCMULTIHEAP true
setenv MP_USE_BULK_XFER yes
setenv MP_LABELIO yes

## Do our best to get sufficient stack memory
limit stacksize unlimited

 

[tjive@udel_edu]

i should mention that I am running a 0.9x1.25 resolution and need to run 40 years AND have 2000 GAUs

thanks!
-Tianna

 

[sfeng2@unl_edu]

Any solutions to this? I am having a similar issue. I tried different combinations of the "BSUB -n", ptile and the OMP_NUM_THREADS. It seems the the standalone CAM5 can only be run for for n=16 (or 32), ptile=16 and OMP_NUM_THREADS=4.

Could anybody give some suggestions, or provide some examples that work?