How many cores make CAM5-SE faster than CAM5-FV

[oyvind_seland@met_no]

 I have tested CAM5.3 with our local extensions (Slightly higher complexity than CAM5.3-MAM4 ) on a NeXtScale nx360M5 WCTThe extensions slow down the model with around 15 % At 7680 cores the turnover of CAM-FV quarter degree resolution is 2.36 years / day.  The equivalent for CAM5.3 -FV would be 2.7 years/day.However CAM5.3 SE yields less than 2 years for the same set-up. Now as far as I can see 7680 is the maximum number of cores for CAM5.3 FV so SE should be faster for a higher number of cores. I could not test SE with higher number of cores since I had only a very limited amount of CPU hours available for my tests.
Is SE only faster due to a higher possible number of cores or are there any tricks I should know of to make the SE version faster?   

 

[shian-jiann_lin@noaa_gov]

My previous attemp of posting failed. here is my take.
I am the original developer of FV core used in the CAM-FV, which is based on lat-lon grid. Its scalability is limitted by its use of polar filter, which is the main reason we at GFDL developed the next generation FV core on the cubed-sphere, called FV3 (FV cubed).
Due to FV's inherent advantage in CFL condition (larger time step) and more efficient tracer transport algorithm (with 2 decades of continued optimization!), it is extremely difficult for the SE core to outperform the FV lat-lon when there are over 20 tracers (as in CAM5). To beat the Fv lat-lon, you wil need to use FV3. There is a version of FV3 within CAM. It should outperform the lat-lon FV at pretty much any resolution and at any core core that is higher than 500 cores. At 7K or 8K cores, the FV3 should be 5 to 10 times faster than the old FV lat-lon (and the SE core). 

 

[oyvind_seland@met_no]

It has been a long time since I looked at the tread, but at last thank you for the information. Do you have examples on how to set up namelist / Makefile etc in <br />order to test FV3 for updated version of CAM / CESM?

Best regards

Øyvind Seland

Norwegian Meteorological Institute