building cam on blue gene architecture

[tannahill1@llnl_gov]

I have been running CAM on a large linux cluster for quite some time. Now I need to start running on blue gene type machines, ubgl here at LLNL and intrepid at ANL. My understanding is that the blue gene environment is quite a bit different than the linux cluster. I have found some ubgl and intrepid files in scripts/ccsm_utils/Machines, but I won't initially be running CCSM, just CAM standalone, so I am not sure how valuable these are to me?

I am looking for any pointers on how to get CAM up and running quickly on the Blue Gene's. I know it has been run on ubgl already and I know that at least CCSM proper has been run on intrepid. If you can point me to anyone who has done this, let me know of any scripts or other info that would be of benefit to me, etc., it would be greatly appreciated.

Thanks,
John

 

[eaton]

The main thing to do is to supply the configure argument "-target_os bgl" on BG/L platforms and "-target_os bgp" on BG/P platforms.

I don't have access to a BG/L platform to test on, so if you find Makefile modifications that you need then please post them back and I'll get them into the next CAM release.

I'm working on getting the code to run on intrepid now. It doesn't work out of the box for BG/P. For one thing the PIO code needs to be updated. And there are some missing include paths. I'll get this fixed in the next CAM release.

 

[tannahill1@llnl_gov]

Brian,

Yes, found the -target_os stuff in the configure script and that did the trick! With a little help from others, the release version of CAM is now running on ubgl.

Please keep me posted on your intrepid work. I have an account on it and should be getting my crypto key shortly. Our current plan is to stick with the release version of the code. I may have to make any BG/P fixes to our local version, in less they are going to make it into the release version as well? We need to make intrepid scaling runs pretty soon for our INCITE proposal that is due by 6/30.

Regards,
John

 

[eaton]

I now have a successful build on intrepid. I needed to make two changes:

1) The pio library needs to be updated to at least pio1_1_0
http://parallelio.googlecode.com/svn/trunk_tags/pio1_1_0/pio

2) The Makefile.in template needs to have an include path added in the BGP section:

ifeq ($(UNAMES),BGP)

CPPDEF += -DAIX -DBGP
ifeq ($(SMP),TRUE)
# THREADED_OMP is used by the timing library
CPPDEF += -DTHREADED_OMP
endif

cpp_path += -I/bgsys/drivers/ppcfloor/arch/include

The "cpp_path += ..." line is the new one.

 

[eaton]

Turns out there is a bugfix in the pio1_1_1 tag which is needed if you aren't building with the PNetCDF library.