Running WACCM on Macs

[bardeenc]

Several people have asked about whether you can run WACCM on a mac. By default, you can not; however, while I was a graduate student at the University of Colorado, we made changes to support running WACCM on a mac. A description of how to build the support libraries (NETCDF and MPICH2) and how to build and run CAM and WACCM is available at the Toon Group wiki:

http://atoc.colorado.edu/aerosol/index.php/NCAR_models

However, the scripts and source code that they use are not available to you. Therefore, I have modified the NCAR scripts and put together a tar file with the important file changes. You can download them from my anonymous ftp site:

ftp://ftp.ucar.edu/pub/acd/bardeenc/downloads/wa319_14_mac.tar

The main changes are additions to the WACCM makefile to support building on the mac using ifort and gcc, the additon of build rules for ESMF to build on the mac using ifort and gcc, and modifications to get_memusage.c to prevent it from crashing WACCM. One limitation with ifort 11.x is that you can not run WACCM with multiple processors using OPEN/MP. However, you can use multiple processors on the mac with MPI. You can also build the model using single CPU, and this is controlled by the variable USE_MPI in the make_waccm.sh and smin_4x_run.sh scripts. The scripts default to using MPI with 2 processors, but this means you will need to have already built MPICH2 and have launch the MPI deamon (mpd &). The variable WACCM_THREADS in the run script controls how many processors are used. I have only modified the smin_4x scripts and namelist, but you can make similar changes to the 2x cases and they may run if you have enough memory. I have built and run the 4 model on a laptop with 4 Gb of RAM in both the stand-alone and MPI (2 threads) modes. I have also run on an older 8-core desktop Mac using up to 8 MPI threads.

I have only tried porting to the Intel compiler, but you may be able to get WACCM to work with other compilers. If you try other compilers, let us know what worked and what additional changes you needed to make. Also, let me know if I left anything out or if you have any problems trying to follow these directions.

Thanks,

Chuck

 

[dpawlows@umich_edu]

I've been trying to get WACCM to install and compile on an intel iMac using the NAG f95 compiler. After modifying the make_waccm.sh and configure scrips, I've gotten it to get into the actual compilation of the source files, but am now stuck. The make finishes compiling the ESMF libraries, as denoted by the line ********End of build ESMF********** in MAKE.out.

Eventually, a few of the files are actually compiled, including string_utils.f90, shr_kind_mod.f90, mpishorthand.f90, among others. Once make gets to BalanceCheckMod.F90, it uses the module TIME_MANAGER. However, this has not been compiled yet, and thus there is no source file in the bld directory. The Depends file lists BalanceCheckMod.F90 as depending on time_manager, as well as other files (all of which have in fact been compiled before BalanceCheckMod). I don't see why time_manager isn't being compiled first. Any ideas?

 

[mmills@ucar_edu]

Note in the CAM makefile that the time_manager object file depends on a library in a directory that is determined by the environment variable ESMF_ARCH:

time_manager.o : $(ESMF_LIB)/$(ESMF_ARCH)/libesmf.a

This variable is set in the section specific to your architecture. Have you added a section to the CAM makefile for Darwin (the Mac version of Unix)? I am not sure if that is part of the tarball that Chuck posted in the first message in this thread. If not, I can post an example Darwin section.

In the Darwin section, there should be a definition of ESMF_ARCH, depending on whether you are building in 32-bit or 64-bit, i.e.:

#------------------------------------------------------------------------
# Darwin
#------------------------------------------------------------------------

ifeq ($(UNAMES),Darwin)

  ifeq ($(SMP),$(null))
    SMP := TRUE
  endif

  # Set the Fortran compilerq
  ifeq ($(USER_FC),$(null))
    FC := xlf90_r
  else
    FC := $(USER_FC)
  endif

  ifeq ($(USER_CC),$(null))
    CC := gcc
  else
    CC := $(USER_FC)
  endif

  ifeq ($(findstring ifort,$(FC)),ifort)
#    ADDRESS    := Q32
    ADDRESS    := Q64
    ifeq ($(ADDRESS),Q32)
      QCMP      :=
      QLDR      :=
      ESMF_ARCH := Darwin_intel
    endif
  
    ifeq ($(ADDRESS),Q64)
      QCMP      := -m64
      QLDR      := -m64
      ESMF_ARCH := Darwin_intel_64 
    endif

Do you have a directory under the ESMF build directory for your ESMF_ARCH, as defined here?

 

[dpawlows@umich_edu]

I saw that dependency, and it seems to me that the ESMF is configuring correctly. So I expected that libesmf.a had been compiled.

There exists a if UNAMES = Darwin, and I've set ESMF_ARCH to Darwin_intel.

 

[mmills@ucar_edu]

I'm looking at the MAKE.out from a successful build I did on a Mac. It shows the compilation of time_manager.F90 shortly (~100 lines) before the compilation of BalanceCheckMod.F90. Do you see that in your MAKE.out?

 

[dpawlows@umich_edu]

That's where I'd expect it too, but it isn't there. All the other dependencies for BalanceCheckMod have been compiled. It's just time_manager that hasn't. The first mention of it in MAKE.out is when BalanceCheckMod is looking for it.

 

[n64tolkien@msn_com]

I have several questions about using the toon group wiki to run CAM4.0 on a Mac. First of all I don't quite understand what MPICH and MPICH2 are because I did not see them mentioned anywhere in the CAM user's guide. I assume I don't need them if I am trying to run the code on a single computer, but I wanted to make sure. Second, I can't quite find the connection between the directories in the wiki and my directories. For example, it says to cd to

/Volumes/Data/Libraries

Is that just the directory where NETCDF is found, and if so, does it need to be in any particular directory of cam? Another example is "Go to the root directory of the source of your model that you checked out from subversion." It says to cd to

/path/to/models/cam/trunk

I don't quite understand the trunk directory. Is that the same as
($CAM_ROOT/models/atm/cam/bld) discussed in the CAM user's guide, or is it somewhere else?
Lastly, I don't understand the collection of "scripts/...-cam.csh", such as scripts/make-cam.csh
or
scripts/run-cam.csh

How do I run these scripts, because my system certainly doesn't recognize them, even after downloading the modifications in the tar file. Could I have positioned the makefile and get_memusage.c incorrectly in the code?

Sorry for so many questions, I'm just confused.

Thanks

 

[mmills@ucar_edu]

The Toon group wiki is used internally at the University of Colorado to help students running CAM3 and WACCM3 with run and build scripts developed by Chuck Bardeen while he was a student there. It does not apply to CAM4 or any released version of CAM or WACCM.

MPICH is an MPI library that is required for running CAM3-CARMA and WACCM3-CARMA on more than one processor. When running on a machine, such as a Mac Pro, with more than one core, you need a way for the processes to talk to one another (unless you run on only 1 core). This means either MPI, or OpenMP. Because the version of the CARMA microphysics package used by the Toon group is not thread-safe, OpenMP is not an option for them. Hence they generally rely on MPI.

You may find some help for running CAM4 on a Mac here or in the CAM support forum. However, it is not an officially supported platform at this time. I don't know if anyone has got that working with patches that could easily be sent to you. New platforms will be added in time, however, so I would hope the Mac would be one that is added soon.