Configuring CESM 2.1.5 without svn

[BenT]

Essentially, what is the correct solution to install and configure CESM 2.1.5 without the need for svn?

By default, it appears to require svn, as directed by /CESM_HOME/components/cam/Externals_CAM.cfg

The thread below appears to provide a solution to remove the need for svn, by modifying (I assume) the above file.

J

Thread 'git instead of svn ./manage_externals/checkout_externals ?'

Jul 1, 2024

Hi DiscussCESM users:

I am trying to get CESM installed on Eiger so I can run benchmarking.

Is there a workaround using GitHub instead of svn?
I am working on a new cluster, Eiger (part of Alps CSCS.ch), which has a similar architecture to Derecho. In all my years working on clusters, this is the first time svn is not part of the machine. The cluster support for Eiger has stated that I need to install svn myself, but svn that has many dependencies, so this is a larger task than anticipated.

For functions, like, manage_externals, and data downloading, is there a GitHub workaround...

• Jbuzan
• Replies: 8
• Forum: Infrastructure (CIME, porting, machines, scripts)

However, after changing the protocal to git, I run ./manage_externals/checkout_externals, and I get this complaint, which seems to arise due to the modification:

Checking local status of required & optional components: cam, chem_proc, carma, clubb, cosp2, cice, cime, cism, clm, mosart, pop, rtm, ww3,
    ./cime
 M  ./components/cam
e-  ./components/cam/chem_proc
e-  ./components/cam/src/physics/carma/base
e-  ./components/cam/src/physics/clubb
e-  ./components/cam/src/physics/cosp2/src
e-  ./components/cice
e-  ./components/cism
e-  ./components/clm
e-  ./components/mosart
e-  ./components/pop
e-  ./components/rtm
e-  ./components/ww3
----------------------------------------------------------------------
The external repositories labeled with 'M' above are not in a clean state.
The following are four options for how to proceed:
(1) Go into each external that is not in a clean state and issue either a 'git status' or
    an 'svn status' command (depending on whether the external is managed by git or
    svn). Either revert or commit your changes so that all externals are in a clean
    state. (To revert changes in git, follow the instructions given when you run 'git
    status'.) (Note, though, that it is okay to have untracked files in your working
    directory.) Then rerun checkout_externals.
(2) Alternatively, you do not have to rely on checkout_externals. Instead, you can manually
    update out-of-sync externals (labeled with 's' above) as described in the
    configuration file Externals.cfg. (For example, run 'git fetch' and 'git checkout'
    commands to checkout the appropriate tags for each external, as given in
    Externals.cfg.)
(3) You can also use checkout_externals to manage most, but not all externals: You can specify
    one or more externals to ignore using the '-x' or '--exclude' argument to
    checkout_externals. Excluding externals labeled with 'M' will allow checkout_externals to
    update the other, non-excluded externals.
(4) As a last resort, if you are confident that there is no work that needs to be saved
    from a given external, you can remove that external (via "rm -rf [directory]") and
    then rerun the checkout_externals tool. This option is mainly useful as a workaround for
    issues with this tool (such as https://github.com/ESMCI/manage_externals/issues/157).
The external repositories labeled with '?' above are not under version
control using the expected protocol. If you are sure you want to switch
protocols, and you don't have any work you need to save from this
directory, then run "rm -rf [directory]" before rerunning the
checkout_externals tool.

 

[BenT]

To update, and I think answer my question:

So, shooting in the dark a bit, it appears that cam6_3_153 is a suitable tag for CAM under CESM 2.1.5, that removes the svn dependence, so I have modified the tag in root level Externals.cfg file:
[cam]
tag = cam6_3_153
protocol = git
repo_url = GitHub - ESCOMP/CAM: Community Atmosphere Model
local_path = components/cam
externals = Externals_CAM.cfg
required = True

./manage_externals/checkout_externals seems to have completed successfully without svn.

(Not really sure why this is no default for CESM 2.1.5 since svn seems kind of obsolete and is a nightmare to build).

 

[BenT]

To clarify, the CAM tag is cam6_3_135, for the commit that completely removes svn.

However, I'm running into issues with CAM chemistry ...
ERROR: Variable 'dep_data_file' from file '<case_root>/Buildconf/camconf/drv_flds_in' is not in the namelist definition.

I assume this is simply because cam6_3_135 is not synchronized with cesm2.1.5.

So, what it is that I'm supposed to do to remove the svn dependency? The changes to Externals_CAM.cfg are trivial, and already included in later CAM versions, but it's not clear how I make those changes without ./manage_externals/checkout_externals telling me that there is a consistency problem...

 

[fischer]

Hi Ben,

cam6_3_135 isn't compatible with cesm2.1.5. I did worked out a way to be able to checkout cesm2.1.5 without svn. I did this by doing a manual checkout for cam. I've included the steps that I used below. Let me know if this works for you. Since all of the systems I have access to have svn installed, it was challenging to test.

git clone -b cesm2.1.5 https://github.com/ESCOMP/cesm cesm2.1.5
cd cesm2.1.5
./manage_externals/checkout_externals -x cam
git clone -b cam_cesm2_1_rel_60 https://github.com/ESCOMP/cam components/cam
cd components/cam
git clone -b chem_proc5_0_04 https://github.com/ESCOMP/CHEM_PREPROCESSOR chem_proc
git clone -b carma3_49 https://github.com/ESCOMP/CARMA_base src/physics/carma/base
git clone -b vendor_clubb_r8099_n03 https://github.com/ESCOMP/CLUBB_CESM src/physics/clubb
git clone -b v2.1.4cesm https://github.com/CFMIP/COSPv2.0 src/physics/cosp2/src

You should be able to cut and paste these commands.

Chris

 

[BenT]

Hi Chris,

Many thanks for the follow-up. I actually went ahead and built a new conda environment since it turned out svn was an easy install that way, although it did take a bit of messing around reconfiguring other libraries. I will take a shot and let you know if this works. I'm getting segfaults on the execution but I can't get any wallclock time to test at the moment.