Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

Question about downloading CESM2?

M

mengqi

mj
Member
Hi,

I am a beginner using CESM2. I am downloading the CESM2 code and scripts; however, the attempt was unsuccessful.

When I ran the script manage_externals/checkout_externals, it did not work. But I have no idea how to solve it.

Could anyone please offer me some insights?

Thanks!

Please see below:

(npl) mengqij@derecho3:~/cesm_MYBRANCH> ./manage_externals/checkout_externals

Processing externals description file : Externals.cfg
Checking status of externals: cam, cice, cime, cism, clm, mosart, pop, rtm, ww3,
Checking out externals: cam, cice, cime, cism, clm, mosart, pop, rtm, ww3,
Processing externals description file : Externals_CAM.cfg
Checking out externals: chem_proc, Error validating server certificate for 'https://svn-ccsm-models.cgd.ucar.edu:443':
- The certificate is not issued by a trusted authority. Use the
fingerprint to validate the certificate manually!
Certificate information:
- Hostname: *.cgd.ucar.edu
- Valid: from Jan 22 00:00:00 2024 GMT until Jan 21 23:59:59 2025 GMT
- Issuer: InCommon RSA Server CA 2, Internet2, US
- Fingerprint: FE:F4:34:1F:65:00:76:40:21:5C:07:15:F8:32:BC:66:9C:55:18:9F

(R)eject, accept (t)emporarily or accept (p)ermanently? p

carma, clubb, cosp2, ERROR:root:Command '['svn', 'checkout', '--quiet', 'https://github.com/CFMIP/COSPv2.0/tags/v2.1.4cesm/src', '/glade/u/home/mengqij/cesm_MYBRANCH/components/cam/src/physics/cosp2/src']' returned non-zero exit status 1.
ERROR:root:Failed with output:
svn: E170013: Unable to connect to a repository at URL 'https://github.com/CFMIP/COSPv2.0/tags/v2.1.4cesm/src'
svn: E160013: '/CFMIP/COSPv2.0/tags/v2.1.4cesm/src' path not found

ERROR: In directory
/glade/u/home/mengqij/cesm_MYBRANCH
Process did not run successfully; returned status 1:
svn checkout --quiet https://github.com/CFMIP/COSPv2.0/tags/v2.1.4cesm/src /glade/u/home/mengqij/cesm_MYBRANCH/components/cam/src/physics/cosp2/src
See above for output from failed command.
ERROR:root:Failed with output:
svn: E170013: Unable to connect to a repository at URL 'https://github.com/CFMIP/COSPv2.0/tags/v2.1.4cesm/src'
svn: E160013: '/CFMIP/COSPv2.0/tags/v2.1.4cesm/src' path not found

ERROR: In directory
/glade/u/home/mengqij/cesm_MYBRANCH
Process did not run successfully; returned status 1:
svn checkout --quiet https://github.com/CFMIP/COSPv2.0/tags/v2.1.4cesm/src /glade/u/home/mengqij/cesm_MYBRANCH/components/cam/src/physics/cosp2/src
See above for output from failed command.
ERROR: Failed with output:
svn: E170013: Unable to connect to a repository at URL 'https://github.com/CFMIP/COSPv2.0/tags/v2.1.4cesm/src'
svn: E160013: '/CFMIP/COSPv2.0/tags/v2.1.4cesm/src' path not found

ERROR: In directory
/glade/u/home/mengqij/cesm_MYBRANCH
Process did not run successfully; returned status 1:
svn checkout --quiet https://github.com/CFMIP/COSPv2.0/tags/v2.1.4cesm/src /glade/u/home/mengqij/cesm_MYBRANCH/components/cam/src/physics/cosp2/src
See above for output from failed command.
 
K

katec

CSEG and Liaisons
Staff member
Hi, so there was a change in Git that caused manage_externals to stop working with some of our code. We have updated the code to fix this. You will need to use a newer version of the CESM code. For more information, see this thread:
bb.cgd.ucar.edu

github changes will require updates to cesm2.x tags

Due to github.com discontinuing support for svn access manage_externals will soon give an error when trying to checkout most cesm and cam tags. The CESM software engineering group, CSEG, is currently in the process of creating new tags: cesm2.1.5, cesm2.2.2 and cesm2.3.beta17 which will...
bb.cgd.ucar.edu bb.cgd.ucar.edu
 
M

mengqi

mj
Member
katec said:
Hi, so there was a change in Git that caused manage_externals to stop working with some of our code. We have updated the code to fix this. You will need to use a newer version of the CESM code. For more information, see this thread:
bb.cgd.ucar.edu

github changes will require updates to cesm2.x tags

Due to github.com discontinuing support for svn access manage_externals will soon give an error when trying to checkout most cesm and cam tags. The CESM software engineering group, CSEG, is currently in the process of creating new tags: cesm2.1.5, cesm2.2.2 and cesm2.3.beta17 which will...
bb.cgd.ucar.edu bb.cgd.ucar.edu
Click to expand...
Thank you! It helps!
The old question seems to be addressed. But there is another one below. Could you please give me some input? Thanks!

(npl) mengqij@derecho3:~/cesm_MYBRANCH> ./manage_externals/checkout_externals
----------------------------------------------------------------------
Processing externals description file : Externals.cfg
Processing externals description file : Externals_CAM.cfg
Processing externals description file : Externals_CISM.cfg
Processing externals description file : Externals_CLM.cfg
Processing externals description file : Externals_POP.cfg
Checking status of externals: cam, chem_proc, carma, clubb, cosp2, cice, cime, cism, source_cism, clm, fates, ptclm, mosart, pop, cvmix, marbl, rtm, ww3,
./cime
sM ./components/cam
./components/cam/chem_proc
./components/cam/src/physics/carma/base
./components/cam/src/physics/clubb
e- ./components/cam/src/physics/cosp2/src
./components/cice
./components/cism
e- ./components/cism/source_cism
./components/clm
e- ./components/clm/src/fates
e- ./components/clm/tools/PTCLM
./components/mosart
./components/pop
e- ./components/pop/externals/CVMix
e- ./components/pop/externals/MARBL
./components/rtm
./components/ww3
----------------------------------------------------------------------
The external repositories labeled with 'M' above are not in a clean state.
The following are two options for how to proceed:
(1) Go into each external that is not in a clean state and issue either
an 'svn status' or a 'git status' command. Either revert or commit
your changes so that all externals are in a clean state. (Note,
though, that it is okay to have untracked files in your working
directory.) Then rerun checkout_externals.

(2) Alternatively, you do not have to rely on checkout_externals. Instead, you
can manually update out-of-sync externals (labeled with 's' above)
as described in the configuration file Externals.cfg.

The external repositories labeled with '?' above are not under version control using the expected protocol. If you are sure you want to switch protocols, and you don't have any work you need to save from this directory, then run "rm -rf [directory]" before re-running the checkout_externals tool.
 
M

mengqi

mj
Member
Hi @katec

To echo the last message, I changed to another version: cesm2.1.5.

I did the following:
$ git clone -b cesm2.1.5 GitHub - ESCOMP/CESM: The Community Earth System Model cesm2.1.5
$ cd cesm2.1.5
$ ./manage_externals/checkout_externals -v

It did not indicate whether the download was successful, and I have no experience. Could you please confirm if it works? Please refer to the information provided below:

(npl) mengqij@derecho1:~/cesm_215> ./manage_externals/checkout_externals -v
Processing externals description file : Externals.cfg (/glade/u/home/mengqij/cesm_215)
Checking local status of required & optional components: cam, cice, cime, cism, clm, mosart, pop, rtm, ww3,
Checking out externals:
e- ./cime
-, not checked out --> cime5.6.49
git clone --quiet GitHub - ESMCI/cime: Common Infrastructure for Modeling the Earth /glade/u/home/mengqij/cesm_215/cime
git -C /glade/u/home/mengqij/cesm_215/cime checkout --quiet cime5.6.49
e- ./components/cam
-, not checked out --> cam_cesm2_1_rel_60
git clone --quiet GitHub - ESCOMP/CAM: Community Atmosphere Model /glade/u/home/mengqij/cesm_215/components/cam
git -C /glade/u/home/mengqij/cesm_215/components/cam checkout --quiet cam_cesm2_1_rel_60
Processing externals description file : Externals_CAM.cfg (/glade/u/home/mengqij/cesm_215/components/cam)
Checking out externals:

e- chem_proc
-, not checked out --> chem_proc5_0_03_rel
svn checkout --quiet https://svn-ccsm-models.cgd.ucar.edu/tools/proc_atm/chem_proc/release_tags/chem_proc5_0_03_rel /glade/u/home/mengqij/cesm_215/components/cam/chem_proc
e- src/physics/carma/base
-, not checked out --> carma3_49_rel
svn checkout --quiet https://svn-ccsm-models.cgd.ucar.edu/carma/release_tags/carma3_49_rel /glade/u/home/mengqij/cesm_215/components/cam/src/physics/carma/base
e- src/physics/clubb
-, not checked out --> vendor_clubb_r8099_n03
git clone --quiet GitHub - ESCOMP/CLUBB_CESM: CESM version of CLUBB /glade/u/home/mengqij/cesm_215/components/cam/src/physics/clubb
git -C /glade/u/home/mengqij/cesm_215/components/cam/src/physics/clubb checkout --quiet vendor_clubb_r8099_n03
e- src/physics/cosp2/src
-, not checked out --> v2.1.4cesm
git clone --quiet GitHub - CFMIP/COSPv2.0: COSP - The CFMIP (Cloud Feedbacks Model Intercomparison Project) Observation Simulator Package /glade/u/home/mengqij/cesm_215/components/cam/src/physics/cosp2/src
git -C /glade/u/home/mengqij/cesm_215/components/cam/src/physics/cosp2/src checkout --quiet v2.1.4cesm
cp /glade/u/home/mengqij/cesm_215/components/cam/src/physics/cosp2/src/../.cosp_sparse_checkout /glade/u/home/mengqij/cesm_215/components/cam/src/physics/cosp2/src/.git/info/sparse-checkout
git -C /glade/u/home/mengqij/cesm_215/components/cam/src/physics/cosp2/src config core.sparsecheckout true
git -C /glade/u/home/mengqij/cesm_215/components/cam/src/physics/cosp2/src read-tree -mu HEAD
e- ./components/cice
-, not checked out --> cice5_cesm2_1_1_20231220
git clone --quiet GitHub - ESCOMP/CESM_CICE5: CESM2 Version of CICE /glade/u/home/mengqij/cesm_215/components/cice
git -C /glade/u/home/mengqij/cesm_215/components/cice checkout --quiet cice5_cesm2_1_1_20231220
e- ./components/cism
-, not checked out --> cism-release-cesm2.1.2_04
git clone --quiet GitHub - ESCOMP/CISM-wrapper: Community Ice Sheet Model wrapper for CESM /glade/u/home/mengqij/cesm_215/components/cism
git -C /glade/u/home/mengqij/cesm_215/components/cism checkout --quiet cism-release-cesm2.1.2_04
Processing externals description file : Externals_CISM.cfg (/glade/u/home/mengqij/cesm_215/components/cism)
Checking out externals:
e- source_cism
-, not checked out --> release-cism2.1.04
git clone --quiet GitHub - ESCOMP/CISM: Community Ice Sheet Model /glade/u/home/mengqij/cesm_215/components/cism/source_cism
git -C /glade/u/home/mengqij/cesm_215/components/cism/source_cism checkout --quiet release-cism2.1.04
e- ./components/clm
-, not checked out --> release-clm5.0.37
git clone --quiet GitHub - ESCOMP/CTSM: Community Terrestrial Systems Model (includes the Community Land Model of CESM) /glade/u/home/mengqij/cesm_215/components/clm
git -C /glade/u/home/mengqij/cesm_215/components/clm checkout --quiet release-clm5.0.37
Processing externals description file : Externals_CLM.cfg (/glade/u/home/mengqij/cesm_215/components/clm)
Checking out externals:
e- src/fates
-, not checked out --> sci.1.30.0_api.8.0.0
git clone --quiet GitHub - NGEET/fates: repository for the Functionally Assembled Terrestrial Ecosystem Simulator (FATES) /glade/u/home/mengqij/cesm_215/components/clm/src/fates
git -C /glade/u/home/mengqij/cesm_215/components/clm/src/fates checkout --quiet sci.1.30.0_api.8.0.0
e- ./components/mosart
-, not checked out --> release-cesm2.0.04
git clone --quiet GitHub - ESCOMP/MOSART: Model for Scale Adaptive River Transport, Mosart, part of the Community Earth System Model /glade/u/home/mengqij/cesm_215/components/mosart
git -C /glade/u/home/mengqij/cesm_215/components/mosart checkout --quiet release-cesm2.0.04
e- ./components/pop
-, not checked out --> pop2_cesm2_1_rel_n15
git clone --quiet GitHub - ESCOMP/POP2-CESM: Parallel Ocean Program (POP2) in CESM /glade/u/home/mengqij/cesm_215/components/pop
git -C /glade/u/home/mengqij/cesm_215/components/pop checkout --quiet pop2_cesm2_1_rel_n15
Processing externals description file : Externals_POP.cfg (/glade/u/home/mengqij/cesm_215/components/pop)
Checking out externals:
e- externals/CVMix
-, not checked out --> v0.93-beta
git clone --quiet GitHub - CVMix/CVMix-src: CVMix source code (including protex documentation) as well as build system and examples / tests /glade/u/home/mengqij/cesm_215/components/pop/externals/CVMix
git -C /glade/u/home/mengqij/cesm_215/components/pop/externals/CVMix checkout --quiet v0.93-beta
e- externals/MARBL
-, not checked out --> cesm2.1-n00
git clone --quiet GitHub - marbl-ecosys/MARBL: Marine Biogeochemistry Library /glade/u/home/mengqij/cesm_215/components/pop/externals/MARBL
git -C /glade/u/home/mengqij/cesm_215/components/pop/externals/MARBL checkout --quiet cesm2.1-n00
e- ./components/rtm
-, not checked out --> release-cesm2.0.04
git clone --quiet GitHub - ESCOMP/RTM: River Transport Model, RTM, part of the Community Earth System Model /glade/u/home/mengqij/cesm_215/components/rtm
git -C /glade/u/home/mengqij/cesm_215/components/rtm checkout --quiet release-cesm2.0.04
e- ./components/ww3
-, not checked out --> ww3_cesm2_1_rel_01
git clone --quiet GitHub - ESCOMP/WW3-CESM: WaveWatch III (WW3) in CESM /glade/u/home/mengqij/cesm_215/components/ww3
git -C /glade/u/home/mengqij/cesm_215/components/ww3 checkout --quiet ww3_cesm2_1_rel_01

(npl) mengqij@derecho1:~/cesm_215>
 
K

katec

CSEG and Liaisons
Staff member
Hi, so for your first post, it looks like there is some modified code in the CAM directory that you need to remove before manage_externals will continue. As the error says: "
The external repositories labeled with 'M' above are not in a clean state.
The following are two options for how to proceed:
(1) Go into each external that is not in a clean state and issue either
an 'svn status' or a 'git status' command. Either revert or commit
your changes so that all externals are in a clean state. (Note,
though, that it is okay to have untracked files in your working
directory.) Then rerun checkout_externals."

For your second one, to see if manage_externals worked, do the following from the root of your code:
./manage_externals/checkout_externals -S

That will give you the status for each component. If you don't see any letters to the right of the component paths, then you have been successful.
 
You must log in or register to reply here.
Top