case submission error

pappu · Aug 28, 2024

./manage_externals/checkout_externals --status --verbose

Processing externals description file : Externals.cfg
Processing externals description file : Externals_CLM.cfg
Processing externals description file : Externals_POP.cfg
Processing externals description file : Externals_CISM.cfg
Checking status of externals: clm, fates, ptclm, mosart, ww3, cime, cice, pop, cvmix, marbl, cism, source_cism, rtm, cam,
M ./cime
modified sandbox, on cime5.6.19
./components/cam
clean sandbox, on cam1/release_tags/cam_cesm2_1_rel_29/components/cam
./components/cice
clean sandbox, on cice5_cesm2_1_1_20190321
./components/cism
clean sandbox, on release-cesm2.0.04
./components/cism/source_cism
clean sandbox, on release-cism2.1.03
./components/clm
clean sandbox, on release-clm5.0.25
./components/clm/src/fates
clean sandbox, on fates_s1.21.0_a7.0.0_br_rev2
./components/clm/tools/PTCLM
clean sandbox, on PTCLM2_180611
./components/mosart
clean sandbox, on release-cesm2.0.03
./components/pop
clean sandbox, on pop2_cesm2_1_rel_n06
./components/pop/externals/CVMix
clean sandbox, on v0.93-beta
./components/pop/externals/MARBL
clean sandbox, on cesm2.1-n00
./components/rtm
clean sandbox, on release-cesm2.0.02
./components/ww3
clean sandbox, on ww3_181001

Steps:
./create_newcase -case testF2000 -compset F2000climo -res f09_f09_mg17 -mach keeling
./case.setup
./preview_run
./case.build
./case.submit

My case build successfully but I got error in ./case.submit step:

Model clm missing file fsnowoptics = '/data/keeling/a/pappup2/a/CESM2/CESM2_input_data/lnd/clm2/snicardata/snicar_optics_5bnd_c090915.nc'
Trying to download file: 'lnd/clm2/snicardata/snicar_optics_5bnd_c090915.nc' to path '/data/keeling/a/pappup2/a/CESM2/CESM2_input_data/lnd/clm2/snicardata/snicar_optics_5bnd_c090915.nc' using WGET protocol.
SUCCESS
Traceback (most recent call last):
File "./case.submit", line 107, in <module>
_main_func(__doc__)
File "./case.submit", line 104, in _main_func
mail_user=mail_user, mail_type=mail_type, batch_args=batch_args)
File "/data/keeling/a/pappup2/my_cesm_sandbox/cime/scripts/Tools/../../scripts/lib/CIME/case/case_submit.py", line 156, in submit
custom_success_msg_functor=verbatim_success_msg)
File "/data/keeling/a/pappup2/my_cesm_sandbox/cime/scripts/Tools/../../scripts/lib/CIME/utils.py", line 1667, in run_and_log_case_status
rv = func()
File "/data/keeling/a/pappup2/my_cesm_sandbox/cime/scripts/Tools/../../scripts/lib/CIME/case/case_submit.py", line 154, in <lambda>
batch_args=batch_args)
File "/data/keeling/a/pappup2/my_cesm_sandbox/cime/scripts/Tools/../../scripts/lib/CIME/case/case_submit.py", line 84, in _submit
case.check_case()
File "/data/keeling/a/pappup2/my_cesm_sandbox/cime/scripts/Tools/../../scripts/lib/CIME/case/case_submit.py", line 170, in check_case
self.check_all_input_data()
File "/data/keeling/a/pappup2/my_cesm_sandbox/cime/scripts/Tools/../../scripts/lib/CIME/case/check_input_data.py", line 161, in check_all_input_data
success = _downloadfromserver(self, input_data_root, data_list_dir)
File "/data/keeling/a/pappup2/my_cesm_sandbox/cime/scripts/Tools/../../scripts/lib/CIME/case/check_input_data.py", line 183, in _downloadfromserver
user=user, passwd=passwd)
File "/data/keeling/a/pappup2/my_cesm_sandbox/cime/scripts/Tools/../../scripts/lib/CIME/case/check_input_data.py", line 340, in check_input_data
verify_chksum(input_data_root, rundir, rel_path.strip(os.sep), isdirectory)
File "/data/keeling/a/pappup2/my_cesm_sandbox/cime/scripts/Tools/../../scripts/lib/CIME/case/check_input_data.py", line 380, in verify_chksum
logger.warning("Did not find hash for file {} in chksum file {}".format(filename, hashfile))
UnboundLocalError: local variable 'hashfile' referenced before assignment

jedwards · Aug 28, 2024

Thank you very much for following the reporting process, it's really helpful.
You have an older version of the cesm, you should update to cesm2.1.5

pappu · Aug 28, 2024

Hi @jedwards
I tried download the CESM V2.1.5 but got this issue

git clone -b release-cesm2.1.5 GitHub - ESCOMP/CESM: The Community Earth System Model CESM2_1_5
git checkout release-cesm2.1.5
./manage_externals/checkout_externals
Processing externals description file : Externals.cfg (/data/keeling/a/pappup2/CESM2_1_5)
Checking local status of required & optional components: cam, cice, cime, cism, clm, mosart, pop, rtm, ww3,
Checking out externals: cime, ERROR:root:Command '['git', '-C', '/data/keeling/a/pappup2/CESM2_1_5/cime', 'remote', 'add', 'ESMCI_cime', 'GitHub - ESMCI/cime: Common Infrastructure for Modeling the Earth']' returned non-zero exit status 129.
ERROR:root:Failed with output:
Unknown option: -C
usage: git [--version] [--help] [-c name=value]
[--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]
[-p|--paginate|--no-pager] [--no-replace-objects] [--bare]
[--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]
<command> [<args>]
ERROR: In directory
/data/keeling/a/pappup2/CESM2_1_5
Process did not run successfully; returned status 129:
git -C /data/keeling/a/pappup2/CESM2_1_5/cime remote add ESMCI_cime GitHub - ESMCI/cime: Common Infrastructure for Modeling the Earth
See above for output from failed command.
ERROR:root:Failed with output:
Unknown option: -C
usage: git [--version] [--help] [-c name=value]
[--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]
[-p|--paginate|--no-pager] [--no-replace-objects] [--bare]
[--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]
<command> [<args>]

ERROR: In directory
/data/keeling/a/pappup2/CESM2_1_5
Process did not run successfully; returned status 129:
git -C /data/keeling/a/pappup2/CESM2_1_5/cime remote add ESMCI_cime GitHub - ESMCI/cime: Common Infrastructure for Modeling the Earth
See above for output from failed command.
ERROR: Failed with output:
Unknown option: -C
usage: git [--version] [--help] [-c name=value]
[--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]
[-p|--paginate|--no-pager] [--no-replace-objects] [--bare]
[--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]
<command> [<args>]
ERROR: In directory
/data/keeling/a/pappup2/CESM2_1_5
Process did not run successfully; returned status 129:
git -C /data/keeling/a/pappup2/CESM2_1_5/cime remote add ESMCI_cime GitHub - ESMCI/cime: Common Infrastructure for Modeling the Earth
See above for output from failed command.

jedwards · Aug 28, 2024

I think that your version of git must be too old. I am using 2.41.0, how about you?

pappu · Aug 28, 2024

Mine is 1.8.3.1

jedwards · Aug 28, 2024

get a new git: GitHub - git/git: Git Source Code Mirror - This is a publish-only repository but pull requests can be turned into patches to the mailing list via GitGitGadget (https://gitgitgadget.github.io/). Please follow Documentation/SubmittingPatches procedure for any of your improvements.

pappu · Aug 28, 2024

I have a Derecho account and git version in Derecho is 2.41.0. I tried download CESM v2.1.5 source code in Derecho got the same error. So I suspect this issue I am facing is not for git.

`````````````````Error is Derecho````````````````````````

pappup@derecho5:~/CESM2> ./manage_externals/checkout_externals

Processing externals description file : Externals.cfg (/glade/u/home/pappup/CESM2)

Processing externals description file : Externals_CAM.cfg (/glade/u/home/pappup/CESM2/components/cam)

Checking local status of required & optional components: cam, chem_proc, carma, clubb, cosp2, cice, cime, cism, clm, mosart, pop, rtm, ww3,

Checking out externals: cime, cam, Processing externals description file : Externals_CAM.cfg (/glade/u/home/pappup/CESM2/components/cam)

Checking out externals: chem_proc, ERROR:root:Command '['svn', 'checkout', '--quiet', 'https://svn-ccsm-models.cgd.ucar.edu/tools/proc_atm/chem_proc/release_tags/chem_proc5_0_03_rel', '/glade/u/home/pappup/CESM2/components/cam/chem_proc']' returned non-zero exit status 1.

ERROR:root:Failed with output:

svn: E170013: Unable to connect to a repository at URL 'https://svn-ccsm-models.cgd.ucar.edu/tools/proc_atm/chem_proc/release_tags/chem_proc5_0_03_rel'

svn: E175003: The server at 'https://svn-ccsm-models.cgd.ucar.edu/tools/proc_atm/chem_proc/release_tags/chem_proc5_0_03_rel' does not support the HTTP/DAV protocol

ERROR: In directory

/glade/u/home/pappup/CESM2/cime

Process did not run successfully; returned status 1:

svn checkout --quiet https://svn-ccsm-models.cgd.ucar.edu/tools/proc_atm/chem_proc/release_tags/chem_proc5_0_03_rel /glade/u/home/pappup/CESM2/components/cam/chem_proc

See above for output from failed command.

ERROR:root:Failed with output:

svn: E170013: Unable to connect to a repository at URL 'https://svn-ccsm-models.cgd.ucar.edu/tools/proc_atm/chem_proc/release_tags/chem_proc5_0_03_rel'

svn: E175003: The server at 'https://svn-ccsm-models.cgd.ucar.edu/tools/proc_atm/chem_proc/release_tags/chem_proc5_0_03_rel' does not support the HTTP/DAV protocol

ERROR: In directory

/glade/u/home/pappup/CESM2/cime

Process did not run successfully; returned status 1:

svn checkout --quiet https://svn-ccsm-models.cgd.ucar.edu/tools/proc_atm/chem_proc/release_tags/chem_proc5_0_03_rel /glade/u/home/pappup/CESM2/components/cam/chem_proc

See above for output from failed command.

ERROR: Failed with output:

svn: E170013: Unable to connect to a repository at URL 'https://svn-ccsm-models.cgd.ucar.edu/tools/proc_atm/chem_proc/release_tags/chem_proc5_0_03_rel'

svn: E175003: The server at 'https://svn-ccsm-models.cgd.ucar.edu/tools/proc_atm/chem_proc/release_tags/chem_proc5_0_03_rel' does not support the HTTP/DAV protocol

ERROR: In directory

/glade/u/home/pappup/CESM2/cime

Process did not run successfully; returned status 1:

svn checkout --quiet https://svn-ccsm-models.cgd.ucar.edu/tools/proc_atm/chem_proc/release_tags/chem_proc5_0_03_rel /glade/u/home/pappup/CESM2/components/cam/chem_proc

See above for output from failed command.

jedwards · Aug 28, 2024

no - it's a different error. See git instead of svn ./manage_externals/checkout_externals ?

pappu · Aug 28, 2024

Thank you for the responses, I have able to solve the issue. I used the steps below

./manage_externals/checkout_externals
*got svn error

cd components/cam/
Edit Externals_CAM.cfg
- replace these two part instead of the original svn command part

[chem_proc]
local_path = chem_proc
protocol = git
repo_url = GitHub - ESCOMP/CHEM_PREPROCESSOR: Chemistry preprocessor used by CAM-Chem and WACCM
tag = chem_proc5_0_04
required = True

[carma]
local_path = src/physics/carma/base
protocol = git
repo_url = GitHub - ESCOMP/CARMA_base: CARMA base source code
tag = carma3_49
required = True

./manage_externals/checkout_externals
cd ../..
Edit Externals.cfg
- remove the [cam] segmanet
./manage_externals/checkout_externals
./manage_externals/checkout_externals -Sv

Checking local status of required & optional components: cice, cime, cism, source_cism, clm, fates, mosart, pop, cvmix, marbl, rtm, ww3,

./cime

clean sandbox, on cime5.6.49

./components/cice

clean sandbox, on cice5_cesm2_1_1_20231220

./components/cism

clean sandbox, on cism-release-cesm2.1.2_04

./components/cism/source_cism

clean sandbox, on release-cism2.1.04

./components/clm

clean sandbox, on release-clm5.0.37

./components/clm/src/fates

clean sandbox, on sci.1.30.0_api.8.0.0

./components/mosart

clean sandbox, on release-cesm2.0.04

./components/pop

clean sandbox, on pop2_cesm2_1_rel_n15

./components/pop/externals/CVMix

clean sandbox, on v0.93-beta

./components/pop/externals/MARBL

clean sandbox, on cesm2.1-n00

./components/rtm

clean sandbox, on release-cesm2.0.04

./components/ww3

clean sandbox, on ww3_181001

wbr2023 · Sep 4, 2024

pappu said:
Thank you for the responses, I have able to solve the issue. I used the steps below

./manage_externals/checkout_externals
*got svn error

cd components/cam/

Edit Externals_CAM.cfg

replace these two part instead of the original svn command part

[chem_proc]
local_path = chem_proc
protocol = git
repo_url = GitHub - ESCOMP/CHEM_PREPROCESSOR: Chemistry preprocessor used by CAM-Chem and WACCM
tag = chem_proc5_0_04
required = True

[carma]
local_path = src/physics/carma/base
protocol = git
repo_url = GitHub - ESCOMP/CARMA_base: CARMA base source code
tag = carma3_49
required = True

./manage_externals/checkout_externals

cd ../..

Edit Externals.cfg

remove the [cam] segmanet

./manage_externals/checkout_externals

./manage_externals/checkout_externals -Sv

Checking local status of required & optional components: cice, cime, cism, source_cism, clm, fates, mosart, pop, cvmix, marbl, rtm, ww3,

./cime

clean sandbox, on cime5.6.49

./components/cice

clean sandbox, on cice5_cesm2_1_1_20231220

./components/cism

clean sandbox, on cism-release-cesm2.1.2_04

./components/cism/source_cism

clean sandbox, on release-cism2.1.04

./components/clm

clean sandbox, on release-clm5.0.37

./components/clm/src/fates

clean sandbox, on sci.1.30.0_api.8.0.0

./components/mosart

clean sandbox, on release-cesm2.0.04

./components/pop

clean sandbox, on pop2_cesm2_1_rel_n15

./components/pop/externals/CVMix

clean sandbox, on v0.93-beta

./components/pop/externals/MARBL

clean sandbox, on cesm2.1-n00

./components/rtm

clean sandbox, on release-cesm2.0.04

./components/ww3

clean sandbox, on ww3_181001

hi! Does this approach ensure the integrity of cam components.

pappu · Sep 5, 2024

wbr2023 said:
hi! Does this approach ensure the integrity of cam components.

I dont know what do mean by integrity of cam components but this method gives me clean sandbox status for all components

wbr2023 · Sep 5, 2024

pappu said:
I dont know what do mean by integrity of cam components but this method gives me clean sandbox status for all components

Thank you for your reply. I used your method to generate redundant cime and component files in the cam directory. Do I need to delete the cime and component directories under the cam directory?

pappu · Sep 9, 2024

wbr2023 said:
Thank you for your reply. I used your method to generate redundant cime and component files in the cam directory. Do I need to delete the cime and component directories under the cam directory?

Nope you are good to go I think

case submission error

pappu

Pappu Paul

New Member

Attachments

jedwards

CSEG and Liaisons

pappu

Pappu Paul

New Member

jedwards

CSEG and Liaisons

pappu

Pappu Paul

New Member

jedwards

CSEG and Liaisons

pappu

Pappu Paul

New Member

jedwards

CSEG and Liaisons

pappu

Pappu Paul

New Member

wbr2023

彬睿王

New Member

pappu

Pappu Paul

New Member

wbr2023

彬睿王

New Member

pappu

Pappu Paul

New Member