Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

Problem in downloading input data sets (ERROR: Could not connect to repo '{0}') -- clm5.0

zorkgtzorkgt

Xie Zhijie
New Member
I have similar problem and I don't think it is related to the CLM5.0. I am working with the latest release cesm2.1.3 on pleaides (NASA)
After compiling successfully my case I tried several ways to get the missing inputdata files:

1) check_input_data with wget- NO SUCCESS
pfe23.divanova 226> ./check_input_data -i /nobackup/divanova/cesm2/inputdata/ --download --p wget --server ftp://ftp.cgd.ucar.edu/cesm/inputdata
Using protocol wget with user None and passwd None
Could not connect to repo 'ftp://ftp.cgd.ucar.edu/cesm/inputdata'
This is most likely either a proxy, or network issue .
ERROR: Could not find all inputdata on any server

2) wget manually (one file) - SUCCESS

pfe23.divanova 227> wget ftp://ftp.cgd.ucar.edu/cesm/inputdata/ocn/pop/gx1v7/grid/region_mask_20151008.ieeei4
--2020-03-31 22:52:15-- ftp://ftp.cgd.ucar.edu/cesm/inputdata/ocn/pop/gx1v7/grid/region_mask_20151008.ieeei4
=> 'region_mask_20151008.ieeei4.1'
Resolving ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)... 128.117.23.220
Connecting to ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)|128.117.23.220|:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done. ==> PWD ... done.
==> TYPE I ... done. ==> CWD (1) /cesm/inputdata/ocn/pop/gx1v7/grid ... done.
==> SIZE region_mask_20151008.ieeei4 ... 491520
==> PASV ... done. ==> RETR region_mask_20151008.ieeei4 ... done.
Length: 491520 (480K) (unauthoritative)
100%[================================================================================================================================>] 491,520 2.20MB/s in 0.2s
2020-03-31 22:52:16 (2.20 MB/s) - 'region_mask_20151008.ieeei4.1' saved [491520]

3) check_input_data with svn - NO SUCCESS
pfe23.divanova 229> ./check_input_data -i /nobackup/divanova/cesm2/inputdata/ --download --p svn --server https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata

Using protocol svn with user None and passwd None
Loading input file list: 'Buildconf/cam.input_data_list'
Model cam missing file ncdata = '/nobackup/divanova/cesm2/inputdata/atm/cam/inic/fv/cami-mam3_0000-01-01_0.9x1.25_L30_c100618.nc'
Trying to download file: 'atm/cam/inic/fv/cami-mam3_0000-01-01_0.9x1.25_L30_c100618.nc' to path '/nobackup/divanova/cesm2/inputdata/atm/cam/inic/fv/cami-mam3_0000-01-01_0.9x1.25_L30_c100618.nc' using SVN protocol.
SUCCESS

ERROR: Failed to find or download file /nobackup/divanova/b.BC5L45BGC.f09_g16/run/inputdata_checksum.dat

4) svn manually (one directory) - SUCCESS
pfe23.divanova 233> svn co https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata/ocn/pop/gx1v7/grid/
A grid/topography_20161215.ieeei4
A grid/region_mask_20151008.ieeei4
A grid/horiz_grid_20010402.ieeer8
Checked out revision 33431.

I suspect the issues are different in the different protocols but any suggestions would be appreciated.
Yes I am in my case directory.
here is the end of what I got running the command you suggested:
===========================================================

Using protocol wget with user anonymous and passwd user@example.edu


RUN: wget --no-check-certificate --user anonymous --password user@example.edu --spider ftp://ftp.cgd.ucar.edu/cesm/inputdata


stat: 8





errput: --2020-04-01 09:02:17-- ftp://ftp.cgd.ucar.edu/cesm/inputdata


=> 'inputdata'


Resolving ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)... 128.117.23.220


Connecting to ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)|128.117.23.220|:21... connected.


Logging in as anonymous ... Logged in!


==> SYST ... done. ==> PWD ... done.


==> TYPE I ... done. ==> CWD (1) /cesm ... done.


==> SIZE inputdata ... done.





==> PASV ... done. --2020-04-01 09:02:17-- ftp://ftp.cgd.ucar.edu/cesm/inputdata


=> '.listing'


==> CWD (1) /cesm ... done.


==> PASV ... done. ==> LIST ... done.





0K 3.72K=0.2s





Removed '.listing'.





File 'inputdata' exists.





Could not connect to repo 'ftp://ftp.cgd.ucar.edu/cesm/inputdata'


This is most likely either a proxy, or network issue .


Trying to download file: '../inputdata_checksum.dat' to path '/nobackup/divanova/b.BC5L45BGC.f09_g16/run/inputdata_checksum.dat.raw' using NoneType protocol.


Traceback (most recent call last):


File "./check_input_data", line 76, in <module>


_main_func(__doc__)


File "./check_input_data", line 71, in _main_func


chksum=chksum) else 1)


File "/home4/divanova/cesm2/cesm2.1.3.repo/cime/scripts/Tools/../../scripts/lib/CIME/case/check_input_data.py", line 163, in check_all_input_data


_download_checksum_file(self.get_value("RUNDIR"))


File "/home4/divanova/cesm2/cesm2.1.3.repo/cime/scripts/Tools/../../scripts/lib/CIME/case/check_input_data.py", line 54, in _download_checksum_file


success = server.getfile(rel_path, new_file)


AttributeError: 'NoneType' object has no attribute 'getfile'


pf
Hello,have you solve your problem 1 now?
I have similar problem to build a case
 
Hi CESM staff,
I, likewise, am having a similar issue. But when I connect using one of the suggested methods above, I get an error saying that there's no such directory as 'cesm'.

Cheers,
-Jonathan


I executed:
> wget --no-check-certificate --user anonymous --password user@example.edu --spider ftp://ftp.cgd.ucar.edu/cesm/inputdata
--2022-01-05 16:10:45-- ftp://ftp.cgd.ucar.edu/cesm/inputdata
=> ‘inputdata’
Resolving ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)... 128.117.23.220
Connecting to ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)|128.117.23.220|:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done. ==> PWD ... done.
==> TYPE I ... done. ==> CWD (1) /cesm ...
No such directory ‘cesm’.
 
Addendum:
I contacted my IT staff at my cluster.

../cime/config/cesm/
modify
config_inputdata.xml

Put the SVN command first. The staff said it downloads the first file, then the checksum cannot be found.

Cheers,
-Jonathan
 

aschloegl

Alois Schlögl
New Member
Seems like a problem on our end, I'm looking into it now.
I git the same problem with cesm-2.1.3. A slash "/" is missing at the end of the address. This patch fixes this:

diff --git a/config/cesm/config_inputdata.xml b/config/cesm/config_inputdata.xml index 985f5ee02..9274bb896 100644 --- a/config/cesm/config_inputdata.xml +++ b/config/cesm/config_inputdata.xml @@ -16,7 +16,7 @@ <server> <protocol>wget</protocol> - <address>ftp://ftp.cgd.ucar.edu/cesm/inputdata</address> + <address>ftp://ftp.cgd.ucar.edu/cesm/inputdata/</address> <user>anonymous</user> <password>user@example.edu</password> <checksum>../inputdata_checksum.dat</checksum> @@ -25,7 +25,7 @@ <server> <comment> ftp requires the python package ftplib </comment> <protocol>ftp</protocol> - <address>ftp.cgd.ucar.edu/cesm/inputdata</address> + <address>ftp.cgd.ucar.edu/cesm/inputdata/</address> <user>anonymous</user> <password>user@example.edu</password> <checksum>../inputdata_checksum.dat</checksum>

To give credit where credit is due, the solution was posted first here:
 
Top