Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

Problem in downloading input data sets (ERROR: Could not connect to repo '{0}') -- clm5.0

A

Anuj

New Member
Dear all,

I am facing problem in downloading input data sets.
I need help from you on following :
How i have to set protocol, passwd, username etc.
From where/which website we have to create username etc.
Please help me this regard.

Model datm missing file file240 = '/home/shaan/CLM/CLMinput/CLMforcings/atm_forcing.datm7.cruncep_qianFill.0.5d.v7.c160715/TPHWL6Hrly/clmforc.cruncep.V7.c2016.0.5d.TPQWL.1920-12.nc'
Model datm missing file domain1 = '/home/shaan/CLM/CLMinput/atm/cam/chem/trop_mozart_aero/aero/'
Model datm missing file file1 = '/home/shaan/CLM/CLMinput/atm/cam/chem/trop_mozart_aero/aero/aerosoldep_WACCM.ensmean_monthly_hist_1849-2015_0.9x1.25_CMIP6_c180926.nc'
Model datm missing file domain1 = '/home/shaan/CLM/CLMinput/atm/datm7/topo_forcing/'
Model datm missing file file1 = '/home/shaan/CLM/CLMinput/atm/datm7/topo_forcing/topodata_0.9x1.25_USGS_070110_stream_c151201.nc'
Model datm missing file domainfile = '/home/shaan/CLM/CLMinput/share/domains/domain.lnd.fv1.9x2.5_gx1v7.181205.nc'
Client protocol gftp not enabled
Using protocol wget with user anonymous and passwd user@example.edu
ERROR: Could not connect to repo '{0}'
This is most likely either a proxy, or network issue .
 

Attachments

  • Screenshot from 2019-12-23 15-15-19.png
    Screenshot from 2019-12-23 15-15-19.png
    672.6 KB · Views: 33
D

divanova@ucsd_edu

New Member
I have similar problem and I don't think it is related to the CLM5.0. I am working with the latest release cesm2.1.3 on pleaides (NASA)
After compiling successfully my case I tried several ways to get the missing inputdata files:

1) check_input_data with wget- NO SUCCESS
pfe23.divanova 226> ./check_input_data -i /nobackup/divanova/cesm2/inputdata/ --download --p wget --server ftp://ftp.cgd.ucar.edu/cesm/inputdata
Using protocol wget with user None and passwd None
Could not connect to repo 'ftp://ftp.cgd.ucar.edu/cesm/inputdata'
This is most likely either a proxy, or network issue .
ERROR: Could not find all inputdata on any server

2) wget manually (one file) - SUCCESS

pfe23.divanova 227> wget ftp://ftp.cgd.ucar.edu/cesm/inputdata/ocn/pop/gx1v7/grid/region_mask_20151008.ieeei4
--2020-03-31 22:52:15-- ftp://ftp.cgd.ucar.edu/cesm/inputdata/ocn/pop/gx1v7/grid/region_mask_20151008.ieeei4
=> 'region_mask_20151008.ieeei4.1'
Resolving ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)... 128.117.23.220
Connecting to ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)|128.117.23.220|:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done. ==> PWD ... done.
==> TYPE I ... done. ==> CWD (1) /cesm/inputdata/ocn/pop/gx1v7/grid ... done.
==> SIZE region_mask_20151008.ieeei4 ... 491520
==> PASV ... done. ==> RETR region_mask_20151008.ieeei4 ... done.
Length: 491520 (480K) (unauthoritative)
100%[================================================================================================================================>] 491,520 2.20MB/s in 0.2s
2020-03-31 22:52:16 (2.20 MB/s) - 'region_mask_20151008.ieeei4.1' saved [491520]

3) check_input_data with svn - NO SUCCESS
pfe23.divanova 229> ./check_input_data -i /nobackup/divanova/cesm2/inputdata/ --download --p svn --server https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata

Using protocol svn with user None and passwd None
Loading input file list: 'Buildconf/cam.input_data_list'
Model cam missing file ncdata = '/nobackup/divanova/cesm2/inputdata/atm/cam/inic/fv/cami-mam3_0000-01-01_0.9x1.25_L30_c100618.nc'
Trying to download file: 'atm/cam/inic/fv/cami-mam3_0000-01-01_0.9x1.25_L30_c100618.nc' to path '/nobackup/divanova/cesm2/inputdata/atm/cam/inic/fv/cami-mam3_0000-01-01_0.9x1.25_L30_c100618.nc' using SVN protocol.
SUCCESS

ERROR: Failed to find or download file /nobackup/divanova/b.BC5L45BGC.f09_g16/run/inputdata_checksum.dat

4) svn manually (one directory) - SUCCESS
pfe23.divanova 233> svn co https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata/ocn/pop/gx1v7/grid/
A grid/topography_20161215.ieeei4
A grid/region_mask_20151008.ieeei4
A grid/horiz_grid_20010402.ieeer8
Checked out revision 33431.

I suspect the issues are different in the different protocols but any suggestions would be appreciated.
 
J

jedwards

CSEG and Liaisons
Staff member
Are you in a case directory when you do this? I don't understand why 2 would work but not 1.
Can you run ./check_input_data --download --debug
 
D

divanova@ucsd_edu

New Member
Yes I am in my case directory.
here is the end of what I got running the command you suggested:
===========================================================

Using protocol wget with user anonymous and passwd user@example.edu


RUN: wget --no-check-certificate --user anonymous --password user@example.edu --spider ftp://ftp.cgd.ucar.edu/cesm/inputdata


stat: 8





errput: --2020-04-01 09:02:17-- ftp://ftp.cgd.ucar.edu/cesm/inputdata


=> 'inputdata'


Resolving ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)... 128.117.23.220


Connecting to ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)|128.117.23.220|:21... connected.


Logging in as anonymous ... Logged in!


==> SYST ... done. ==> PWD ... done.


==> TYPE I ... done. ==> CWD (1) /cesm ... done.


==> SIZE inputdata ... done.





==> PASV ... done. --2020-04-01 09:02:17-- ftp://ftp.cgd.ucar.edu/cesm/inputdata


=> '.listing'


==> CWD (1) /cesm ... done.


==> PASV ... done. ==> LIST ... done.





0K 3.72K=0.2s





Removed '.listing'.





File 'inputdata' exists.





Could not connect to repo 'ftp://ftp.cgd.ucar.edu/cesm/inputdata'


This is most likely either a proxy, or network issue .


Trying to download file: '../inputdata_checksum.dat' to path '/nobackup/divanova/b.BC5L45BGC.f09_g16/run/inputdata_checksum.dat.raw' using NoneType protocol.


Traceback (most recent call last):


File "./check_input_data", line 76, in <module>


_main_func(__doc__)


File "./check_input_data", line 71, in _main_func


chksum=chksum) else 1)


File "/home4/divanova/cesm2/cesm2.1.3.repo/cime/scripts/Tools/../../scripts/lib/CIME/case/check_input_data.py", line 163, in check_all_input_data


_download_checksum_file(self.get_value("RUNDIR"))


File "/home4/divanova/cesm2/cesm2.1.3.repo/cime/scripts/Tools/../../scripts/lib/CIME/case/check_input_data.py", line 54, in _download_checksum_file


success = server.getfile(rel_path, new_file)


AttributeError: 'NoneType' object has no attribute 'getfile'


pf
 
J

jedwards

CSEG and Liaisons
Staff member
Looks like it works and then it doesn't. What are the permissions of directory $DIN_LOC_ROOT?
 
J

jedwards

CSEG and Liaisons
Staff member
This is a problem on your end somehow and I'm not sure how to help you further. If you run check_input_data with no arguments it should provide a list of the files that you need, since downloading by hand seems to work you can do that from the provided list.
 
B

bidyut

BIDYUT BIKASH GOSWAMI
Member
Client protocol gftp not enabled
Using protocol wget with user anonymous and passwd user@example.edu ... it stays there forever ... any clue? I have svn and got already loaded. Thanks.
 
B

bidyut

BIDYUT BIKASH GOSWAMI
Member
Connecting to ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)|128.117.23.220|:21... failed: Connection timed out.
 
J

jedwards

CSEG and Liaisons
Staff member
So either your domain is blocking ours or ours is blacklisting yours. If you can confirm that you should be able to reach our website then
send email to help@cgd.ucar.edu and ask if you are blacklisted - include your ip address in the message.
 
B

bidyut

BIDYUT BIKASH GOSWAMI
Member
"If you can confirm that you should be able to reach our website" ... I "ping"-ed ftp.cgd.ucar.edu and there is no loss ... does it confirm?
 
You must log in or register to reply here.
Top