Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

Issue downloading input data on CESM2.1.3

R

Renji2021

RENJI
New Member
Hi all,
I'm trying to run CESM2.1.3 on the HPC .when I use the ./check_input_data --download to download the input files,I meet the error:
Loading input file list: 'Buildconf/cice.input_data_list'
Loading input file list: 'Buildconf/docn.input_data_list'
Model docn missing file domain1 = '/home/renj2021/cesm/inputdata/ocn/docn7/SOM/UNSET'
Model docn missing file file1 = '/home/renj2021/cesm/inputdata/ocn/docn7/SOM/UNSET'
Model docn missing file domainfile = '/home/renj2021/cesm/inputdata/share/domains/domain.ocn.gx1v7.151008.nc'
Loading input file list: 'Buildconf/clm.input_data_list'
Model clm missing file finidat = '/home/renj2021/cesm/inputdata/lnd/clm2/initdata_map/clmi.B1850.1171-01-01.0.9x1.25_gx1v7_simyr1850_c181029.nc'
Loading input file list: 'Buildconf/cism.input_data_list'
Loading input file list: 'Buildconf/cpl.input_data_list'
Loading input file list: 'Buildconf/cam.input_data_list'
Model cam missing file ncdata = '/home/renj2021/cesm/inputdata/atm/cam/inic/fv/aqua_0006-01-01_0.9x1.25_L32_c161020.nc'
Model cam missing file depvel_lnd_file = '/home/renj2021/cesm/inputdata/atm/cam/chem/trop_mozart/dvel/regrid_vegetation_all_zero_aquaplanet_1deg_regularGrid_c20170421.nc'
Client protocol gftp not enabled
Using protocol wget with user anonymous and passwd user@example.edu
Could not connect to repo 'ftp://ftp.cgd.ucar.edu/cesm/inputdata'
This is most likely either a proxy, or network issue .
Trying to download file: '../inputdata_checksum.dat' to path '/home/renj2021/cesm/scratch/slocean/run/inputdata_checksum.dat.raw' using NoneType protocol.
Traceback (most recent call last):
File "./check_input_data", line 76, in <module>
_main_func(__doc__)
File "./check_input_data", line 71, in _main_func
chksum=chksum) else 1)
File "/home/renj2021/CESM2.1.3/cesm2.1.3/cime/scripts/Tools/../../scripts/lib/CIME/case/check_input_data.py", line 163, in check_all_input_data
_download_checksum_file(self.get_value("RUNDIR"))
File "/home/renj2021/CESM2.1.3/cesm2.1.3/cime/scripts/Tools/../../scripts/lib/CIME/case/check_input_data.py", line 54, in _download_checksum_file
success = server.getfile(rel_path, new_file)
AttributeError: 'NoneType' object has no attribute 'getfile'
It seems like my domain is blocked ,so i tried yo send the E-mail to help@cgd.ucar.edu ,but my E-mail was returned.
Please help me this regard.
 
joneill

joneill

Joseph O'Neill
Administrator
Staff member
Renji,

We've had a few users experience an issue with this particular address today/this weekend. When you get a chance, try running your instance of CESM2.1.3 again and, if doesn't work again, let's try to see if your public IP is being blocked by our system.

Public IP command:
dig +short myip.opendns.com @resolver1.opendns.com
Click to expand...

Command to test connectivity:
wget --no-check-certificate --user anonymous --password user@example.edu ftp://ftp.cgd.ucar.edu/cesm/inputdata/../inputdata_checksum.dat
Click to expand...

Let us know the result of those 2 commands and we'll go from there!

Joseph O'Neill
 
R

Renji2021

RENJI
New Member
joneill said:
Renji,

We've had a few users experience an issue with this particular address today/this weekend. When you get a chance, try running your instance of CESM2.1.3 again and, if doesn't work again, let's try to see if your public IP is being blocked by our system.

Public IP command:


Command to test connectivity:


Let us know the result of those 2 commands and we'll go from there!

Joseph O'Neill
Click to expand...
HI,O'Neill!
Thanks for your reply ~
First I tried the command to test connectivity I got the error message
<wget --no-check-certificate --user anonymous --password user@example.edu ftp://ftp.cgd.ucar.edu/cesm/inputdata/../inputdata_checksum.dat
--2022-09-27 14:03:04-- ftp://ftp.cgd.ucar.edu/cesm/inputdata_checksum.dat
=> ‘inputdata_checksum.dat’
Resolving ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)... 128.117.13.41
Connecting to ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)|128.117.13.41|:21... connected.
Logging in as anonymous ...
Error in server response, closing control connection.
Retrying.>

and I tried the command to get my public IP address but it gave me the error
<connection timed out; no servers could be reached>
 
E

engeir

Eirik Enger
New Member
joneill said:
Renji,

We've had a few users experience an issue with this particular address today/this weekend. When you get a chance, try running your instance of CESM2.1.3 again and, if doesn't work again, let's try to see if your public IP is being blocked by our system.

Public IP command:


Command to test connectivity:


Let us know the result of those 2 commands and we'll go from there!

Joseph O'Neill
Click to expand...
Hi,

I believe I have the same problem, did you figure out what the issue was in the end?

I am running CESM2.1.3, and the issue arise during the submit step. I attempted to reorder the servers in 'config_inputdata.xml', but with no luck.

The output from the commands you listed are:

Bash: 
$ dig +short myip.opendns.com @resolver1.opendns.com
$ # Returns nothing, but exits successfully.
$ wget --no-check-certificate --user anonymous --password user@example.edu ftp://ftp.cgd.ucar.edu/cesm/inputdata/../inputdata_checksum.dat
--2023-01-31 13:51:20--  ftp://ftp.cgd.ucar.edu/cesm/inputdata_checksum.dat
           => ‘inputdata_checksum.dat’
Resolving ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)... 128.117.13.41
Connecting to ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)|128.117.13.41|:21... failed: Connection timed out.
Retrying.

--2023-01-31 13:53:28--  ftp://ftp.cgd.ucar.edu/cesm/inputdata_checksum.dat
  (try: 2) => ‘inputdata_checksum.dat’
Connecting to ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)|128.117.13.41|:21... ^C

Let me know if I should provide further info about how I run the code.
 
G

GaoYa

GaoYa
New Member
engeir said:
Hi,

I believe I have the same problem, did you figure out what the issue was in the end?

I am running CESM2.1.3, and the issue arise during the submit step. I attempted to reorder the servers in 'config_inputdata.xml', but with no luck.

The output from the commands you listed are:

Bash: 
$ dig +short myip.opendns.com @resolver1.opendns.com
$ # Returns nothing, but exits successfully.
$ wget --no-check-certificate --user anonymous --password user@example.edu ftp://ftp.cgd.ucar.edu/cesm/inputdata/../inputdata_checksum.dat
--2023-01-31 13:51:20--  ftp://ftp.cgd.ucar.edu/cesm/inputdata_checksum.dat
           => ‘inputdata_checksum.dat’
Resolving ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)... 128.117.13.41
Connecting to ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)|128.117.13.41|:21... failed: Connection timed out.
Retrying.

--2023-01-31 13:53:28--  ftp://ftp.cgd.ucar.edu/cesm/inputdata_checksum.dat
  (try: 2) => ‘inputdata_checksum.dat’
Connecting to ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)|128.117.13.41|:21... ^C

Let me know if I should provide further info about how I run the code.
Click to expand...
Hello, I also meet the same problem, do you slove it?
 
E

engeir

Eirik Enger
New Member
GaoYa said:
Hello, I also meet the same problem, do you slove it?
Click to expand...
No, I have instead been downloading the files manually, and placing them where where the model was looking.

The file `cime/config/cesm/config_inputdata.xml` lists the strategies for downloading, the svn option has the URL to where they can be found from a browser: inputdata
 
G

GaoYa

GaoYa
New Member
engeir said:
No, I have instead been downloading the files manually, and placing them where where the model was looking.

The file `cime/config/cesm/config_inputdata.xml` lists the strategies for downloading, the svn option has the URL to where they can be found from a browser: inputdata
Click to expand...
Thanks for your reply, I downloaded the inputdata_checksum.dat, but I am not sure where to put it. The following is the file path of my error, so I need to put the file to '/WORK/sysu_fliu_1/cesm2/cesm_output//B1850test2.f09_g17/run/inputdata_checksum.dat.raw'?

"Using protocol wget with user anonymous and passwd user@example.edu
Could not connect to repo 'ftp://ftp.cgd.ucar.edu/cesm/inputdata/'
This is most likely either a proxy, or network issue .
Trying to download file: '../inputdata_checksum.dat' to path '/WORK/sysu_fliu_1/cesm2/cesm_output//B1850test2.f09_g17/run/inputdata_checksum.dat.raw' using NoneType protocol."
 
E

engeir

Eirik Enger
New Member
Sorry, I meant the `.nc` file that the model initially look for, before trying to download it. The error I had and that I assume you have was that during submit of a simulation, it said something like:

Cannot find `path/to/some_file.nc`
Trying to download...
Using protocol wget ...

And so on, like the output you have above. The file `some_file.nc` is what you can find on https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata, and then place the file at `path/to/`.
 
G

GaoYa

GaoYa
New Member
engeir said:
Sorry, I meant the `.nc` file that the model initially look for, before trying to download it. The error I had and that I assume you have was that during submit of a simulation, it said something like:

Cannot find `path/to/some_file.nc`
Trying to download...
Using protocol wget ...

And so on, like the output you have above. The file `some_file.nc` is what you can find on https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata, and then place the file at `path/to/`.
Click to expand...
Ok, I understand, thank you very much.
 
K

khj

Parisa
Member
Looks like I figured out the problem. When the input data is being downloaded inside the site or the input data server, the following data is followed by the following path:
/home/user/inputdata/cesm2_init/f.e21.FWsc2000climo.f09_f09_mg17.cesm2.1-exp011.001/0003-01-01/f.e21.FWsc2000climo.f09_f09_mg17.cesm2.1-exp011.001.cam.i .0003-01-01-00000.nc
But inside your site, this path has a v2 both inside the path and the file name.
/cesm/inutdata/cesm2_init/f.e21.fwsc2000Climo.f09_f09_mg17.cesm2.1-exp011.001_v2/0003-01-01-01/f.e21.FWsc2000climo.f09_f09_mg17.cesm2.1-exp011.001_v2.cam.i.0003-01-01-00000.nc
However, what can be done? How to continue downloading input data?
 
You must log in or register to reply here.
Top