Issue downloading input data on CESM2.1.3

[Renji2021]

Hi all,
I'm trying to run CESM2.1.3 on the HPC .when I use the ./check_input_data --download to download the input files,I meet the error:
Loading input file list: 'Buildconf/cice.input_data_list'
Loading input file list: 'Buildconf/docn.input_data_list'
Model docn missing file domain1 = '/home/renj2021/cesm/inputdata/ocn/docn7/SOM/UNSET'
Model docn missing file file1 = '/home/renj2021/cesm/inputdata/ocn/docn7/SOM/UNSET'
Model docn missing file domainfile = '/home/renj2021/cesm/inputdata/share/domains/domain.ocn.gx1v7.151008.nc'
Loading input file list: 'Buildconf/clm.input_data_list'
Model clm missing file finidat = '/home/renj2021/cesm/inputdata/lnd/clm2/initdata_map/clmi.B1850.1171-01-01.0.9x1.25_gx1v7_simyr1850_c181029.nc'
Loading input file list: 'Buildconf/cism.input_data_list'
Loading input file list: 'Buildconf/cpl.input_data_list'
Loading input file list: 'Buildconf/cam.input_data_list'
Model cam missing file ncdata = '/home/renj2021/cesm/inputdata/atm/cam/inic/fv/aqua_0006-01-01_0.9x1.25_L32_c161020.nc'
Model cam missing file depvel_lnd_file = '/home/renj2021/cesm/inputdata/atm/cam/chem/trop_mozart/dvel/regrid_vegetation_all_zero_aquaplanet_1deg_regularGrid_c20170421.nc'
Client protocol gftp not enabled
Using protocol wget with user anonymous and passwd user@example.edu
Could not connect to repo 'ftp://ftp.cgd.ucar.edu/cesm/inputdata'
This is most likely either a proxy, or network issue .
Trying to download file: '../inputdata_checksum.dat' to path '/home/renj2021/cesm/scratch/slocean/run/inputdata_checksum.dat.raw' using NoneType protocol.
Traceback (most recent call last):
File "./check_input_data", line 76, in <module>
_main_func(__doc__)
File "./check_input_data", line 71, in _main_func
chksum=chksum) else 1)
File "/home/renj2021/CESM2.1.3/cesm2.1.3/cime/scripts/Tools/../../scripts/lib/CIME/case/check_input_data.py", line 163, in check_all_input_data
_download_checksum_file(self.get_value("RUNDIR"))
File "/home/renj2021/CESM2.1.3/cesm2.1.3/cime/scripts/Tools/../../scripts/lib/CIME/case/check_input_data.py", line 54, in _download_checksum_file
success = server.getfile(rel_path, new_file)
AttributeError: 'NoneType' object has no attribute 'getfile'
It seems like my domain is blocked ,so i tried yo send the E-mail to help@cgd.ucar.edu ,but my E-mail was returned.
Please help me this regard.

 

[joneill]

Renji,

We've had a few users experience an issue with this particular address today/this weekend. When you get a chance, try running your instance of CESM2.1.3 again and, if doesn't work again, let's try to see if your public IP is being blocked by our system.

Public IP command:

dig +short myip.opendns.com @resolver1.opendns.com

Command to test connectivity:

wget --no-check-certificate --user anonymous --password user@example.edu ftp://ftp.cgd.ucar.edu/cesm/inputdata/../inputdata_checksum.dat

Let us know the result of those 2 commands and we'll go from there!

Joseph O'Neill

 

[Renji2021]

Renji,

We've had a few users experience an issue with this particular address today/this weekend. When you get a chance, try running your instance of CESM2.1.3 again and, if doesn't work again, let's try to see if your public IP is being blocked by our system.

Public IP command:


Command to test connectivity:


Let us know the result of those 2 commands and we'll go from there!

Joseph O'Neill

HI,O'Neill!
Thanks for your reply ~
First I tried the command to test connectivity I got the error message
<wget --no-check-certificate --user anonymous --password user@example.edu ftp://ftp.cgd.ucar.edu/cesm/inputdata/../inputdata_checksum.dat
--2022-09-27 14:03:04-- ftp://ftp.cgd.ucar.edu/cesm/inputdata_checksum.dat
=> ‘inputdata_checksum.dat’
Resolving ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)... 128.117.13.41
Connecting to ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)|128.117.13.41|:21... connected.
Logging in as anonymous ...
Error in server response, closing control connection.
Retrying.>

and I tried the command to get my public IP address but it gave me the error
<connection timed out; no servers could be reached>

 

[engeir]

Renji,

We've had a few users experience an issue with this particular address today/this weekend. When you get a chance, try running your instance of CESM2.1.3 again and, if doesn't work again, let's try to see if your public IP is being blocked by our system.

Public IP command:


Command to test connectivity:


Let us know the result of those 2 commands and we'll go from there!

Joseph O'Neill

Hi,

I believe I have the same problem, did you figure out what the issue was in the end?

I am running CESM2.1.3, and the issue arise during the submit step. I attempted to reorder the servers in 'config_inputdata.xml', but with no luck.

The output from the commands you listed are:

$ dig +short myip.opendns.com @resolver1.opendns.com
$ # Returns nothing, but exits successfully.
$ wget --no-check-certificate --user anonymous --password user@example.edu ftp://ftp.cgd.ucar.edu/cesm/inputdata/../inputdata_checksum.dat
--2023-01-31 13:51:20--  ftp://ftp.cgd.ucar.edu/cesm/inputdata_checksum.dat
           => ‘inputdata_checksum.dat’
Resolving ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)... 128.117.13.41
Connecting to ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)|128.117.13.41|:21... failed: Connection timed out.
Retrying.

--2023-01-31 13:53:28--  ftp://ftp.cgd.ucar.edu/cesm/inputdata_checksum.dat
  (try: 2) => ‘inputdata_checksum.dat’
Connecting to ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)|128.117.13.41|:21... ^C

Let me know if I should provide further info about how I run the code.

 

[GaoYa]

Hi,

I believe I have the same problem, did you figure out what the issue was in the end?

I am running CESM2.1.3, and the issue arise during the submit step. I attempted to reorder the servers in 'config_inputdata.xml', but with no luck.

The output from the commands you listed are:

$ dig +short myip.opendns.com @resolver1.opendns.com
$ # Returns nothing, but exits successfully.
$ wget --no-check-certificate --user anonymous --password user@example.edu ftp://ftp.cgd.ucar.edu/cesm/inputdata/../inputdata_checksum.dat
--2023-01-31 13:51:20--  ftp://ftp.cgd.ucar.edu/cesm/inputdata_checksum.dat
           => ‘inputdata_checksum.dat’
Resolving ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)... 128.117.13.41
Connecting to ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)|128.117.13.41|:21... failed: Connection timed out.
Retrying.

--2023-01-31 13:53:28--  ftp://ftp.cgd.ucar.edu/cesm/inputdata_checksum.dat
  (try: 2) => ‘inputdata_checksum.dat’
Connecting to ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)|128.117.13.41|:21... ^C

Let me know if I should provide further info about how I run the code.

Hello, I also meet the same problem, do you slove it?

 

[engeir]

Hello, I also meet the same problem, do you slove it?

No, I have instead been downloading the files manually, and placing them where where the model was looking.

The file `cime/config/cesm/config_inputdata.xml` lists the strategies for downloading, the svn option has the URL to where they can be found from a browser: inputdata

 

[GaoYa]

No, I have instead been downloading the files manually, and placing them where where the model was looking.

The file `cime/config/cesm/config_inputdata.xml` lists the strategies for downloading, the svn option has the URL to where they can be found from a browser: inputdata

Thanks for your reply, I downloaded the inputdata_checksum.dat, but I am not sure where to put it. The following is the file path of my error, so I need to put the file to '/WORK/sysu_fliu_1/cesm2/cesm_output//B1850test2.f09_g17/run/inputdata_checksum.dat.raw'?

"Using protocol wget with user anonymous and passwd user@example.edu
Could not connect to repo 'ftp://ftp.cgd.ucar.edu/cesm/inputdata/'
This is most likely either a proxy, or network issue .
Trying to download file: '../inputdata_checksum.dat' to path '/WORK/sysu_fliu_1/cesm2/cesm_output//B1850test2.f09_g17/run/inputdata_checksum.dat.raw' using NoneType protocol."

 

[engeir]

Sorry, I meant the `.nc` file that the model initially look for, before trying to download it. The error I had and that I assume you have was that during submit of a simulation, it said something like:

Cannot find `path/to/some_file.nc`
Trying to download...
Using protocol wget ...

And so on, like the output you have above. The file `some_file.nc` is what you can find on https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata, and then place the file at `path/to/`.

 

[GaoYa]

Sorry, I meant the `.nc` file that the model initially look for, before trying to download it. The error I had and that I assume you have was that during submit of a simulation, it said something like:

Cannot find `path/to/some_file.nc`
Trying to download...
Using protocol wget ...

And so on, like the output you have above. The file `some_file.nc` is what you can find on https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata, and then place the file at `path/to/`.

Ok, I understand, thank you very much.

 

[khj]

Looks like I figured out the problem. When the input data is being downloaded inside the site or the input data server, the following data is followed by the following path:
/home/user/inputdata/cesm2_init/f.e21.FWsc2000climo.f09_f09_mg17.cesm2.1-exp011.001/0003-01-01/f.e21.FWsc2000climo.f09_f09_mg17.cesm2.1-exp011.001.cam.i .0003-01-01-00000.nc
But inside your site, this path has a v2 both inside the path and the file name.
/cesm/inutdata/cesm2_init/f.e21.fwsc2000Climo.f09_f09_mg17.cesm2.1-exp011.001_v2/0003-01-01-01/f.e21.FWsc2000climo.f09_f09_mg17.cesm2.1-exp011.001_v2.cam.i.0003-01-01-00000.nc
However, what can be done? How to continue downloading input data?