CTSM5.2.0/5.4.0 inputdata download error

J

jbuzan_vub

Jonathan R. Buzan
New Member
Hello CESMDiscuss users:

I am attempting to download the input data for a test run for CTSM5.2.0 and CTSM5.4.0.
compset: IHistClm50BgcCrop
res: f09_g17
Below i post an example from CTSM5.2.0, where i have everything but one file.

Cheers,
-Jonathan


./check_input_data --download
Setting resource.RLIMIT_STACK to -1 from (8388608, -1)
Loading input file list: 'Buildconf/mosart.input_data_list'
Loading input file list: 'Buildconf/datm.input_data_list'
Loading input file list: 'Buildconf/cpl.input_data_list'
Loading input file list: 'Buildconf/ctsm.input_data_list'
Model ctsm missing file flanduse_timeseries = '/dodrio/scratch/projects/2022_200/project_input/cesm/inputdata/lnd/clm2/surfdata_esmf/ctsm5.2.0/landuse.timeseries_0.9x1.25_SSP2-4.5_1850-2100_78pfts_c240216.nc'

Checking server ftp://gridanon.cgd.ucar.edu:2811/cesm/inputdata/ with protocol gftp
Setting resource.RLIMIT_STACK to -1 from (-1, -1)
Client protocol gftp not enabled
Checking server ftp://ftp.cgd.ucar.edu/cesm/inputdata/ with protocol wget
Setting resource.RLIMIT_STACK to -1 from (-1, -1)
Using protocol wget with user anonymous and passwd user@example.edu
Could not connect to repo 'ftp://ftp.cgd.ucar.edu/cesm/inputdata/'
This is most likely either a proxy, or network issue .(location 1)
Checking server ftp.cgd.ucar.edu/cesm/inputdata with protocol ftp
Setting resource.RLIMIT_STACK to -1 from (-1, -1)
Using protocol ftp with user anonymous and passwd user@example.edu
server address ftp.cgd.ucar.edu root path cesm/inputdata
ftp login timeout!
Checking server - Revision 70792: /trunk/inputdata with protocol svn

Setting resource.RLIMIT_STACK to -1 from (-1, -1)
Using protocol svn with user and passwd
Loading input file list: 'Buildconf/mosart.input_data_list'
Loading input file list: 'Buildconf/datm.input_data_list'
Loading input file list: 'Buildconf/cpl.input_data_list'
Loading input file list: 'Buildconf/ctsm.input_data_list'


Model ctsm missing file flanduse_timeseries = '/dodrio/scratch/projects/2022_200/project_input/cesm/inputdata/lnd/clm2/surfdata_esmf/ctsm5.2.0/landuse.timeseries_0.9x1.25_SSP2-4.5_1850-2100_78pfts_c240216.nc'
Trying to download file: 'lnd/clm2/surfdata_esmf/ctsm5.2.0/landuse.timeseries_0.9x1.25_SSP2-4.5_1850-2100_78pfts_c240216.nc' to path '/dodrio/scratch/projects/2022_200/project_input/cesm/inputdata/lnd/clm2/surfdata_esmf/ctsm5.2.0/landuse.timeseries_0.9x1.25_SSP2-4.5_1850-2100_78pfts_c240216.nc' using SVN protocol.
svn export failed with output: and errput svn: E000013: Can't create temporary file from template '/dodrio/scratch/projects/2022_200/project_input/cesm/inputdata/lnd/clm2/surfdata_esmf/ctsm5.2.0/svn-XXXXXX': Permission denied

Checking server None with protocol None
Setting resource.RLIMIT_STACK to -1 from (-1, -1)
Client protocol None not enabled
ERROR: Could not find all inputdata on any server
 
J

jbuzan_vub

Jonathan R. Buzan
New Member
Thanks, Jim.
The Hortense system is down for maintenance this week, so I will try again next week.

Cheers,
-Jonathan
 
J

jbuzan_vub

Jonathan R. Buzan
New Member
Hi Jim,

I tried to do the manual download, but it seems to be timing out.

wget https://osdf-data.gdex.ucar.edu/nca...0.9x1.25_SSP2-4.5_1850-2100_78pfts_c240216.nc

--2026-03-18 13:49:24-- https://osdf-data.gdex.ucar.edu/nca...0.9x1.25_SSP2-4.5_1850-2100_78pfts_c240216.nc
Resolving osdf-data.gdex.ucar.edu (osdf-data.gdex.ucar.edu)... 128.117.251.130
Connecting to osdf-data.gdex.ucar.edu (osdf-data.gdex.ucar.edu)|128.117.251.130|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently

Location: https://osdf-director.osg-htc.org:4...0.9x1.25_SSP2-4.5_1850-2100_78pfts_c240216.nc [following]
--2026-03-18 13:49:25-- https://osdf-director.osg-htc.org/n...0.9x1.25_SSP2-4.5_1850-2100_78pfts_c240216.nc
Resolving osdf-director.osg-htc.org (osdf-director.osg-htc.org)... 128.105.82.132, 2607:f388:2200:c3::3
Connecting to osdf-director.osg-htc.org (osdf-director.osg-htc.org)|128.105.82.132|:443... connected.
HTTP request sent, awaiting response... 307 Temporary Redirect

Location: https://osg-cache.ms4.surfsara.nl:8...0.9x1.25_SSP2-4.5_1850-2100_78pfts_c240216.nc [following]
--2026-03-18 13:49:25-- https://osg-cache.ms4.surfsara.nl:8...0.9x1.25_SSP2-4.5_1850-2100_78pfts_c240216.nc
Resolving osg-cache.ms4.surfsara.nl (osg-cache.ms4.surfsara.nl)... 145.38.219.74
Connecting to osg-cache.ms4.surfsara.nl (osg-cache.ms4.surfsara.nl)|145.38.219.74|:8443... failed: Connection timed out.
Retrying.

(went on for 7 times before i killed the job).
 
J

jbuzan_vub

Jonathan R. Buzan
New Member
Hello Jim,
I have worked with IT at Hortense, and they think part of the problem is the random server link.

"
I tried on my side, and it works equally badly from my home than from Hortense. The issue is that Director | Pelican Platform redirects to a random mirror server, and then depending on where you land you get a different response.

For instance, this is from my home, in the following case I landed in https://amst-fiona.nationalresearchplatform.org and that resulted in a 500 error, which means that that server is missconfigured

$ wget https://osdf-data.gdex.ucar.edu/nca...0.9x1.25_SSP2-4.5_1850-2100_78pfts_c240216.nc
HTTP response 301 Moved Permanently [https://osdf-data.gdex.ucar.edu/nca...0.9x1.25_SSP2-4.5_1850-2100_78pfts_c240216.nc]
Enqueue https://osdf-director.osg-htc.org:4...0.9x1.25_SSP2-4.5_1850-2100_78pfts_c240216.nc
HTTP response 307 [https://osdf-director.osg-htc.org:4...0.9x1.25_SSP2-4.5_1850-2100_78pfts_c240216.nc]
Enqueue https://amst-fiona.nationalresearch...0.9x1.25_SSP2-4.5_1850-2100_78pfts_c240216.nc
HTTP ERROR response 500 Internal Server Error [https://amst-fiona.nationalresearch...0.9x1.25_SSP2-4.5_1850-2100_78pfts_c240216.nc]

...
and in a third case I landed onto a Spanish server, that one started the download, but it was extremely slow at 600 kb/s

$ wget https://osdf-data.gdex.ucar.edu/nca...0.9x1.25_SSP2-4.5_1850-2100_78pfts_c240216.nc
--2026-03-19 09:39:26-- https://osdf-data.gdex.ucar.edu/nca...0.9x1.25_SSP2-4.5_1850-2100_78pfts_c240216.nc
Resolving osdf-data.gdex.ucar.edu (osdf-data.gdex.ucar.edu)... 128.117.251.130
Connecting to osdf-data.gdex.ucar.edu (osdf-data.gdex.ucar.edu)|128.117.251.130|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://osdf-director.osg-htc.org:4...0.9x1.25_SSP2-4.5_1850-2100_78pfts_c240216.nc [following]
--2026-03-19 09:39:27-- https://osdf-director.osg-htc.org/n...0.9x1.25_SSP2-4.5_1850-2100_78pfts_c240216.nc
Resolving osdf-director.osg-htc.org (osdf-director.osg-htc.org)... 128.105.82.132, 2607:f388:2200:c3::3
Connecting to osdf-director.osg-htc.org (osdf-director.osg-htc.org)|128.105.82.132|:443... connected.
HTTP request sent, awaiting response... 307 Temporary Redirect
Location: https://lond-osdf-xcache01.es.net:8...0.9x1.25_SSP2-4.5_1850-2100_78pfts_c240216.nc [following]
--2026-03-19 09:39:28-- https://lond-osdf-xcache01.es.net:8...0.9x1.25_SSP2-4.5_1850-2100_78pfts_c240216.nc
Resolving lond-osdf-xcache01.es.net (lond-osdf-xcache01.es.net)... 198.124.239.10, 2001:400:bc01:11d0::2
Connecting to lond-osdf-xcache01.es.net (lond-osdf-xcache01.es.net)|198.124.239.10|:8443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 26020603668 (24G)
Saving to: ‘landuse.timeseries_0.9x1.25_SSP2-4.5_1850-2100_78pfts_c240216.nc’

landuse.timeseries_0.9x1.25_SSP2-4.5_1850-2100_78pfts_c240216. 0%[ ] 10.00M 603.2KB/s eta 6d 3h

"
 
You must log in or register to reply here.
Back
Top