input data downloading

[devarajun@gmail_com]

Hello, I am trying to download a 28GB inputdata file ( https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata/ocn/pop/tx0.1v2/ic/GC007v3.pop.r.0001-01-16-00000 ).I tried both manually and through svn. But i am not able to download it. Is there any easy way to download this huge file. Many Thanks

 

[mai]

I do not know whether this helps you or not, but I could split the file into 10 pieces, put the pieces into a public ftp area and you could download it with a parallel wget.

 

[fischer]

What error message are you getting?

 

[devarajun@gmail_com]

Hi mai, Thanks a lot for thereply. I dont know how do I split the file into 10parts. Can you please guide me how do I do that.If  I download manually, it says download completed but the downloaded file size will be 500mb . and through wget I tried below is the error:  wget https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata/ocn/pop/tx0.1v2/ic/GC007v3.pop.r.0001-01-16-00000
--2013-06-14 22:18:02--  https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata/ocn/pop/tx0.1v2/ic/GC007v3.pop.r.0001-01-16-00000
Resolving svn-ccsm-inputdata.cgd.ucar.edu... 128.117.23.213
Connecting to svn-ccsm-inputdata.cgd.ucar.edu|128.117.23.213|:443... failed: Connection timed out.
Retrying.

--2013-06-14 22:21:12--  (try: 2)  https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata/ocn/pop/tx0.1v2/ic/GC007v3.pop.r.0001-01-16-00000
Connecting to svn-ccsm-inputdata.cgd.ucar.edu|128.117.23.213|:443... failed: Connection timed out.
Retrying.

--2013-06-14 22:24:23--  (try: 3)  https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata/ocn/pop/tx0.1v2/ic/GC007v3.pop.r.0001-01-16-00000
Connecting to svn-ccsm-inputdata.cgd.ucar.edu|128.117.23.213|:443... failed: Connection timed out.
Retrying.


 

 

[devarajun@gmail_com]

Hi fischer,Thank you for the reply.If  I download manually, it says download complete but the downloaded file size will be 500mb ( but the file size is 28GB, so it downloaded half, I tried many times but no use). and through wget I tried below is the error: wget https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata/ocn/pop/tx0.1v2/ic/GC007v3.pop.r.0001-01-16-00000
--2013-06-14 22:18:02--  https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata/ocn/pop/tx0.1v2/ic/GC007v3.pop.r.0001-01-16-00000
Resolving svn-ccsm-inputdata.cgd.ucar.edu... 128.117.23.213
Connecting to svn-ccsm-inputdata.cgd.ucar.edu|128.117.23.213|:443... failed: Connection timed out.
Retrying.

--2013-06-14 22:21:12--  (try: 2)  https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata/ocn/pop/tx0.1v2/ic/GC007v3.pop.r.0001-01-16-00000
Connecting to svn-ccsm-inputdata.cgd.ucar.edu|128.117.23.213|:443... failed: Connection timed out.
Retrying.

--2013-06-14 22:24:23--  (try: 3)  https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata/ocn/pop/tx0.1v2/ic/GC007v3.pop.r.0001-01-16-00000
Connecting to svn-ccsm-inputdata.cgd.ucar.edu|128.117.23.213|:443... failed: Connection timed out.
Retrying.

 

[mai]

It appears that your network connection is not good.
I have split the file into 10 pieces and put them in our anonymous ftp area. You can try retrieving them one at a time. The 10 commands you can try are:wget ftp.cgd.ucar.edu:/pub/mai/GC007v3.pop.r.0wget ftp.cgd.ucar.edu:/pub/mai/GC007v3.pop.r.1...wget ftp.cgd.ucar.edu:/pub/mai/GC007v3.pop.r.9 You can also try using the ftp or sftp command. The login is "anonymous" and your email address is the password. Once logged in, you must "cd pub/mai" before getting the 10 files.

 

[mai]

I forgot to mention that, once you have the 10 pieces, you can re-assemble them with the following command:cat GC007v3.pop.r.* > GC007v3.pop.r.0001-01-16-00000 If your downloads always stop at 500mb, this suggests that your system administration has set a limit for the maximum size of downloads at that point.

 

[amatcaos@caos_iisc_ernet_in]

Hello Mai,I too need to download that inputdataset but facing similar problem as Dev.I visited the ftp site you asked to download the dataset ( ftp://ftp.cgd.ucar.edu/pub/mai/). But there exists two sets of files GC007v3.pop.r.00*; type=a or type=i.
Can you please let me know which one to download and what does that type=a type=i means?Many ThanksAngshuman

 

[mai]

There is only one set of files there. You need all 60 files. Once you have them, you can reassemble using the "cat" command posted above. Make sure you have all 60 files and that they are all the same size.

 

[devarajun@gmail_com]

Hi Mai, Thanks a lot for your help. I am able to download all 60 files and cat it to single. Thanks agian.

 

[amatcaos@caos_iisc_ernet_in]

Hello Mai
It worked. ThanksAngshuman