Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

Porting error when executing "./manage_externals/checkout_externals"

Jerry

Cao
New Member
Hi everyone,
I'm trying to port CESM version2 to a CentOS machine following the instructions of the manual.
And I have confirmed to the machine engineer that all required software has been installed.
When executing the script "/manage_externals/checkout_externals", errors occured with
a message: "Checking out externals: clm, ERROR:root:Command '[u'git', u'clone', u'--quiet', u'GitHub - ESCOMP/CTSM: Community Terrestrial Systems Model (includes the Community Land Model of CESM)', u'clm']' returned non-zero exit status 128 ERROR:root:Failed with output: fatal: unable to access 'GitHub - ESCOMP/CTSM: Community Terrestrial Systems Model (includes the Community Land Model of CESM)': Failed connect to github.com:443; Connection timed out".
I guess it may result from the Internet connection because the machine is not connected to the internet. So I wonder if the components
can be downloaded manually.
Besides, it should be noted that I don't have root privileges and could this be the cause of the error?
 

leelay

leelay
New Member
The screen has told you the reason:
Failed connect to github.com:443; Connection timed out.
You can find all the input data you need to drive the program in /YOURCASE/Buildconf. If the SC does not link to the internet, you can download the data you need according to the file in that directory.
You can also deploy a similar version in a machine linked to the net. Compile the program and download the data. Then, transfer the data in the archive you define to the SC.
 

Jerry

Cao
New Member
The screen has told you the reason:
Failed connect to github.com:443; Connection timed out.
You can find all the input data you need to drive the program in /YOURCASE/Buildconf. If the SC does not link to the internet, you can download the data you need according to the file in that directory.
You can also deploy a similar version in a machine linked to the net. Compile the program and download the data. Then, transfer the data in the archive you define to the SC.
Thanks for your reply.
I know input data is needed to drive the data, while the error I encountered seems to appear before preparing input data.
The problem is that the individual model components can't be downloaded automatically. Do you mean the components, such the sea ice model,
can also be downloaded from other machines with internet and then transfer to the target machine?
 

leelay

leelay
New Member
Thanks for your reply.
I know input data is needed to drive the data, while the error I encountered seems to appear before preparing input data.
The problem is that the individual model components can't be downloaded automatically. Do you mean the components, such the sea ice model,
can also be downloaded from other machines with internet and then transfer to the target machine?
Yes. For example, using a machine linked to the AMERICAN internet. Download source file of CESM and run:
/manage_externals/checkout_externals
If you are in come countries not linked to the AMERICAN internet or the DNS is banned, you will never download successfully. You need some tools.
 

yanhp2009

Huiping Yan
New Member
Try to use ssh to clone or download the CESM code from github.com, and you can follow the instruction Using SSH over the HTTPS port - GitHub Docs
There are three steps:
1. add git account to the ssh config file which locate at the directory .ssh/config
Host github.com
Hostname ssh.github.com
Port 443

User git
2. change all https port to ssh in the file Externals.cfg. (You may have to change the Externals_CAM.cfg at components/cam.)
for example:
repo_url = ssh://github.com/ESCOMP/CAM

3. run the command before you execute ./manage_externals/checkout_externals
ssh -T git@github.com
 

Jerry

Cao
New Member
Yes. For example, using a machine linked to the AMERICAN internet. Download source file of CESM and run:
/manage_externals/checkout_externals
If you are in come countries not linked to the AMERICAN internet or the DNS is banned, you will never download successfully. You need some tools.
I will have a try. Thanks!
 

Jerry

Cao
New Member
Try to use ssh to clone or download the CESM code from github.com, and you can follow the instruction Using SSH over the HTTPS port - GitHub Docs
There are three steps:
1. add git account to the ssh config file which locate at the directory .ssh/config
Host github.com
Hostname ssh.github.com
Port 443

User git
2. change all https port to ssh in the file Externals.cfg. (You may have to change the Externals_CAM.cfg at components/cam.)
for example:
repo_url = ssh://github.com/ESCOMP/CAM

3. run the command before you execute ./manage_externals/checkout_externals
ssh -T git@github.com
Thanks for your reply. I tried to use your method, but it seems that the root privilege is required to complete the step one.
Unfortunately, I am not a rooter of the machine.
Maybe I have to ask for help of the machine administrator.
 

yanhp2009

Huiping Yan
New Member
Thanks for your reply. I tried to use your method, but it seems that the root privilege is required to complete the step one.
Unfortunately, I am not a rooter of the machine.
Maybe I have to ask for help of the machine administrator.
I don't know why root privilege is needed to modify the ssh config file on your machine. Since the ssh config file should be located at your own home directory, if there is no such file, you could create one.

Also, you could try this method on your own PC/laptop with Linux system. About one week ago, using this method I downloaded CESM2.3-beta code on my Macbook, and then uploaded it to the supercomputer. So I think this should be done on your own computer. Hope this could help you.
 

ykp990521

ykp990521
Member
Thanks for your reply. I tried to use your method, but it seems that the root privilege is required to complete the step one.
Unfortunately, I am not a rooter of the machine.
Maybe I have to ask for help of the machine administrator.
Hi, I have the downloaded full package of CESM2.1.3 and 2.2, I could send it to you, we both have no root.
 

Jerry

Cao
New Member
I don't know why root privilege is needed to modify the ssh config file on your machine. Since the ssh config file should be located at your own home directory, if there is no such file, you could create one.

Also, you could try this method on your own PC/laptop with Linux system. About one week ago, using this method I downloaded CESM2.3-beta code on my Macbook, and then uploaded it to the supercomputer. So I think this should be done on your own computer. Hope this could help you.
When I tried to execute "ssh -T git@github.com", error occured with message "Bad owner or permissions on /home/username/.ssh/config".
I tried to find a solution to this error on the Internet and someone told me that I should execute the script of "sudo chmod 600 config".
That means the root privilege is needed. And the operation above is carried in a supercompute. Now I will have a try in my PC.
Thanks!
 
Top