external download from dev branch failure

[henry_leroy_miller@noaa_gov]

Hello,we have recently been granted access to the developer branch of the CESM, in order that we could access a particular version of the CESM.  The request was for CESM only, which the form states would provide read-only access of the HOMME system.  However, when we downloaded the cesm code (https://svn-ccsm-models.cgd.ucar.edu/cesm1/exp/branch_tags/waccm5.5_tags/cesm1_5_beta02_waccm06), all of the externals were checked out except one from the repository 'https://svn-homme-model.cgd.ucar.edu/branch_tags/DynGridHistory_tags/DynGridHistory04_4759/src/share'.I tried accessing that repository directly to do a listing of contents, and connection was denied.I'm not even sure what should have been downloaded, and am looking for advice about how to proceed from here.The syntax of the error from the initial checkout of the CESM was:Error validating server certificate for 'https://svn-homme-model.cgd.ucar.edu:443': - The certificate is not issued by a trusted authority. Use the   fingerprint to validate the certificate manually!Certificate information: - Hostname: *.cgd.ucar.edu - Valid: from Dec 28 00:00:00 2016 GMT until Dec 28 23:59:59 2019 GMT - Issuer: InCommon RSA Server CA, InCommon, Internet2, Ann Arbor, MI, US - Fingerprint: 39:F1:46:85:A2:2A:86:E5:D4:B8:8B:0B:C0:F8:28:D6:71:74:1D:22(R)eject, accept (t)emporarily or accept (p)ermanently? psvn: warning: W170013: Unable to connect to a repository at URL 'https://svn-homme-model.cgd.ucar.edu/branch_tags/DynGridHistory_tags/DynGridHistory04_4759/src/share' thanks for your anticipated assistance,Roy Miller

 

[jedwards]

This morning we replaced the hardware that provides our subversion service which has created a host of problems.   We are working to resolve them as quickly as possible, please bear with us.   

 

[jedwards]

This problem should now be resolved, please try again.

 

[jedwards]

Thanks for letting us know. we will continue to work to try to address this problem as quickly as possible.

 

[mmoore]

The new hardware and IP were implmented about 6:50am MST on Wed 7 Feb 2018. The report of
initial problems beginning on 5 Feb 2018 indicates there may be two problems...one prior to the
HW upgrade, one after.For debugging puposes, try the following: . Accept the cert permanently, not temporarily.
. Answer 'yes' to the 'save password prompt'. Yes, it's bad form, but otherwise you'll be entering it
30+ times due to the way the repositories are interwoven.
. The cert theory can be test w/ by adding --trust-server-cert to the 'svn co' command. Mark
--0-

 

[henry_leroy_miller@noaa_gov]

sadly, access is now worse than initially, on 5 Feb, approximatel 1400 hours MST.  This was the date and time at which the results reported in this ticket where observed.  It appears that the account (guestuser) to access public available branches is working, while the credentials issued to allow us access to the development branches are no longer working at all!  Further, I cannot even list the public available branch from the NOAA HPC systems Jet and Theia.  This last most likely due to changes of the host or host IP Address during these hardware updates.  The NOAA HPC systems limit access by protocol and IP Address, therefore my conclusion. It is possible this is also a certificate issue....here below are some snippets of attempts from my office workstation, from which I've accessed the public releases in the past, and have the guestuser credentials cached in my .subversion directory:• this works:Thu Feb 08 12:11:32 beowulf reforecast-1> svn list https://svn-ccsm-models.cgd.ucar.edu/cesm1/release_tags/Error validating server certificate for 'https://svn-ccsm-models.cgd.ucar.edu:443': - The certificate is not issued by a trusted authority. Use the   fingerprint to validate the certificate manually!Certificate information: - Hostname: *.cgd.ucar.edu - Valid: from Dec 28 00:00:00 2016 GMT until Dec 28 23:59:59 2019 GMT - Issuer: InCommon RSA Server CA, InCommon, Internet2, Ann Arbor, MI, US - Fingerprint: 39:F1:46:85:A2:2A:86:E5:D4:B8:8B:0B:C0:F8:28:D6:71:74:1D:22(R)eject, accept (t)emporarily or accept (p)ermanently? tcesm1_0_6/cesm1_1_x_LENS_tags/cesm1_2_1/cesm1_2_2/cesm1_2_2_1/cesm1_2_2_CAMChem/ • this doesn't work - when I try listing the development branch, with the command that worked on 5 Feb, I see different results:Thu Feb 08 12:12:30 beowulf cesm> svn list --username amy.butler@noaa.gov https://svn-ccsm-models.cgd.ucar.edu/cesm1/exp/branch_tags/waccm5.5_tagsError validating server certificate for 'https://svn-ccsm-models.cgd.ucar.edu:443': - The certificate is not issued by a trusted authority. Use the   fingerprint to validate the certificate manually!Certificate information: - Hostname: *.cgd.ucar.edu - Valid: from Dec 28 00:00:00 2016 GMT until Dec 28 23:59:59 2019 GMT - Issuer: InCommon RSA Server CA, InCommon, Internet2, Ann Arbor, MI, US - Fingerprint: 39:F1:46:85:A2:2A:86:E5:D4:B8:8B:0B:C0:F8:28:D6:71:74:1D:22(R)eject, accept (t)emporarily or accept (p)ermanently? tAuthentication realm: ccsm:modelsPassword for 'name@institution': ******** Authentication realm: ccsm:modelsUsername: name@institutionPassword for 'name@institution': ******** Authentication realm: ccsm:modelsUsername: name@institutionPassword for 'name@institution': ******** svn: E170013: Unable to connect to a repository at URL 'https://svn-ccsm-models.cgd.ucar.edu/cesm1/exp/branch_tags/waccm5.5_tags'svn: E215004: No more credentials or we tried too many times.Authentication failedThu Feb 08 12:15:13 beowulf cesm>  so, at this point, we are unable to access the development branch at all.thanks. 2018.02.09 - Note: redacted user account name.  User account info available if needed, by private communication.

 

[henry_leroy_miller@noaa_gov]

Thanks Mark,I'll pick this up Monday, when I return to work.Roy

 

[henry_leroy_miller@noaa_gov]

Hi, it took me longer to get back to these issues.  The good news is that one of the three issues has been resolved, that is that access to the new subversion system has now been re-established from the NOAA HPC Theia system.  It is likely that the Jet system is also again able to access the subversion repository, the check of which will have to wait until tomorrow as Jet is offline for maintenance today. The initial issue, from prior to the subversion hardware upgrade, was that a checkout of an external from the HOMME system failed on my initial checkout.  Testing and debugging this has been hampered by the second remaining issue, which is that the developer authentication credentials appear to have a problem.   That account, which authenticated and allowed the initial checkout to proceed, no longer seems to authenticate.  I suspect some flag has been set, perhaps for too many failed connection attempts, or some such.  That account has not been able to connect since the subversion system upgrade.  To test that account, I've moved the existing ".subversion" directory in my home directory to ".subversion_old", in order to get the guestuser credentials and any other cruff out of the way.  On 15 Feb, an attempt from my office workstation, in which I permanently saved the cert, still failed, after 3 times of entering account user and password. The system returns:svn: E170013: Unable to connect to a repository at URL 'https://svn-ccsm-models.cgd.ucar.edu/cesm1/exp/branch_tags/waccm5.5_tags'svn: E215004: No more credentials or we tried too many times.Authentication failed I suggest that I apply for a developer account with my email address, and then would have the possibility to discover whether the current account has an issue, or both accounts have an issue.

 

[henry_leroy_miller@noaa_gov]

okay, I've now received my own user account to access the development branch of the CESM.  Using that account (henry.leroy.miller@noaa.gov), I was able to checkout the developers version of the cesm that I need to port to the NOAA RDHPC system.  However, the same external for which I began this thread failed to check out.  So, I'd like to return to just focusing on that external checkout failure, and we'll pursue issues with Amy's account independently.My procedure was:

1. remove the existing $HOME/.subversion directory
2. enter the svn co command for the tag url we were provided
3. immediately accept the initial certificate - permanently
4. enter my username (henry.leroy.miller@noaa.gov) and password
5. answer "yes" to store password unencrypted
6. watch the successful checkout of items, until about 2/3 of the way through, when the certificate error occurred for "https://svn-homme-model.cgd.ucar.edu:443"
7. accept that certificate permanently
8. enter my username (henry.leroy.miller@noaa.gov) and password
9. svn "Unable to connect" warning appeared
10. watch rest of the items checkout successfully

I've attached the full transcript of the checkout in file "jet_cesm-1.3.x.txt", and here below is the shorter section with the error, along with the surrounding line or two: A    cesm1_5_beta02_waccm06/components/cam/src/physics/icarus-scops/test_congvec.kshChecked out external at revision 88640. Error validating server certificate for 'https://svn-homme-model.cgd.ucar.edu:443': - The certificate is not issued by a trusted authority. Use the   fingerprint to validate the certificate manually!Certificate information: - Hostname: *.cgd.ucar.edu - Valid: from Dec 28 00:00:00 2016 GMT until Dec 28 23:59:59 2019 GMT - Issuer: InCommon RSA Server CA, InCommon, Internet2, Ann Arbor, MI, US - Fingerprint: 39:F1:46:85:A2:2A:86:E5:D4:B8:8B:0B:C0:F8:28:D6:71:74:1D:22(R)eject, accept (t)emporarily or accept (p)ermanently? pAuthentication realm: homme:modelPassword for 'Henry.LeRoy.Miller': Authentication realm: homme:modelUsername: henry.leroy.miller@noaa.govPassword for 'henry.leroy.miller@noaa.gov': ********svn: warning: W170013: Unable to connect to a repository at URL 'https://svn-homme-model.cgd.ucar.edu/branch_tags/DynGridHistory_tags/DynGridHistory04_4759/src/share' Fetching external item into 'cesm1_5_beta02_waccm06/components/cam/src/physics/carma/base':A    cesm1_5_beta02_waccm06/components/cam/src/physics/carma/base/carma_globaer.hSo, I'm now wondering if the username and password this external HOMME item needs is the GIT credentials, rather than the CESM subversion credentials.thanks, Roy

 

[henry_leroy_miller@noaa_gov]

Well, as of today, this situation has been resolved.  In the process of performing a complete new checkout of the tag"https://svn-ccsm-models.cgd.ucar.edu/cesm1/exp/branch_tags/waccm5.5_tags/cesm1_5_beta02_waccm06",this time when I entered my credentials in mid-checkout for the homme sources, they were accepted and the external module was downloaded/checked out.I've tried new checkouts a couple times a week since this began, and previously never had success.  The external that was downloaded from the homme realm was:Fetching external item into 'cesm1_5_beta02_waccm06/components/cam/src/dynamics/se/share'I've attached the section of the checkout in a text file, containing the credential entry, and the external that never downloaded previously.  Maybe this will fix some of my other issues with attempting this port.I consider this matter resolved, and this ticket closedthanks, Roy