Availability of CESM 2.1.3 Ensemble Consistency Test Summary Files

[johnsonb]

Hi CGD BB,

I'm in the process of porting CESM2.1.3 to Shaheen II, which is a Cray XC40 at King Abdullah University of Science and Technology. I've attached the output of describe_version to this post as version_info.txt.

I'm attempting to run the ensemble consistency tests and am at the step in the README:

$CIMEROOT/tools/statistical_ensemble_test/README

that states, "Before the test can be run, a summary file is needed of the ensemble runs to which the comparison will be made. Ensemble summary files (NetCDF) for existing tags for CAM-ECT, UF-CAM-ECT, and POP-ECT that were created by CSEG are located (respectively) in the CESM input data directories."

I've looked through the directories on the CGD ftp server:

ftp://ftp.cgd.ucar.edu/cesm/inputdata/validation/

and within the analogous directories on GLADE in this directory:

/glade/p/cesm/cseg/inputdata/validation

but I am unable to find summary files for CESM2.1.3. Here is a quick description of the available summary files within the above directories:

• ensembles has summary files for releases up to cesm2_0_beta05
• pop_ensembles has summary files for releases up to cesm2.1.2
• uf_ensembles has summary files for releases up to cesm2.1.1 and then skips to cesm2_2_beta04

In the absence of summary files for CESM2.1.3, the README states, "If none of our ensembles are suitable for your needs, then you may create your own ensemble (and summary file) using the following instructions: (1) To create a new ensemble, use the ensemble.py script in this directory. This script creates and compiles a case, then creates clones of the original case, where the initial temperature perturbation is slightly modified for each ensemble member. At this time, cime includes functionality to create ensembles for CAM-ECT, UF-CAM-ECT, and POP-ECT."

Earlier in the README, the overview of the test states, "The ECT process involves comparing runs generated with the new scenario to an ensemble built on a trusted machine (currently cheyenne)."

Thus, in order to successfully run the ECT test, does the summary file need to be generated on cheyenne first?

Should I check out an older release of CESM and attempt to port it instead? Or, since CESM2.1.3 is the currently supported release, is there a plan to generate the CESM2.1.3 summary files?

I can help with generating the summary files, but I don't have an allocation to run the ensembles on cheyenne.

Thank you,
Ben Johnson / johnsonb

 

[fischer]

Hi Ben,

There is no significant difference between cesm2.1.1, 2.1.2, and 2.1.3 for the UF-ECT-CAM. So the cesm2.1.1 is valid for cesm2.1.3. The POP-ECT did
have a change for cesm2.1.2.

But a better way to validate your port is to use our verification website.

CESM2 Ensemble Verification

Online CESM2 ensemble consistency test using PyCECT

www.cesm.ucar.edu

It will automatically select the correct summary file for you. Here's a link to the instructions.

Python Tools | Community Earth System Model

www.cesm.ucar.edu

Chris

 

[johnsonb]

Hi Chris,

Thanks for the prompt response! I'll use the website to upload our output.

Would it be a good idea to rewrite the text in the README?

In its current wording it was unclear to me that use of the website negates the need to obtain a summary file. See line 31 of bd7e7fd, "Before the test can be run, a summary file is needed of the ensemble runs to which the comparison will be made. Ensemble summary files (NetCDF) for existing tags for CAM-ECT, UF-CAM-ECT, and POP-ECT that were created by CSEG are located (respectively) in the CESM input data directories."

Thanks again and hope this helps,
Ben Johnson / johnsonb

 

[johnsonb]

Hi again,

I was able to get the CAM-ECT, UF-CAM-ECT and POP-ECT runs to complete. I uploaded the history files to the verification website and the UF-CAM-ECT and POP-ECT tests passed without issue.

For the CAM-ECT, all three runs failed all 50 PCA tests (CAM-ECT_result.txt, attached).

The CESM python tools website says to see the ensemble consistency test section of the DiscussCESM forum but the link is broken.

I've read through README_pyCECT.rst, the relevant section of the CIME documentation, and Baker et al. (2015) but was unable to determine how to diagnose the issue that brought about such a resounding failure of the test.

Is there a way to get more comprehensive output from the verification website? Otherwise, is there anywhere else I can look to get information on how to progress from here?

Thank you,
Ben Johnson / johnsonb

 

[fairyeszhuo]

Any solution to this problem? I get the same "failed" results for the CAM-ECT test. I will try the other two tests...

 

[fischer]

CAM-ECT is only for CESM2.0. The web interface shouldn't be allowing CAM-ECT verifications with CESM2.1.x,
and should give an error message. Which looks like that's not working.