F2000climo bug on Derecho

[skramer]

Hello,

I am having trouble building a new case on Derecho after successfully using the same method prior on Cheyenne:

Command: qcmd -A UCUB##### -1 walltime=2:00:00 -- ./case.build
Error (Have an image of the entire error but too large to attach):
build-namelist failed rc=255
out =
err=ERROR : CLM build-namelist :: CLMBuildNamelist :: process_namelist_comandline_infile() : Invalid namelist variable in '-infile' ....

I am using the F2000climo compset and saw this bug posted in December of 2023 which may be the source of my issue:

Relevant Bug: Bugs and Updates - MUSICA - wiki.ucar.edu
FHIST and F2000climo compsets not defined properly
Date of report: December 2023
Title: Failure to build FHIST and F2000climo compsets
Issue: For CESM2.2 on Derecho, the default FHIST and F2000climo compsets are not defined properly.
Fix: Try using:
--compset HIST_CAM60_CLM50%SP_CICE%PRES_DOCN%DOM_MOSART_SGLC_SWAV_SIAC_SESP

I am not sure if this has been fixed since, but I have not seen any other relevant updates. I am afraid to try using the "fix" listed above because there are quite a few discrepancies between the "fix" compset and the compset I previously used, and I would like my new model experiment to be as similar as possible.

Previous reference case/version on Cheyenne: (CESM 2.0.0) F2000climo

2000_CAM60_CLM50%BGC-CROP_CICE%PRES_DOCN%DOM_MOSART_CISM2%NOEVOLVE_SWAV

New reference case/version on Derecho: (CESM 2.2.0) F2000climo
2000_CAM60_CLM50%SP_CICE%PRES_DOCN%DOM_MOSART_CISM2%NOEVOLVE_SWAV

I did follow the instructions found here: Run CESM with Chemistry on Derecho - CAM-Chem - wiki.ucar.edu to download CESM 2.2.2 and compsets found at the website here

CESM2.2.0 Component Sets Definitions​

Component Sets Definitions for CESM2.2.0

docs.cesm.ucar.edu
are only listed through CESM 2.2.0, not CESM 2.2.2.


Can this problem be solved by downloading a different/earlier version of CESM that is still supported on Derecho (i.e. CESM 2.1.3) where this bug may not be a problem, or do I need to use the "fix" listed in the bug update? If I use the "fix" compset how would this affect my experiment (specifically I need my forcing held constant at the 2000-year level)? Is my error something entirely unrelated to this bug, and this bug has already been fixed?

Any advice is appreciated,

Sydney

 

[jedwards]

Hi Sydney,

Since you were using cesm2.0.0 on cheyenne I would suggest using cesm2.1.5 on derecho unless there is a compelling reason to update to cesm2.2.2?
It would help if you provide the full set of instructions to reproduce the case that you want to run. (Use the cheyenne instructions if necessary)
A pointer to your case directory might be adequate.

 

[skramer]

Hi Jim,

Thank you for the quick response. The case is here: /glade/u/home/skramer/casefiles/2000_docn_plio_ctrl . I updated to CESM2.2.2 because I was not sure what previous versions of CESM were being supported on Derecho and instructions found here: Run CESM with Chemistry on Derecho - CAM-Chem - wiki.ucar.edu instructed the porting of CESM2.2.2. I was also unsure if this bug was the origin of the error and if using a previous version of CESM2 would solve this issue.

My previously successful cases on Cheyenne are also located in my casefiles here: /glade/u/home/skramer/casefiles/
and I placed cesm2.2.2 here: /glade/u/home/skramer/derecho_cases/cesm2.2.2.

Previous case Cheyenne
./create_newcase -case /glade/u/home/skramer/casefiles/2000_docn_Shift -compset=2000_CAM60_CLM50%BGC-CROP_CICE%PRES_DOCN%DOM_MOSART_CISM2%NOEVOLVE_SWAV -res=f09_f09_mg17 -mach=cheyenne -project=UCUB0130 --run-unsupported

New case Derecho
./create_newcase -case /glade/u/home/skramer/casefiles/2000_docn_plio_ctrl -compset=2000_CAM60_CLM50%SP_CICE%PRES_DOCN%DOM_MOSART_CISM2%NOEVOLVE_SWAV -res=f09_f09_mg17 -mach=derecho -project=UCUB0114 --run-unsupported

After trying to ./case.build the new case in Derecho, I ran into the error. Please let me know if there is any other information I can provide that could be helpful in determining a solution.

Thank you again so much for your time,

Sydney

 

[oleson]

I happened to notice that your first post reported an error with the CLM namelist. I see this in your user_nl_clm in your case directory: phys_loadbalance = 1. I think that is a CAM namelist item, not a CLM one.

 

[skramer]

Hello Keith,

I placed the phys_loadbalance = 1 due to the instruction found here: Run CESM with Chemistry on Derecho - CAM-Chem - wiki.ucar.edu where the note states:

The pe-layout of existing compsets in the new code base of CESM2 for derecho (cesm2.2.2) has not been adjusted to the new computer, and running out of the box can lead to large differences in computer costs compared to running on cheyenne.
New PE-layouts are still being developed. We are working on updating these compsets. One way to increase performance is to use a namelist setting. To improve improve the performance of derecho, you can add the following to your user_nl_cam file.

phys_loadbalance = 1 After you pointed out it may have been entered into the wrong place I double checked and I had incorrectly placed phys_loadbalance = 1 into user_nl_clm instead of the user_nl_cam file. I have fixed this issue and will attempt to build my case now, but am still worried I may run into an error with the known big issue with the F2000climo compset. Thank you so much, Sydney