Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

Paleo application of CESM 1.2.1 crashing in the deep convection scheme

rdeitrick

Russell Deitrick
New Member
I am attempting some paleoclimate (neo-proterozoic) simulations with CESM 1.2.1. The eventual plan is to run the model with different rotation rates, however, I run into a model crash even before attempting to change anything with rotation.

I am using what might be an unusual user compset and resolution:
COMPSET=2000_CAM4%WCSC_CLM40%SP_CICE_DOCN%SOM_RTM_SGLC_SWAV
RES=f45_g37

That is, CAM4 with WACCM specified chemistry (waccm_ghg), CLM 4.0 with satellite phenology, and a slab-ocean model (pop_frc.gx3v7.110128.nc), and 4x5 horizontal FV grid, with the displaced pole for the oceans. The particular problem I am trying to tackle doesn't require chemistry but does require the inclusion of the full stratosphere and mesosphere (which is why I am using WACCM specified chemistry). I am hoping to stick to the slab-ocean model for the moment, because I need to do lots of runs and can't really afford to wait on the dynamic ocean to equilibrate.

I am posting here because the crash ultimately comes from the convection scheme, however, please let me know if I should post in the WACCM or Paleoclimate forum instead.

Getting more specific, the crash occurs after about 8 sim years and seems to be coming from the ZM deep convection scheme. The error message in the CESM log is:
*** ZM_CONV: IENTROPY: Failed and about to exit, info follows ****
ZM_CONV: IENTROPY. Details: call#,lchnk,icol= 2 167 1 lat: -86.00 lon: 320.00 P(mb)= 43.19 Tfg(K)= 156.53 qt(g/kg) = 0.14 qst(g/kg) = NaN, s(J/kg) = Inf
(shr_sys_abort) ERROR: **** ZM_CONV IENTROPY: Tmix did not converge ****

I've tried decreasing the time step from 30 minutes to 15 minutes. Interestingly, this caused the crash to occur much sooner (but still the same error in the convection scheme). It also seems that the crash is delayed, but not prevented, by setting 'div24del2flag=42' in user_nl_cam (using 4th order div damping and 2nd order velocity damping sponge layer).

I also see numerous warnings in the CESM log like this:
filew failed, worst i, j, qtmp, q = 1 11
-5.334483749512089E-210 1.103722200454226E-219

I haven't figured out where those warnings are coming from and if they are related. I have seen them in equal abundance in successful runs, however.

I have a few of ideas of things to test (changing resolution, changing land or ocean models, or using components of CAM5 physics instead of CAM4), but I wanted to ask--is there anyone knowledgeable about the deep convection scheme that might see the issue or be able to point me in the right direction?

Thanks in advance,
Russell Deitrick
 
Top