WACCM with added isotopologues crashes after several model days

[ebuzan@odu_edu]

I have put together a model run with WACCM to study the isotopes of CO and CO2. I'm using CESM 1.0.5 with compset FW and the addition of carbon-13 isotopologues of CH4, CH3O2, CH3OOH, CH2O, CO, and CO2. I believe I've modified all the necessary files: the mechanism file, IC data, LBC data, and photolysis cross sections. When I run the model it throws a bunch of error messages as follows, then aborts:imp_sol: Time step   1.8000000000000E+03 failed to converge @ (lchnk,lev,col,nstep) =     17     1     1     0
imp_sol: Time step   9.0000000000000E+02 failed to converge @ (lchnk,lev,col,nstep) =     17     1     1     0
imp_sol: Time step   4.5000000000000E+02 failed to converge @ (lchnk,lev,col,nstep) =     17     1     1     0
imp_sol: Time step   2.2500000000000E+02 failed to converge @ (lchnk,lev,col,nstep) =     17     1     1     0
imp_sol: Time step   1.1250000000000E+02 failed to converge @ (lchnk,lev,col,nstep) =     17     1     1     0
imp_sol: Time step   1.1250000000000E+01 failed to converge @ (lchnk,lev,col,nstep) =     17     1     1     0
imp_sol: Failed to converge @ (lchnk,lev,col,nstep,dt,time) =     17     1     1     0  1.1250000000000E+01  1.1250000000000E+01
O         9.911E-01
O2_1D     4.436E-03
OH        1.479E-03
NO3       0.000E+00
HO2       1.359E-03
CLO       0.000E+00
HOCL      0.000E+00
HOBR      0.000E+00
BRONO2    0.000E+00I tried using the Rosenbrock solver for all the implicitly solved species and the model would run for several model days. After a few days I begin to see the following warning multiple times:DADADJ: Convergence criterion doubled to EPS=.4000E-04 for
        DRY CONVECTIVE ADJUSTMENT at Lat,Lon=   34   43and the model aborts after 7 days.

 

[mmills]

Eric,My apologies friends for missing this post. It is not uncommon for instabilities to arise in the implicit solver after chemistry has been changed. The first step in trying to get around this is to increase nsplit, which sets the number of dynamics timesteps per physics timestep. If you continue to have problems, you can keep increasing nsplit to several hundred. The default is 8, and we often double it until we get stability. After you have run a few days or weeks this way, you should decrease nsplit again, as it can affect cloud physics. 

 

[ebuzan@odu_edu]

I've spent some time tweaking nsplit and haven't suecceeded in making a stable run. I increased it up to 512 and the model makes it about a month before crashing. Setting nsplit to a high value for a couple weeks then turning it down also didn't help. I've attached my preprocessor file in case I made any mistakes there, but since the model compiles and runs for a short time I suspect everything with that is fine.

 

[santos]

It might be helpful to see your namelist as well (or user_nl_cam), in case there are any red flags there.I don't believe it is the ultimate cause of the problem, but addressing this may improve stability:https://bb.cgd.ucar.edu/cesm-104-106-cesm-11z-and-cesm-12z-molecular-diffusion-does-not-affect-heattemperatureAs I recall, the file for 1.0.6 should work for 1.0.5 as well.In fact, you may want to consider switching to CESM 1.0.6 altogether. Most of the fixes in 1.0.6 are for specified chemistry cases, but I seem to recall one or two tweaks for stability of WACCM in general.

 

[ebuzan@odu_edu]

Here is my usr_nl_cam. I recently upgraded to 1.0.6 as well and that hasn't had any effect on stability.

 

[santos]

Hmm. I'm stumped, I'm afraid, though I admit that I'm not very good with chemistry issues.