The error named "negative moist layer thickness. timestep or remap time too large" in CESM2.2.0

[zhendf]

Hi,everyone, when I tried to run the QPC6 case with the unsupported resolution named "ne0CONUSne30x8_ne0CONUSne30x8_mg17",the error happened after I submitted it, I try to find any method to solve this problem but only found in "https://bb.cgd.ucar.edu/cesm/thread...ompiler-13-0-0-at-o2-optimization-level.3135/",we have the same error named "timestep or remap time too large",someone said to change the "O2"to"O1" in the config_compiler.xml, I had try to do the same things, but the error happened again

I build this case with DEBUG=TRUE, but could not settle that error, the followings are my atm.log and the cesm.log
my intercompiler version is 18.0.0, is this version too old to run the this case with VR grids?

I would appreciate it very much if someone can help me solve this problem！！！

 

[Lumoss]

Hello, I have the same problem as you. Have you solved it?

 

[peverley]

Hello, I have the same problem as you. Have you solved it?

In the atm.log* file, the warning messages indicate your timesteps are unstable:

* dt_dyn (time-stepping dycore ; u,v,T,dM) < 37.85s 37.50s
* dt_dyn_vis (hyperviscosity) ; u,v,T,dM) < 47.09s 18.75s
* dt_tracer_se (time-stepping tracers ; q ) < 32.84s 37.50s
WARNING: dt_tracer_se theoretically unstable
* dt_tracer_vis (hyperviscosity tracers; q ) < 117.72s 37.50s
* dt_remap (vertical remap dt) 112.50
* dt (del2 sponge ; u,v,T,dM) < 5.26s 9.38s
WARNING: theoretically unstable in sponge; increase se_hypervis_subcycle_sponge
* dt (del2 sponge ; u,v,T,dM) < 17.96s 9.38s
* dt (del2 sponge ; u,v,T,dM) < 60.45s 9.38s

One thing to try would be to increment your se_rsplit value by one and set se_hypervis_subcycle=1 in the namelist. You may also want to play around with se_nsplit if it's not a horizontal instability.

Hope that helps.
Courtney

 

[Lumoss]

In the atm.log* file, the warning messages indicate your timesteps are unstable:

One thing to try would be to increment your se_rsplit value by one and set se_hypervis_subcycle=1 in the namelist. You may also want to play around with se_nsplit if it's not a horizontal instability.

Hope that helps.
Courtney

Hi Peverley,

Thank you for your reply. I tried setting se_rsplit=4 and se_hypervis_subcycle=1, but the results didn't seem to change. Set se_nsplit=3 again and the result is still the same.

Perhaps you have any other good suggestions? Thank you so much!

All the best,
Lumos.

 

[peverley]

Hi Lumos,

Can you post your atm.log* file for this latest run? Does it say anything about your timesteps?

Courtney

 

[Lumoss]

Hi Lumos,

Can you post your atm.log* file for this latest run? Does it say anything about your timesteps?

Courtney

Hi Peverley,

Thank you for your reply. Enclosed is my atm.log and cesm.log. I don't seem to notice any difference.

All the best,
Lumos

 

[peverley]

Hi again,

The logging about the dynamical core timesteps now looks looks like:

* dt_dyn (time-stepping dycore ; u,v,T,dM) < 37.85s 28.12s
* dt_dyn_vis (hyperviscosity) ; u,v,T,dM) < 46.89s 28.12s
* dt_tracer_se (time-stepping tracers ; q ) < 32.84s 28.12s
* dt_tracer_vis (hyperviscosity tracers; q ) < 210.60s 28.12s
* dt_remap (vertical remap dt) 112.50
* dt (del2 sponge ; u,v,T,dM) < 8.41s 14.06s
WARNING: theoretically unstable in sponge; increase se_hypervis_subcycle_sponge
(this WARNING can sometimes be ignored in level 1)
* dt (del2 sponge ; u,v,T,dM) < 28.74s 14.06s
* dt (del2 sponge ; u,v,T,dM) < 96.72s 14.06s

Which indicates that the changes you made did make some difference to the time-stepping tracers! I'd recommend doing what the log message says and increasing se_hypervis_subcycle_sponge. You can see what it is by default in your run/atm_in file and increase it from there (in your user_nl_cam file).

Courtney

 

[Lumoss]

Hi again,

The logging about the dynamical core timesteps now looks looks like:



Which indicates that the changes you made did make some difference to the time-stepping tracers! I'd recommend doing what the log message says and increasing se_hypervis_subcycle_sponge. You can see what it is by default in your run/atm_in file and increase it from there (in your user_nl_cam file).

Courtney

Thank you very much for your reply. At the suggestion of jedwards, I decided to update the version of my compiler and some necessary libraries first, which may take some time. After that, I will try your suggestion and hope to solve the problem. Thanks again!

 

[Lumoss]

Hi again,

The logging about the dynamical core timesteps now looks looks like:



Which indicates that the changes you made did make some difference to the time-stepping tracers! I'd recommend doing what the log message says and increasing se_hypervis_subcycle_sponge. You can see what it is by default in your run/atm_in file and increase it from there (in your user_nl_cam file).

Courtney

Hi,

I added se_hypervis_subcycle_sponge=6 as you suggested. It still doesn't seem to be working. The log file is enclosed. Perhaps you could provide me with some other guidance. Thanks again!

Lumos

 

[peverley]

Hi again.

Taking a look at your atm.log.* file now, you have:

* dt_dyn (time-stepping dycore ; u,v,T,dM) < 37.85s 37.50s
* dt_dyn_vis (hyperviscosity) ; u,v,T,dM) < 46.89s 12.50s
* dt_tracer_se (time-stepping tracers ; q ) < 32.84s 37.50s
WARNING: dt_tracer_se theoretically unstable
* dt_tracer_vis (hyperviscosity tracers; q ) < 210.60s 37.50s
* dt_remap (vertical remap dt) 112.50
* dt (del2 sponge ; u,v,T,dM) < 8.41s 6.25s
* dt (del2 sponge ; u,v,T,dM) < 28.74s 6.25s
* dt (del2 sponge ; u,v,T,dM) < 96.72s 6.25s

The WARNING is now the same as in your initial post. Can you try incrementing se_rsplit by one again?

Courtney

 

[Lumoss]

Hi

I actually tried adjusting se_rsplit, and the warning disappeared, but I ended up reporting the same error.

Lumos

 

[peverley]

Bummer!

Is the error occurring on the same timestep? Can you post your atm.log* and cesm.log* files again?

Court

 

[Lumoss]

Of course, here are my atm.log* and cesm.log* files.

Lumos