Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

The error named "negative moist layer thickness. timestep or remap time too large" in CESM2.2.0

zhendf

fanfan zheng
New Member
Hi,everyone, when I tried to run the QPC6 case with the unsupported resolution named "ne0CONUSne30x8_ne0CONUSne30x8_mg17",the error happened after I submitted it, I try to find any method to solve this problem but only found in "https://bb.cgd.ucar.edu/cesm/thread...ompiler-13-0-0-at-o2-optimization-level.3135/",we have the same error named "timestep or remap time too large",someone said to change the "O2"to"O1" in the config_compiler.xml, I had try to do the same things, but the error happened again

I build this case with DEBUG=TRUE, but could not settle that error, the followings are my atm.log and the cesm.log
my intercompiler version is 18.0.0, is this version too old to run the this case with VR grids?

I would appreciate it very much if someone can help me solve this problem!!!
 

Attachments

  • log.zip
    93 KB · Views: 5

peverley

Courtney Peverley
Moderator
Hello, I have the same problem as you. Have you solved it?
In the atm.log* file, the warning messages indicate your timesteps are unstable:
* dt_dyn (time-stepping dycore ; u,v,T,dM) < 37.85s 37.50s
* dt_dyn_vis (hyperviscosity) ; u,v,T,dM) < 47.09s 18.75s
* dt_tracer_se (time-stepping tracers ; q ) < 32.84s 37.50s
WARNING: dt_tracer_se theoretically unstable
* dt_tracer_vis (hyperviscosity tracers; q ) < 117.72s 37.50s
* dt_remap (vertical remap dt) 112.50
* dt (del2 sponge ; u,v,T,dM) < 5.26s 9.38s
WARNING: theoretically unstable in sponge; increase se_hypervis_subcycle_sponge
* dt (del2 sponge ; u,v,T,dM) < 17.96s 9.38s
* dt (del2 sponge ; u,v,T,dM) < 60.45s 9.38s
One thing to try would be to increment your se_rsplit value by one and set se_hypervis_subcycle=1 in the namelist. You may also want to play around with se_nsplit if it's not a horizontal instability.

Hope that helps.
Courtney
 

Lumoss

Member
In the atm.log* file, the warning messages indicate your timesteps are unstable:

One thing to try would be to increment your se_rsplit value by one and set se_hypervis_subcycle=1 in the namelist. You may also want to play around with se_nsplit if it's not a horizontal instability.

Hope that helps.
Courtney
Hi Peverley,

Thank you for your reply. I tried setting se_rsplit=4 and se_hypervis_subcycle=1, but the results didn't seem to change. Set se_nsplit=3 again and the result is still the same.

Perhaps you have any other good suggestions? Thank you so much!

All the best,
Lumos.
 

peverley

Courtney Peverley
Moderator
Hi Lumos,

Can you post your atm.log* file for this latest run? Does it say anything about your timesteps?

Courtney
 

Lumoss

Member
Hi Lumos,

Can you post your atm.log* file for this latest run? Does it say anything about your timesteps?

Courtney
Hi Peverley,

Thank you for your reply. Enclosed is my atm.log and cesm.log. I don't seem to notice any difference.

All the best,
Lumos
 

Attachments

  • atm.log.796044.mgr.231213-092627.txt
    396.4 KB · Views: 4
  • cesm.log.796044.mgr.231213-092627.txt
    34.9 KB · Views: 5

peverley

Courtney Peverley
Moderator
Hi again,

The logging about the dynamical core timesteps now looks looks like:

* dt_dyn (time-stepping dycore ; u,v,T,dM) < 37.85s 28.12s
* dt_dyn_vis (hyperviscosity) ; u,v,T,dM) < 46.89s 28.12s
* dt_tracer_se (time-stepping tracers ; q ) < 32.84s 28.12s
* dt_tracer_vis (hyperviscosity tracers; q ) < 210.60s 28.12s
* dt_remap (vertical remap dt) 112.50
* dt (del2 sponge ; u,v,T,dM) < 8.41s 14.06s
WARNING: theoretically unstable in sponge; increase se_hypervis_subcycle_sponge
(this WARNING can sometimes be ignored in level 1)
* dt (del2 sponge ; u,v,T,dM) < 28.74s 14.06s
* dt (del2 sponge ; u,v,T,dM) < 96.72s 14.06s

Which indicates that the changes you made did make some difference to the time-stepping tracers! I'd recommend doing what the log message says and increasing se_hypervis_subcycle_sponge. You can see what it is by default in your run/atm_in file and increase it from there (in your user_nl_cam file).

Courtney
 

Lumoss

Member
Hi again,

The logging about the dynamical core timesteps now looks looks like:



Which indicates that the changes you made did make some difference to the time-stepping tracers! I'd recommend doing what the log message says and increasing se_hypervis_subcycle_sponge. You can see what it is by default in your run/atm_in file and increase it from there (in your user_nl_cam file).

Courtney
Thank you very much for your reply. At the suggestion of jedwards, I decided to update the version of my compiler and some necessary libraries first, which may take some time. After that, I will try your suggestion and hope to solve the problem. Thanks again!
 

Lumoss

Member
Hi again,

The logging about the dynamical core timesteps now looks looks like:



Which indicates that the changes you made did make some difference to the time-stepping tracers! I'd recommend doing what the log message says and increasing se_hypervis_subcycle_sponge. You can see what it is by default in your run/atm_in file and increase it from there (in your user_nl_cam file).

Courtney
Hi,

I added se_hypervis_subcycle_sponge=6 as you suggested. It still doesn't seem to be working. The log file is enclosed. Perhaps you could provide me with some other guidance. Thanks again!

Lumos
 

Attachments

  • atm.log.797378.mgr.231224-212944.txt
    379.6 KB · Views: 1
  • cesm.log.797378.mgr.231224-212944.txt
    22.4 KB · Views: 0

peverley

Courtney Peverley
Moderator
Hi again.

Taking a look at your atm.log.* file now, you have:
* dt_dyn (time-stepping dycore ; u,v,T,dM) < 37.85s 37.50s
* dt_dyn_vis (hyperviscosity) ; u,v,T,dM) < 46.89s 12.50s
* dt_tracer_se (time-stepping tracers ; q ) < 32.84s 37.50s
WARNING: dt_tracer_se theoretically unstable
* dt_tracer_vis (hyperviscosity tracers; q ) < 210.60s 37.50s
* dt_remap (vertical remap dt) 112.50
* dt (del2 sponge ; u,v,T,dM) < 8.41s 6.25s
* dt (del2 sponge ; u,v,T,dM) < 28.74s 6.25s
* dt (del2 sponge ; u,v,T,dM) < 96.72s 6.25s
The WARNING is now the same as in your initial post. Can you try incrementing se_rsplit by one again?

Courtney
 

Lumoss

Member
Hi

I actually tried adjusting se_rsplit, and the warning disappeared, but I ended up reporting the same error.

Lumos
 

peverley

Courtney Peverley
Moderator
Bummer!

Is the error occurring on the same timestep? Can you post your atm.log* and cesm.log* files again?

Court
 

Lumoss

Member
Of course, here are my atm.log* and cesm.log* files.

Lumos
 

Attachments

  • cesm.log.799593.mgr.240109-090141.txt
    24.8 KB · Views: 1
  • atm.log.799593.mgr.240109-090141.txt
    395.6 KB · Views: 3
Top