size of cesm.log file grows to several GBs

Y

yugong

yugong
Member
I’ve encountered a strange issue where the size of the cesm.log file grows to several GBs.The reason appears to be that the model keeps printing the following message repeatedly in the log file

imp_sol: time step 1800.000 failed to converge @ (lchnk,vctrpos,nstep) = 2984 489 44
imp_sol: time step 1800.000 failed to converge @ (lchnk,vctrpos,nstep) = 410 489 44
imp_sol: time step 1800.000 failed to converge @ (lchnk,vctrpos,nstep) = 1331 489 45
imp_sol: time step 1800.000 failed to converge @ (lchnk,vctrpos,nstep) = 3040 521 46
max rss=1259.6 MB
imp_sol: time step 1800.000 failed to converge @ (lchnk,vctrpos,nstep) = 2038 1965 63
imp_sol: time step 900.0000 failed to converge @ (lchnk,vctrpos,nstep) = 2038 1965 63
imp_sol: time step 450.0000 failed to converge @ (lchnk,vctrpos,nstep) = 2038 1965 63
imp_sol: time step 225.0000 failed to converge @ (lchnk,vctrpos,nstep) = 2038 1965 63
imp_sol: time step 112.5000 failed to converge @ (lchnk,vctrpos,nstep) = 2038 1965 63
imp_sol: time step 11.25000 failed to converge @ (lchnk,vctrpos,nstep) = 2038 1965 63
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = 2038 1965 63 11.25000 1811.250
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = ******************************************************
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = ******************************************************
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = ******************************************************
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = ******************************************************
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = ******************************************************
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = ******************************************************
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = ********
O2_1S 0.000E+00
imp_sol: time step 11.25000 failed to converge @ (lchnk,vctrpos,nstep) = 2038 1965 63
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = 2038 1965 63 11.25000 1811.250
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = ******************************************************
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = ******************************************************
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = ******************************************************
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = ******************************************************
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = ******************************************************
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = ******************************************************
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = ********


Here is what I did:
  1. I ran an FWHIST compset and saved the historical fields of U, V, T, Q, PS, TS, and PHIS.
  2. I then ran an FWSD compset using the output from step 1 as the meteorological forcing (met) file.
  3. The initial condition files used in both experiments were identical.
  4. For comparison, I also conducted an FWSD experiment using MERRA2 as the met forcing, and it ran successfully without issues.

Has anyone encountered this issue before? Any suggestions or insights would be greatly appreciated! And here are the log files （I canceled the job）
 
Sort by date Sort by votes
S

shawnh

New Member
Hi Yugong,

You may want to try temporarily increasing the time step in your run. Based on your included log files, you are doing an FV run with the default value of 'fv_nsplit=16'. I would try setting it to 32 and try running the model out a week. If it runs without crashing, then you can try setting it back to 16 and continue the run and hopefully the issue will be solved.

-Shawn Honomichl
 
Vote Upvote 0 Downvote
X

xiaoxiaokuishu

Ru Xu
Member
yugong said:
I’ve encountered a strange issue where the size of the cesm.log file grows to several GBs.The reason appears to be that the model keeps printing the following message repeatedly in the log file

imp_sol: time step 1800.000 failed to converge @ (lchnk,vctrpos,nstep) = 2984 489 44
imp_sol: time step 1800.000 failed to converge @ (lchnk,vctrpos,nstep) = 410 489 44
imp_sol: time step 1800.000 failed to converge @ (lchnk,vctrpos,nstep) = 1331 489 45
imp_sol: time step 1800.000 failed to converge @ (lchnk,vctrpos,nstep) = 3040 521 46
max rss=1259.6 MB
imp_sol: time step 1800.000 failed to converge @ (lchnk,vctrpos,nstep) = 2038 1965 63
imp_sol: time step 900.0000 failed to converge @ (lchnk,vctrpos,nstep) = 2038 1965 63
imp_sol: time step 450.0000 failed to converge @ (lchnk,vctrpos,nstep) = 2038 1965 63
imp_sol: time step 225.0000 failed to converge @ (lchnk,vctrpos,nstep) = 2038 1965 63
imp_sol: time step 112.5000 failed to converge @ (lchnk,vctrpos,nstep) = 2038 1965 63
imp_sol: time step 11.25000 failed to converge @ (lchnk,vctrpos,nstep) = 2038 1965 63
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = 2038 1965 63 11.25000 1811.250
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = ******************************************************
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = ******************************************************
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = ******************************************************
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = ******************************************************
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = ******************************************************
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = ******************************************************
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = ********
O2_1S 0.000E+00
imp_sol: time step 11.25000 failed to converge @ (lchnk,vctrpos,nstep) = 2038 1965 63
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = 2038 1965 63 11.25000 1811.250
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = ******************************************************
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = ******************************************************
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = ******************************************************
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = ******************************************************
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = ******************************************************
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = ******************************************************
imp_sol: step failed to converge @ (lchnk,vctrpos,nstep,dt,time) = ********


Here is what I did:
  1. I ran an FWHIST compset and saved the historical fields of U, V, T, Q, PS, TS, and PHIS.
  2. I then ran an FWSD compset using the output from step 1 as the meteorological forcing (met) file.
  3. The initial condition files used in both experiments were identical.
  4. For comparison, I also conducted an FWSD experiment using MERRA2 as the met forcing, and it ran successfully without issues.

Has anyone encountered this issue before? Any suggestions or insights would be greatly appreciated! And here are the log files （I canceled the job）
Click to expand...

Hi, Yugong,

I have met the same problems, that is use FHIST to generate atm forcing, then use them for simulation.
Have you finally solved the problems?

Best
Ru
 
Vote Upvote 0 Downvote
You must log in or register to reply here.
Back
Top