Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

CAM5.3 with GEOS5 offline meteorology simulation terminates on 2014-June-01

Hi,     I am using CESM1_2_2 with user defined component set that has atmospheric component CAM5.3 with GEOS5 offline meteorology, at resolution f09_f09. I am performing some sensitivity experiments for period 2010 to 2015 by using CAM5.3 with GEOS5 offline meteorology. My simulation crashed, when model start simulating for 2014-June-01. I found that model with offline meteorology was not able to do simulation between 2014-June-01 to 2014-September-30. Before and after this period model is able to simulate.Model crashed with following error in this time period: ...   0.000000000000000        9.8404087795655556E-037   1.0064131316984391E-036
 dpcoup cant adjust           19         3401            7
    0.000000000000000         0.000000000000000        1.0009329722761768E-036
 dpcoup cant adjust           19         1040            5
    0.000000000000000         0.000000000000000        1.0038706644489480E-036
 QNEG3 from convtran2/dst_a3:m= 22 lat/lchnk=   3156 Min. mixing ratio violated at    1 points.  Reset to  1.0E-36 Worst =-6.1E-11 at i,k=  14 51
 QNEG3 from convtran2/ncl_a3:m= 23 lat/lchnk=   3156 Min. mixing ratio violated at    1 points.  Reset to  1.0E-36 Worst =-1.6E-11 at i,k=  14 51
 QNEG3 from vertical diffusion/H2O2:m=  6 lat/lchnk=   1664 Min. mixing ratio violated at    1 points.  Reset to  1.0E-36 Worst =-6.4E-12 at i,k=   8 56
 QNEG3 from vertical diffusion/H2SO4:m=  7 lat/lchnk=   3585 Min. mixing ratio violated at    1 points.  Reset to  1.0E-36 Worst =-2.0E-12 at i,k=   8 56
 QNEG3 from vertical diffusion/SO2:m=  8 lat/lchnk=   3072 Min. mixing ratio violated at    1 points.  Reset to  1.0E-36 Worst =-1.0E-11 at i,k=   8 56
 QNEG3 from vertical diffusion/H2O2:m=  6 lat/lchnk=   3193 Min. mixing ratio violated at    1 points.  Reset to  1.0E-36 Worst =-1.1E-12 at i,k=   8 56
 QNEG3 from vertical diffusion/SO2:m=  8 lat/lchnk=    555 Min. mixing ratio violated at    1 points.  Reset to  1.0E-36 Worst =-1.1E-12 at i,k=   8 56
  filew failed, worst i, j, qtmp, q =             1          185
  -3.3448716151483166E-016   5.1208836450786653E-020
  filew failed, worst i, j, qtmp, q =             1          186
  -7.6087175594883724E-015   1.9793913218082724E-022
 dpcoup cant adjust           19          241            7
    0.000000000000000         0.000000000000000        1.0002781147262478E-036
 dpcoup cant adjust            7          877            9
    0.000000000000000         0.000000000000000        1.0065436744817294E-036
 dpcoup cant adjust            7         3341            3
   1.6074034262715951E-039   2.3411313270475035E-039   1.0029901493386215E-036
 dpcoup cant adjust           19         3401            7
    0.000000000000000        7.7951532955515773E-038   1.0006849384412761E-036
 dpcoup cant adjust            7         2132            9
    0.000000000000000         0.000000000000000        1.0043991598736111E-036
 dpcoup cant adjust           19         2977            7
   2.3802280451412106E-040   3.5867181475560873E-039   1.0004399546443275E-036
 dpcoup cant adjust           19         1040            5
    0.000000000000000         0.000000000000000        1.0036020276055295E-036
 QNEG3 from convtran2/ncl_a1:m= 16 lat/lchnk=   1618 Min. mixing ratio violated at    1 points.  Reset to  1.0E-36 Worst =-5.6E-12 at i,k=  16 50
 QNEG3 from convtran2/dst_a3:m= 22 lat/lchnk=   1618 Min. mixing ratio violated at    1 points.  Reset to  1.0E-36 Worst =-1.2E-12 at i,k=  16 50
 QNEG3 from convtran2/ncl_a3:m= 23 lat/lchnk=   1618 Min. mixing ratio violated at    1 points.  Reset to  1.0E-36 Worst =-5.9E-11 at i,k=  16 50
 QNEG3 from convtran2/num_a3:m= 25 lat/lchnk=   1618 Min. mixing ratio violated at    1 points.  Reset to  1.0E-36 Worst =-6.2E+03 at i,k=  16 50
 QNEG3 from vertical diffusion/SO2:m=  8 lat/lchnk=   1281 Min. mixing ratio violated at    1 points.  Reset to  1.0E-36 Worst =-2.5E-12 at i,k=   8 56
 QNEG3 from vertical diffusion/H2O2:m=  6 lat/lchnk=   1665 Min. mixing ratio violated at    1 points.  Reset to  1.0E-36 Worst =-5.3E-12 at i,k=   8 56
 QNEG3 from vertical diffusion/H2SO4:m=  7 lat/lchnk=   3057 Min. mixing ratio violated at    1 points.  Reset to  1.0E-36 Worst =-1.3E-12 at i,k=  10 56
 QNEG3 from vertical diffusion/H2SO4:m=  7 lat/lchnk=   1059 Min. mixing ratio violated at    2 points.  Reset to  1.0E-36 Worst =-1.1E-12 at i,k=  10 56
 QNEG3 from vertical diffusion/SO2:m=  8 lat/lchnk=    606 Min. mixing ratio violated at    2 points.  Reset to  1.0E-36 Worst =-1.4E-12 at i,k=   9 56
--------------------------------------------------------------------------
WARNING: A process refused to die despite all the efforts!
This process may still be running and/or consuming resources.

Host: cn202
PID:  27367

--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun noticed that process rank 2 with PID 27351 on node cn202 exited on signal 11 (Segmentation fault).
-------------------------------------------------------------------------- I didn’t find out the origin of this problem. I am attaching all the logs of current run.Please help me to solve this problem.Thanking you in anticipation.  Best regardsPawan Vats 
 
Top