pawanvats5086@gmail_com
New Member
Hi, I am using CESM1_2_2 with user defined component set that has atmospheric component CAM5.3 with GEOS5 offline meteorology, at resolution f09_f09. I am performing some sensitivity experiments for period 2010 to 2015 by using CAM5.3 with GEOS5 offline meteorology. My simulation crashed, when model start simulating for 2014-June-01. I found that model with offline meteorology was not able to do simulation between 2014-June-01 to 2014-September-30. Before and after this period model is able to simulate.Model crashed with following error in this time period: ... 0.000000000000000 9.8404087795655556E-037 1.0064131316984391E-036
dpcoup cant adjust 19 3401 7
0.000000000000000 0.000000000000000 1.0009329722761768E-036
dpcoup cant adjust 19 1040 5
0.000000000000000 0.000000000000000 1.0038706644489480E-036
QNEG3 from convtran2/dst_a3:m= 22 lat/lchnk= 3156 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-6.1E-11 at i,k= 14 51
QNEG3 from convtran2/ncl_a3:m= 23 lat/lchnk= 3156 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-1.6E-11 at i,k= 14 51
QNEG3 from vertical diffusion/H2O2:m= 6 lat/lchnk= 1664 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-6.4E-12 at i,k= 8 56
QNEG3 from vertical diffusion/H2SO4:m= 7 lat/lchnk= 3585 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-2.0E-12 at i,k= 8 56
QNEG3 from vertical diffusion/SO2:m= 8 lat/lchnk= 3072 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-1.0E-11 at i,k= 8 56
QNEG3 from vertical diffusion/H2O2:m= 6 lat/lchnk= 3193 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-1.1E-12 at i,k= 8 56
QNEG3 from vertical diffusion/SO2:m= 8 lat/lchnk= 555 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-1.1E-12 at i,k= 8 56
filew failed, worst i, j, qtmp, q = 1 185
-3.3448716151483166E-016 5.1208836450786653E-020
filew failed, worst i, j, qtmp, q = 1 186
-7.6087175594883724E-015 1.9793913218082724E-022
dpcoup cant adjust 19 241 7
0.000000000000000 0.000000000000000 1.0002781147262478E-036
dpcoup cant adjust 7 877 9
0.000000000000000 0.000000000000000 1.0065436744817294E-036
dpcoup cant adjust 7 3341 3
1.6074034262715951E-039 2.3411313270475035E-039 1.0029901493386215E-036
dpcoup cant adjust 19 3401 7
0.000000000000000 7.7951532955515773E-038 1.0006849384412761E-036
dpcoup cant adjust 7 2132 9
0.000000000000000 0.000000000000000 1.0043991598736111E-036
dpcoup cant adjust 19 2977 7
2.3802280451412106E-040 3.5867181475560873E-039 1.0004399546443275E-036
dpcoup cant adjust 19 1040 5
0.000000000000000 0.000000000000000 1.0036020276055295E-036
QNEG3 from convtran2/ncl_a1:m= 16 lat/lchnk= 1618 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-5.6E-12 at i,k= 16 50
QNEG3 from convtran2/dst_a3:m= 22 lat/lchnk= 1618 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-1.2E-12 at i,k= 16 50
QNEG3 from convtran2/ncl_a3:m= 23 lat/lchnk= 1618 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-5.9E-11 at i,k= 16 50
QNEG3 from convtran2/num_a3:m= 25 lat/lchnk= 1618 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-6.2E+03 at i,k= 16 50
QNEG3 from vertical diffusion/SO2:m= 8 lat/lchnk= 1281 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-2.5E-12 at i,k= 8 56
QNEG3 from vertical diffusion/H2O2:m= 6 lat/lchnk= 1665 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-5.3E-12 at i,k= 8 56
QNEG3 from vertical diffusion/H2SO4:m= 7 lat/lchnk= 3057 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-1.3E-12 at i,k= 10 56
QNEG3 from vertical diffusion/H2SO4:m= 7 lat/lchnk= 1059 Min. mixing ratio violated at 2 points. Reset to 1.0E-36 Worst =-1.1E-12 at i,k= 10 56
QNEG3 from vertical diffusion/SO2:m= 8 lat/lchnk= 606 Min. mixing ratio violated at 2 points. Reset to 1.0E-36 Worst =-1.4E-12 at i,k= 9 56
--------------------------------------------------------------------------
WARNING: A process refused to die despite all the efforts!
This process may still be running and/or consuming resources.
Host: cn202
PID: 27367
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun noticed that process rank 2 with PID 27351 on node cn202 exited on signal 11 (Segmentation fault).
-------------------------------------------------------------------------- I didn’t find out the origin of this problem. I am attaching all the logs of current run.Please help me to solve this problem.Thanking you in anticipation. Best regardsPawan Vats
dpcoup cant adjust 19 3401 7
0.000000000000000 0.000000000000000 1.0009329722761768E-036
dpcoup cant adjust 19 1040 5
0.000000000000000 0.000000000000000 1.0038706644489480E-036
QNEG3 from convtran2/dst_a3:m= 22 lat/lchnk= 3156 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-6.1E-11 at i,k= 14 51
QNEG3 from convtran2/ncl_a3:m= 23 lat/lchnk= 3156 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-1.6E-11 at i,k= 14 51
QNEG3 from vertical diffusion/H2O2:m= 6 lat/lchnk= 1664 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-6.4E-12 at i,k= 8 56
QNEG3 from vertical diffusion/H2SO4:m= 7 lat/lchnk= 3585 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-2.0E-12 at i,k= 8 56
QNEG3 from vertical diffusion/SO2:m= 8 lat/lchnk= 3072 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-1.0E-11 at i,k= 8 56
QNEG3 from vertical diffusion/H2O2:m= 6 lat/lchnk= 3193 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-1.1E-12 at i,k= 8 56
QNEG3 from vertical diffusion/SO2:m= 8 lat/lchnk= 555 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-1.1E-12 at i,k= 8 56
filew failed, worst i, j, qtmp, q = 1 185
-3.3448716151483166E-016 5.1208836450786653E-020
filew failed, worst i, j, qtmp, q = 1 186
-7.6087175594883724E-015 1.9793913218082724E-022
dpcoup cant adjust 19 241 7
0.000000000000000 0.000000000000000 1.0002781147262478E-036
dpcoup cant adjust 7 877 9
0.000000000000000 0.000000000000000 1.0065436744817294E-036
dpcoup cant adjust 7 3341 3
1.6074034262715951E-039 2.3411313270475035E-039 1.0029901493386215E-036
dpcoup cant adjust 19 3401 7
0.000000000000000 7.7951532955515773E-038 1.0006849384412761E-036
dpcoup cant adjust 7 2132 9
0.000000000000000 0.000000000000000 1.0043991598736111E-036
dpcoup cant adjust 19 2977 7
2.3802280451412106E-040 3.5867181475560873E-039 1.0004399546443275E-036
dpcoup cant adjust 19 1040 5
0.000000000000000 0.000000000000000 1.0036020276055295E-036
QNEG3 from convtran2/ncl_a1:m= 16 lat/lchnk= 1618 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-5.6E-12 at i,k= 16 50
QNEG3 from convtran2/dst_a3:m= 22 lat/lchnk= 1618 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-1.2E-12 at i,k= 16 50
QNEG3 from convtran2/ncl_a3:m= 23 lat/lchnk= 1618 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-5.9E-11 at i,k= 16 50
QNEG3 from convtran2/num_a3:m= 25 lat/lchnk= 1618 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-6.2E+03 at i,k= 16 50
QNEG3 from vertical diffusion/SO2:m= 8 lat/lchnk= 1281 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-2.5E-12 at i,k= 8 56
QNEG3 from vertical diffusion/H2O2:m= 6 lat/lchnk= 1665 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-5.3E-12 at i,k= 8 56
QNEG3 from vertical diffusion/H2SO4:m= 7 lat/lchnk= 3057 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-1.3E-12 at i,k= 10 56
QNEG3 from vertical diffusion/H2SO4:m= 7 lat/lchnk= 1059 Min. mixing ratio violated at 2 points. Reset to 1.0E-36 Worst =-1.1E-12 at i,k= 10 56
QNEG3 from vertical diffusion/SO2:m= 8 lat/lchnk= 606 Min. mixing ratio violated at 2 points. Reset to 1.0E-36 Worst =-1.4E-12 at i,k= 9 56
--------------------------------------------------------------------------
WARNING: A process refused to die despite all the efforts!
This process may still be running and/or consuming resources.
Host: cn202
PID: 27367
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun noticed that process rank 2 with PID 27351 on node cn202 exited on signal 11 (Segmentation fault).
-------------------------------------------------------------------------- I didn’t find out the origin of this problem. I am attaching all the logs of current run.Please help me to solve this problem.Thanking you in anticipation. Best regardsPawan Vats