Debugging Failed Functionality Tests

Thanks in large part to the advice provided on this forum post, https://bb.cgd.ucar.edu/porting-some-validation-tests-fail-createtest, I have successfully build and compuled all 11 of the suggested functionality tests in the user guide. However, even the simplest, first case appears to have failed. I know that it ran and completed successfully (albeit I had to resubmit it thrice to do so). Below is the TestCase.out file. I cannot seem to find any other error messages. Any help is greatly appreciated.Also, this individual who wrote this post appears to be having similar problems. https://bb.cgd.ucar.edu/fail-status-eri-functionality-testsdoing a 11 ndays initial testdoing a 11 ndays initial testdoing a 11 ndays initial testpass = 1doing a 5 ndays restart testInitial Test log is /users/jnickla1/scratch/cesm1_2_2/ERS_D.f19_g16.X.oscar_intel.t01/run/cpl.log.170625-092635Restart Test log is /users/jnickla1/scratch/cesm1_2_2/ERS_D.f19_g16.X.oscar_intel.t01/run/cpl.log.170625-093207Initial Test hist is /users/jnickla1/scratch/cesm1_2_2/ERS_D.f19_g16.X.oscar_intel.t01/run/ERS_D.f19_g16.X.oscar_intel.t01.cpl.hi.0001-01-12-00000.nc.baseRestart Test hist is /users/jnickla1/scratch/cesm1_2_2/ERS_D.f19_g16.X.oscar_intel.t01/run/ERS_D.f19_g16.X.oscar_intel.t01.cpl.hi.0001-01-12-00000.ncComparing initial log file with second log fileDifference found beginning at 10107 1800 :< comm_diag xxx sorr   1 5.1009969907037340000E+15 send lnd Sa_z> comm_diag xxx sorr   1 5.1009969907037350000E+15 send lnd Sa_z< comm_diag xxx sorr   2 5.1009969907037350000E+15 send lnd Sa_u> comm_diag xxx sorr   2 5.1009969907037340000E+15 send lnd Sa_u< comm_diag xxx sorr   4 5.1009969907037380000E+15 send lnd Sa_tbot> comm_diag xxx sorr   4 5.1009969907037400000E+15 send lnd Sa_tbot< comm_diag xxx sorr   9 2.0706193616059508910E+29 send lnd Faxa_rainc> comm_diag xxx sorr   9 2.0706193616059512429E+29 send lnd Faxa_rainc< comm_diag xxx sorr  13 2.0706193616059547613E+29 send lnd Faxa_lwdn> comm_diag xxx sorr  13 2.0706193616059540576E+29 send lnd Faxa_lwdn< comm_diag xxx sorr  14 2.0706193616059501873E+29 send lnd Faxa_swndr> comm_diag xxx sorr  14 2.0706193616059498355E+29 send lnd Faxa_swndr< comm_diag xxx sorr  15 2.0706193616059554650E+29 send lnd Faxa_swvdr> comm_diag xxx sorr  15 2.0706193616059561687E+29 send lnd Faxa_swvdr< comm_diag xxx sorr  16 2.0706193616059533539E+29 send lnd Faxa_swndf> comm_diag xxx sorr  16 2.0706193616059537058E+29 send lnd Faxa_swndf< comm_diag xxx sorr  17 2.0706193616059565205E+29 send lnd Faxa_swvdf> comm_diag xxx sorr  17 2.0706193616059561687E+29 send lnd Faxa_swvdf< comm_diag xxx sorr  19 2.0706193616059572242E+29 send lnd Faxa_bcphodry> comm_diag xxx sorr  19 2.0706193616059565205E+29 send lnd Faxa_bcphodry< comm_diag xxx sorr  20 2.0706193616059554650E+29 send lnd Faxa_bcphiwet> comm_diag xxx sorr  20 2.0706193616059565205E+29 send lnd Faxa_bcphiwetFAIL ======================= (seq_io_wopen)  create file ERS_D.f19_g16.X.oscar_intel.t01.cpl.hi.0001-01-12-00000.nc tStamp_write: model date =    10112       0 wall clock = 2017-06-25 09:32:03 avg dt =     0.52 dt =     1.41 memory_write: model date =    10112       0 memory =       0.00 MB (highwater)  0.00 MB (usage)  (pe=    0 comps= cpl ATM LND OCN ICE GLC ROF WAV) nextsw_cday =    13.0000000000000   (seq_mct_drv): ===============       SUCCESSFUL TERMINATION OF CPL7-CCSM ===============(seq_mct_drv): ===============        at YMD,TOD =    10112       0      ===============(seq_mct_drv): ===============  # simulated days (this run) =    11.000  ===============(seq_mct_drv): ===============  compute time (hrs) =     0.076  ===============(seq_mct_drv): ===============  # simulated years / cmp-day =     9.459  ===============(seq_mct_drv): ===============  pes min memory highwater  (MB)    0.000  ===============(seq_mct_drv): ===============  pes max memory highwater  (MB)  226.946  ===============(seq_mct_drv): ===============  pes min memory last usage (MB)   -0.001  ===============(seq_mct_drv): ===============  pes max memory last usage (MB)    0.000  =============== CHECK ERS_D.f19_g16.X.oscar_intel.perf npes=16 tput=9.459 memh=226.946 memr=0.000 tag=cesm1_2_2 NOTE:  At user request, clean up not done.  Use the following commands to clean up by hand: rm -rf /users/jnickla1/scratch/cesm1_2_2/ERS_D.f19_g16.X.oscar_intel.t01/blddoing a 11 ndays initial testdoing a 11 ndays initial testpass = 1doing a 5 ndays restart testInitial Test log is /users/jnickla1/scratch/cesm1_2_2/ERS_D.f19_g16.X.oscar_intel.t01/run/cpl.log.170625-092635Restart Test log is /users/jnickla1/scratch/cesm1_2_2/ERS_D.f19_g16.X.oscar_intel.t01/run/cpl.log.170625-093207 Initial Test hist is /users/jnickla1/scratch/cesm1_2_2/ERS_D.f19_g16.X.oscar_intel.t01/run/ERS_D.f19_g16.X.oscar_intel.t01.cpl.hi.0001-01-12-00000.nc.baseRestart Test hist is /users/jnickla1/scratch/cesm1_2_2/ERS_D.f19_g16.X.oscar_intel.t01/run/ERS_D.f19_g16.X.oscar_intel.t01.cpl.hi.0001-01-12-00000.ncComparing initial log file with second log fileDifference found beginning at 10107 1800 :< comm_diag xxx sorr   1 5.1009969907037340000E+15 send lnd Sa_z> comm_diag xxx sorr   1 5.1009969907037350000E+15 send lnd Sa_z< comm_diag xxx sorr   2 5.1009969907037350000E+15 send lnd Sa_u> comm_diag xxx sorr   2 5.1009969907037340000E+15 send lnd Sa_u< comm_diag xxx sorr   4 5.1009969907037380000E+15 send lnd Sa_tbot> comm_diag xxx sorr   4 5.1009969907037400000E+15 send lnd Sa_tbot< comm_diag xxx sorr   9 2.0706193616059508910E+29 send lnd Faxa_rainc> comm_diag xxx sorr   9 2.0706193616059512429E+29 send lnd Faxa_rainc< comm_diag xxx sorr  13 2.0706193616059547613E+29 send lnd Faxa_lwdn> comm_diag xxx sorr  13 2.0706193616059540576E+29 send lnd Faxa_lwdn< comm_diag xxx sorr  14 2.0706193616059501873E+29 send lnd Faxa_swndr> comm_diag xxx sorr  14 2.0706193616059498355E+29 send lnd Faxa_swndr< comm_diag xxx sorr  15 2.0706193616059554650E+29 send lnd Faxa_swvdr> comm_diag xxx sorr  15 2.0706193616059561687E+29 send lnd Faxa_swvdr< comm_diag xxx sorr  16 2.0706193616059533539E+29 send lnd Faxa_swndf> comm_diag xxx sorr  16 2.0706193616059537058E+29 send lnd Faxa_swndf< comm_diag xxx sorr  17 2.0706193616059565205E+29 send lnd Faxa_swvdf> comm_diag xxx sorr  17 2.0706193616059561687E+29 send lnd Faxa_swvdf< comm_diag xxx sorr  19 2.0706193616059572242E+29 send lnd Faxa_bcphodry> comm_diag xxx sorr  19 2.0706193616059565205E+29 send lnd Faxa_bcphodry< comm_diag xxx sorr  20 2.0706193616059554650E+29 send lnd Faxa_bcphiwet> comm_diag xxx sorr  20 2.0706193616059565205E+29 send lnd Faxa_bcphiwetFAIL ======================= (seq_io_wopen)  create file ERS_D.f19_g16.X.oscar_intel.t01.cpl.hi.0001-01-12-00000.nc tStamp_write: model date =    10112       0 wall clock = 2017-06-25 09:32:03 avg dt =     0.52 dt =     1.41 memory_write: model date =    10112       0 memory =       0.00 MB (highwater)  0.00 MB (usage)  (pe=    0 comps= cpl ATM LND OCN ICE GLC ROF WAV)   nextsw_cday =    13.0000000000000   (seq_mct_drv): ===============       SUCCESSFUL TERMINATION OF CPL7-CCSM ===============(seq_mct_drv): =============== at YMD,TOD =    10112       0      ===============(seq_mct_drv): ===============  # simulated days (this run) =    11.000  ===============(seq_mct_drv): ===============  compute time (hrs) =     0.076  ===============(seq_mct_drv): ===============  # simulated years / cmp-day =     9.459  ===============(seq_mct_drv): ===============  pes min memory highwater  (MB)    0.000  ===============(seq_mct_drv): ===============  pes max memory highwater  (MB)  226.946  ===============(seq_mct_drv): ===============  pes min memory last usage (MB)   -0.001  ===============(seq_mct_drv): ===============  pes max memory last usage (MB)    0.000  ===============   =======================  CHECK ERS_D.f19_g16.X.oscar_intel.perf npes=16 tput=9.459 memh=226.946 memr=0.000 tag=cesm1_2_2 NOTE:  At user request, clean up not done.  Use the following commands to clean up by hand:  rm -rf /users/jnickla1/scratch/cesm1_2_2/ERS_D.f19_g16.X.oscar_intel.t01/bld
 
Back
Top