CESM BRANCH RUN ERROR

[sunjzh13@163_com]

Hi there, When I setup a startup or hybrid run exactly according to the guidelines in the user's guide, the model can complete successfully.But when I setup a branch run the same way as the hybrid run, also carefully referring to the user's guide step by step, the model aborted with below error information:

• Opened existing file b40.1850.track1.2deg.003.cam.r.0501-01-01-00000.nc
•            65536
• NetCDF: Attribute not found
• NetCDF: Attribute not found
• WARNING: Rearr optional argument is a pio2 feature, ignored in pio1
• WARNING: Rearr optional argument is a pio2 feature, ignored in pio1
• NetCDF: Variable not found
• pio_support::pio_die:: myrank=      -1  :  ERROR: nf_mod.F90:             730 :
• NetCDF: Variable not found

For all three types of runs, the initial condition files are downloaded from the official site.Please offer me some help with this problem, your help will be much appreciated.Thanks!

 

[xueyanzhang520@gmail_com]

Hi,I have a same error: but I just use CLM5. Have you solved it?NetCDF: Variable not foundpio_support::pio_die:: myrank=      -1  :  ERROR: nf_mod.F90:             730 :[/list]

 

[jedwards]

These netcdf messages are not model errors - they only indicate that a variable was not found on the input file.  If the variable was required there would be an additional abort message.   Is this the end of the cesm.log or is there more to follow?   If you can't find the correct source of the error here it may be nessasary to recompile in DEBUG mode. 

 

[xueyanzhang520@gmail_com]

Hello, When I use ./case.build --debug, there is no error. But after I ./case.submit, the run is always failed. Just error I said above. Attached files are my cesm and atm log.

 

[sunjzh13@163_com]

Hi,Thanks for your reply. Below shows the complete end of cesm.log:

•  Opened existing file b40.1850.track1.2deg.003.cam.r.0501-01-01-00000.nc
•        65536
•  NetCDF: Attribute not found
•  NetCDF: Attribute not found
•  WARNING: Rearr optional argument is a pio2 feature, ignored in pio1
•  WARNING: Rearr optional argument is a pio2 feature, ignored in pio1
•  NetCDF: Variable not found
•  pio_support::pio_die:: myrank=          -1 : ERROR: nf_mod.F90:         730 :
•  NetCDF: Variable not found
• Image              PC                Routine            Line        Source
• cesm.exe           00000000018C06CD  Unknown               Unknown  Unknown
• cesm.exe           0000000001716EE1  pio_support_mp_pi         120  pio_support.F90
• cesm.exe           000000000171502D  pio_utils_mp_chec          74  pio_utils.F90
• cesm.exe           00000000016FD198  nf_mod_mp_inq_var         730  nf_mod.F90
• cesm.exe           000000000056A7A3  physics_buffer_mp        1498  physics_buffer.F90.in
• cesm.exe           0000000000568ABC  physics_buffer_mp        1438  physics_buffer.F90.in
• cesm.exe           00000000005E8CC3  restart_physics_m         547  restart_physics.F90
• cesm.exe           00000000004F9F93  cam_restart_mp_ca         396  cam_restart.F90
• cesm.exe           00000000004BA8DA  cam_comp_mp_cam_i         168  cam_comp.F90
• cesm.exe           00000000004B2D46  atm_comp_mct_mp_a         262  atm_comp_mct.F90
• cesm.exe           000000000042C8DB  component_mod_mp_         235  component_mod.F90
• cesm.exe           000000000041D857  cesm_comp_mod_mp_        1037  cesm_comp_mod.F90
• cesm.exe           0000000000427F48  MAIN__                     92  cesm_driver.F90
• cesm.exe           00000000004130DE  Unknown               Unknown  Unknown
• libc-2.17.so       00002B96A91B6B15  __libc_start_main     Unknown  Unknown
• cesm.exe           0000000000412FE9  Unknown               Unknown  Unknown
• application called MPI_Abort(MPI_COMM_WORLD, 1) - process 1

Does "myrank=   -1" give some hint about the error source? Cause -1 is not a reasonable value for MPI rank.Another opinon I got is that I cannot setup a branch run without knowing the settings for generating the restart files I use, cause the settings for a branch run should be COMPLETELY the same as the previous run, is this a possible reason?What's more, when I recompile in DEBUG mode (set DEBUG in env_build.xml to TRUE), I cannot even pass the build with error information in mct.bldlog shown as below:

• gmake
• gmake[1]: Entering directory `/home/zhengc/sun/work/w1/bld/intel/impi/debug/nothreads/mct/mpeu'
• rm -f libmpeu.a
• ar cq libmpeu.a get_zeits.o m_IndexBin_char.o m_IndexBin_integer.o m_IndexBin_logical.o m_List.o m_MergeSorts.o m_Filename.o m_FcComms.o m_Permuter.o m_SortingTools.o m_String.o m_StrTemplate.o m_chars.o m_die.o m_dropdead.o m_FileResolv.o m_flow.o m_inpak90.o m_ioutil.o m_mall.o m_mpif.o m_mpif90.o m_mpout.o m_rankMerge.o m_realkinds.o m_stdio.o m_TraceBack.o m_zeit.o
• ar: get_zeits.o: No such file or directory
• gmake[1]: *** [libmpeu.a] Error 1
• gmake[1]: Leaving directory `/home/zhengc/sun/work/w1/bld/intel/impi/debug/nothreads/mct/mpeu'
• gmake: *** [subdirs] Error 2
• exit 1

I have met this "no get_zeits.o" error before, to get rid of it I used to manually copy this .o file to the required directory (though I do not know why this specfic file is missing). But after manually copied, ./case.build can be completed successfully under non-DEBUG mode. Thanks.

 

[sunjzh13@163_com]

Hi,The problem has not been solved.Do you also meet this error only in a BRANCH run setup?I recompile the model in DEBUG mode by setting variable "DEBUG" in env_build.xml to TRUE and got another error. I do not know the difference between how we compile in DEBUG mode, maybe you can try what I did to see if you can get some clues.Please let me know if you can fix the problem.Thanks.

 

[jedwards]

You never mentioned what version of the model you are using but this error is due to a variable expected to be in the restart file that was not found.  You should look in the atm.log or in line  1498  physics_buffer.F90.in   to determine the variable in question.    I have never seen the error with get_zeits.o before.

 

[xueyanzhang520@gmail_com]

Hey! Did you solve it?