Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

ADF diagnostic error ERROR NC_EHDFERR Error at HDF5 layer

Hi everyone

I am trying to run the ADF diagnostic package in CASPER. ( git clone GitHub - NCAR/ADF: A unified collection of python scripts used to generate standard plots from CAM outputs.),
After clone the package in my Work directory, load all the modules and npl environment modify the config_cam_baseline_example.yaml and config_amwg_default_plots.yaml.
I get an error but is something related to nco and hdf5 library I think:

ERROR NC_EHDFERR Error at HDF5 layer
HINT: NC_EHDFERR errors indicate that the HDF5-backend to netCDF is unable to perform the requested task. NCO can receive this devilishly inscrutable error for a variety of possible reasons including: 1) The run-time dynamic linker attempts to resolve calls from the netCDF library to the HDF library with an HDF5 libhdf5.a that is incompatible with the version used to build NCO and netCDF. 2) The file system does not allow the HDF5 flock() function, as of HDF5 1.10.x, to enable multiple processes to open the same file for reading, a feature known as SWMR (Single Write Multiple Read). The fix is to disable the HDF5 flock() by setting an environment variable thusly: "export HDF5_USE_FILE_LOCKING=FALSE". 3) An incorrect netCDF4 library implementation of a procedure (e.g., nc_rename_var()) in terms of HDF function calls (e.g., HDF5Lmove()) manifests an error or inconsistent state within the HDF5 layer. This often occurs during renaming operations (Current issues renaming dimensions and coordinates · Issue #597 · Unidata/netcdf-c). 4) Attempting to compress or decompress a netCDF4 dataset with a non-standard (i.e., non-DEFLATE) filter when the requisite shared library to encode/decode that compression filter is not present in either the default location (/usr/local/hdf5/lib/plugin) or in the user-configurable location referred to by the HDF5_PLUGIN_PATH environment variable. One can determine if missing plugin libraries are the culprit by dumping the hidden attributes of the dataset with, e.g., ncks --hdn -m in.nc or ncdump -s -h in.nc. Any variables with (hidden) "_Filter" attributes require the corresponding shared libraries to be located in HDF5_PLUGIN_PATH. Some HDF5 implementations (at least MacOSX with MacPorts as of 20200907) may also require explicitly setting the plugin path in the environment, even for the default location! To test this, re-try your NCO command after doing this: "export HDF5_PLUGIN_PATH=/usr/local/hdf5/lib/plugin". 5) Bad vibes.
nco_err_exit(): ERROR Short NCO-generated message (usually name of function that triggered error): nco_enddef()
nco_err_exit(): ERROR Error code is -101. Translation into English with nc_strerror(-101) is "NetCDF: HDF error"
nco_err_exit(): ERROR NCO will now exit with system call exit(EXIT_FAILURE)



Could anyone tell me which is the problem??

Thanks in advances

Adriana
 

nusbaume

Jesse Nusbaumer
CSEG and Liaisons
Staff member
Hi Adriana,

Apologies for the slow response! Would you be able to share the config (yaml) file you used that resulted in that error? Also were you attempting to run the ADF on an NCAR machine, or somewhere else?

Thanks, and have a great day!

Jesse
 
Hi Jesse!

Thank you very much for your answer. I was able to resolve the NC ERROR, I think was a problem of disk space,
I am running the ADF package in casper.
Now start to work but I get an error when the package start to do the polar map:

Generating polar maps...
NOTE: Plot type is set to png
NOTE: redo_plot is set to False
- polar maps for SWCF
Traceback (most recent call last):
File "/glade/work/abossolasco/Work/ADF/./run_adf_diag", line 185, in <module>
diag.create_plots()
File "/glade/work/abossolasco/Work/ADF/lib/adf_diag.py", line 846, in create_plots
self.__diag_scripts_caller(
File "/glade/work/abossolasco/Work/ADF/lib/adf_diag.py", line 300, in __diag_scripts_caller
self.__function_caller(
File "/glade/work/abossolasco/Work/ADF/lib/adf_diag.py", line 338, in __function_caller
return func(self)
File "/glade/work/abossolasco/Work/ADF/lib/../scripts/plotting/polar_map.py", line 282, in polar_map
pf.make_polar_plot(plot_name, case_nickname, base_nickname,
File "/glade/work/abossolasco/Work/ADF/lib/plotting_functions.py", line 806, in make_polar_plot
ax2.set_ylabel(f"{dif.units}")
File "/glade/u/apps/opt/conda/envs/npl/lib/python3.10/site-packages/xarray/core/common.py", line 278, in __getattr__
raise AttributeError(


I attached here my config*.yaml files
 

nusbaume

Jesse Nusbaumer
CSEG and Liaisons
Staff member
Hi Adriana,

Glad you were able to figure out the first error! Sadly it looks like the config files didn't attach properly? Alternatively if you can just point me to their location on glade (assuming you are trying the ADF on an NCAR machine) then I should be able to find them that way instead, as long as the file/location is globally readable.

Thanks!

Jesse
 
Top