Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

netcdf error message while running ncar cesm postprocessing tool

sur23beeb

ST
Member
I am analyzing 0.1 degree POP output using the ncar CESM postprocessing tool. I am gettng the following error when I run the ocn_diagnostics script:

-----------------------
ERROR NC_EVARSIZE One or more variable sizes violate format constraints
HINT: NC_EVARSIZE errors occur when attempting to copy or aggregate input files together into an output file that exceeds the per-file capacity of the output file format, and when trying to copy, aggregate, or define individual variables that exceed the per-variable constraints of the output file format. The per-file limit of all netCDF formats is not less than 8 EiB on modern computers, so any NC_EVARSIZE error is almost certainly due to violating a per-variable limit. Relevant limits: netCDF3 NETCDF_CLASSIC format limits fixed variables to sizes smaller than 2^31 B = 2 GiB ~ 2.1 GB, and record variables to that size per record. A single variable may exceed this limit if and only if it is the last defined variable. netCDF3 NETCDF_64BIT_OFFSET format limits fixed variables to sizes smaller than 2^32 B = 4 GiB ~ 4.2 GB, and record variables to that size per record. Any number of variables may reach, though not exceed, this size for fixed variables, or this size per record for record variables. The netCDF3 NETCDF_64BIT_DATA and netCDF4 NETCDF4 formats have no variable size limitations of real-world import. If any variable in your dataset exceeds these limits, alter the output file to a format capacious enough, either netCDF3 classic with 64-bit offsets (with -6 or --64), to PnetCDF/CDF5 with 64-bit data (with -5), or to netCDF4 (with -4 or -7). For more details, see http://nco.sf.net/nco.html#fl_fmt
-----------------------

I tried changing the file format from netcdf4c to netcdf4 (see below) in env_timeseries.xml, but get the same error message shown above. Here is the relevant part of the env_timeseries.xml file (as pasting only a snippet below):

XML:
  <comp_archive_spec name="pop">
    <rootdir>ocn</rootdir>
    <multi_instance>True</multi_instance>
    <default_calendar>noleap</default_calendar>
    <files>
      <file_extension suffix=".h.[0-9]">   
        <subdir>hist</subdir>
        <tseries_create>TRUE</tseries_create>
        <tseries_output_format>netcdf4</tseries_output_format>
        <tseries_tper>month_1</tseries_tper>
        <tseries_filecat_tper>years</tseries_filecat_tper>
        <tseries_filecat_n>50</tseries_filecat_n>

Any help is appreciated. Thanks!
 

strandwg

Moderator
Staff member
I am analyzing 0.1 degree POP output using the ncar CESM postprocessing tool. I am gettng the following error when I run the ocn_diagnostics script:

-----------------------
ERROR NC_EVARSIZE One or more variable sizes violate format constraints
HINT: NC_EVARSIZE errors occur when attempting to copy or aggregate input files together into an output file that exceeds the per-file capacity of the output file format, and when trying to copy, aggregate, or define individual variables that exceed the per-variable constraints of the output file format. The per-file limit of all netCDF formats is not less than 8 EiB on modern computers, so any NC_EVARSIZE error is almost certainly due to violating a per-variable limit. Relevant limits: netCDF3 NETCDF_CLASSIC format limits fixed variables to sizes smaller than 2^31 B = 2 GiB ~ 2.1 GB, and record variables to that size per record. A single variable may exceed this limit if and only if it is the last defined variable. netCDF3 NETCDF_64BIT_OFFSET format limits fixed variables to sizes smaller than 2^32 B = 4 GiB ~ 4.2 GB, and record variables to that size per record. Any number of variables may reach, though not exceed, this size for fixed variables, or this size per record for record variables. The netCDF3 NETCDF_64BIT_DATA and netCDF4 NETCDF4 formats have no variable size limitations of real-world import. If any variable in your dataset exceeds these limits, alter the output file to a format capacious enough, either netCDF3 classic with 64-bit offsets (with -6 or --64), to PnetCDF/CDF5 with 64-bit data (with -5), or to netCDF4 (with -4 or -7). For more details, see http://nco.sf.net/nco.html#fl_fmt
-----------------------

I tried changing the file format from netcdf4c to netcdf4 (see below) in env_timeseries.xml, but get the same error message shown above. Here is the relevant part of the env_timeseries.xml file (as pasting only a snippet below):

XML:
  <comp_archive_spec name="pop">
    <rootdir>ocn</rootdir>
    <multi_instance>True</multi_instance>
    <default_calendar>noleap</default_calendar>
    <files>
      <file_extension suffix=".h.[0-9]">  
        <subdir>hist</subdir>
        <tseries_create>TRUE</tseries_create>
        <tseries_output_format>netcdf4</tseries_output_format>
        <tseries_tper>month_1</tseries_tper>
        <tseries_filecat_tper>years</tseries_filecat_tper>
        <tseries_filecat_n>50</tseries_filecat_n>

Any help is appreciated. Thanks!

Where are you running ocn_diagnostics?
 

strandwg

Moderator
Staff member
Thanks for your reply. Can I not use netcdf3 with the 64-bit-offset setting?

I doubt that will work with 0.1 degree POP.

One thing you could try is creating timeseries files with one year in them, instead of 50 as you have specified with "tseries_filecat_tper"
 
Top