Welcome to the new DiscussCESM forum!
We are still working on the website migration, so you may experience downtime during this process.

Existing users, please reset your password before logging in here: https://xenforo.cgd.ucar.edu/cesm/index.php?lost-password/

netcdf error message while running ncar cesm postprocessing tool

sur23beeb

ST
Member
I am analyzing 0.1 degree POP output using the ncar CESM postprocessing tool. I am gettng the following error when I run the ocn_diagnostics script:

-----------------------
ERROR NC_EVARSIZE One or more variable sizes violate format constraints
HINT: NC_EVARSIZE errors occur when attempting to copy or aggregate input files together into an output file that exceeds the per-file capacity of the output file format, and when trying to copy, aggregate, or define individual variables that exceed the per-variable constraints of the output file format. The per-file limit of all netCDF formats is not less than 8 EiB on modern computers, so any NC_EVARSIZE error is almost certainly due to violating a per-variable limit. Relevant limits: netCDF3 NETCDF_CLASSIC format limits fixed variables to sizes smaller than 2^31 B = 2 GiB ~ 2.1 GB, and record variables to that size per record. A single variable may exceed this limit if and only if it is the last defined variable. netCDF3 NETCDF_64BIT_OFFSET format limits fixed variables to sizes smaller than 2^32 B = 4 GiB ~ 4.2 GB, and record variables to that size per record. Any number of variables may reach, though not exceed, this size for fixed variables, or this size per record for record variables. The netCDF3 NETCDF_64BIT_DATA and netCDF4 NETCDF4 formats have no variable size limitations of real-world import. If any variable in your dataset exceeds these limits, alter the output file to a format capacious enough, either netCDF3 classic with 64-bit offsets (with -6 or --64), to PnetCDF/CDF5 with 64-bit data (with -5), or to netCDF4 (with -4 or -7). For more details, see http://nco.sf.net/nco.html#fl_fmt
-----------------------

I tried changing the file format from netcdf4c to netcdf4 (see below) in env_timeseries.xml, but get the same error message shown above. Here is the relevant part of the env_timeseries.xml file (as pasting only a snippet below):

XML:
  <comp_archive_spec name="pop">
    <rootdir>ocn</rootdir>
    <multi_instance>True</multi_instance>
    <default_calendar>noleap</default_calendar>
    <files>
      <file_extension suffix=".h.[0-9]">   
        <subdir>hist</subdir>
        <tseries_create>TRUE</tseries_create>
        <tseries_output_format>netcdf4</tseries_output_format>
        <tseries_tper>month_1</tseries_tper>
        <tseries_filecat_tper>years</tseries_filecat_tper>
        <tseries_filecat_n>50</tseries_filecat_n>

Any help is appreciated. Thanks!
 

strandwg

Moderator
Staff member
I am analyzing 0.1 degree POP output using the ncar CESM postprocessing tool. I am gettng the following error when I run the ocn_diagnostics script:

-----------------------
ERROR NC_EVARSIZE One or more variable sizes violate format constraints
HINT: NC_EVARSIZE errors occur when attempting to copy or aggregate input files together into an output file that exceeds the per-file capacity of the output file format, and when trying to copy, aggregate, or define individual variables that exceed the per-variable constraints of the output file format. The per-file limit of all netCDF formats is not less than 8 EiB on modern computers, so any NC_EVARSIZE error is almost certainly due to violating a per-variable limit. Relevant limits: netCDF3 NETCDF_CLASSIC format limits fixed variables to sizes smaller than 2^31 B = 2 GiB ~ 2.1 GB, and record variables to that size per record. A single variable may exceed this limit if and only if it is the last defined variable. netCDF3 NETCDF_64BIT_OFFSET format limits fixed variables to sizes smaller than 2^32 B = 4 GiB ~ 4.2 GB, and record variables to that size per record. Any number of variables may reach, though not exceed, this size for fixed variables, or this size per record for record variables. The netCDF3 NETCDF_64BIT_DATA and netCDF4 NETCDF4 formats have no variable size limitations of real-world import. If any variable in your dataset exceeds these limits, alter the output file to a format capacious enough, either netCDF3 classic with 64-bit offsets (with -6 or --64), to PnetCDF/CDF5 with 64-bit data (with -5), or to netCDF4 (with -4 or -7). For more details, see http://nco.sf.net/nco.html#fl_fmt
-----------------------

I tried changing the file format from netcdf4c to netcdf4 (see below) in env_timeseries.xml, but get the same error message shown above. Here is the relevant part of the env_timeseries.xml file (as pasting only a snippet below):

XML:
  <comp_archive_spec name="pop">
    <rootdir>ocn</rootdir>
    <multi_instance>True</multi_instance>
    <default_calendar>noleap</default_calendar>
    <files>
      <file_extension suffix=".h.[0-9]">  
        <subdir>hist</subdir>
        <tseries_create>TRUE</tseries_create>
        <tseries_output_format>netcdf4</tseries_output_format>
        <tseries_tper>month_1</tseries_tper>
        <tseries_filecat_tper>years</tseries_filecat_tper>
        <tseries_filecat_n>50</tseries_filecat_n>

Any help is appreciated. Thanks!

Where are you running ocn_diagnostics?
 

strandwg

Moderator
Staff member
Thanks for your reply. Can I not use netcdf3 with the 64-bit-offset setting?

I doubt that will work with 0.1 degree POP.

One thing you could try is creating timeseries files with one year in them, instead of 50 as you have specified with "tseries_filecat_tper"
 
Top