Date inconsistency in RIVER_DISCHARGE_OVER_LAND_LIQ variable file name and data

[jjoseph]

Hello CGD Team:

I trying to combine and find the annual maximum value for RIVER_DISCHARGE_OVER_LAND_LIQ variable using the following Python code in CASPER

## Resource: CASPER
## Needs more than 25 GB RAM and around 20 minutes

## Import packages
import xarray as xr

## Campaign Directory with daily data
path='/glade/campaign/cgd/cesm/CESM2-LE/rof/proc/tseries/day_1/RIVER_DISCHARGE_OVER_LAND_LIQ/'

## Simulation Name
sim_name='BHISTcmip6'
sim_num=1231
counter_glade=1

## Combine the data
ds=xr.open_mfdataset([path+f'b.e21.{sim_name}.f09_g17.LE2-{sim_num}.{counter_glade:03}.mosart.h1.RIVER_DISCHARGE_OVER_LAND_LIQ.18500101-18600101.nc',
path+f'b.e21.{sim_name}.f09_g17.LE2-{sim_num}.{counter_glade:03}.mosart.h1.RIVER_DISCHARGE_OVER_LAND_LIQ.18600102-18700101.nc',
path+f'b.e21.{sim_name}.f09_g17.LE2-{sim_num}.{counter_glade:03}.mosart.h1.RIVER_DISCHARGE_OVER_LAND_LIQ.18700102-18800101.nc',
path+f'b.e21.{sim_name}.f09_g17.LE2-{sim_num}.{counter_glade:03}.mosart.h1.RIVER_DISCHARGE_OVER_LAND_LIQ.18800102-18900101.nc',
path+f'b.e21.{sim_name}.f09_g17.LE2-{sim_num}.{counter_glade:03}.mosart.h1.RIVER_DISCHARGE_OVER_LAND_LIQ.18900102-19000101.nc',
path+f'b.e21.{sim_name}.f09_g17.LE2-{sim_num}.{counter_glade:03}.mosart.h1.RIVER_DISCHARGE_OVER_LAND_LIQ.19000102-19100101.nc',
path+f'b.e21.{sim_name}.f09_g17.LE2-{sim_num}.{counter_glade:03}.mosart.h1.RIVER_DISCHARGE_OVER_LAND_LIQ.19100102-19200101.nc',
path+f'b.e21.{sim_name}.f09_g17.LE2-{sim_num}.{counter_glade:03}.mosart.h1.RIVER_DISCHARGE_OVER_LAND_LIQ.19200102-19300101.nc',
path+f'b.e21.{sim_name}.f09_g17.LE2-{sim_num}.{counter_glade:03}.mosart.h1.RIVER_DISCHARGE_OVER_LAND_LIQ.19300102-19400101.nc',
path+f'b.e21.{sim_name}.f09_g17.LE2-{sim_num}.{counter_glade:03}.mosart.h1.RIVER_DISCHARGE_OVER_LAND_LIQ.19400102-19500101.nc',
path+f'b.e21.{sim_name}.f09_g17.LE2-{sim_num}.{counter_glade:03}.mosart.h1.RIVER_DISCHARGE_OVER_LAND_LIQ.19500102-19600101.nc',
path+f'b.e21.{sim_name}.f09_g17.LE2-{sim_num}.{counter_glade:03}.mosart.h1.RIVER_DISCHARGE_OVER_LAND_LIQ.19600102-19700101.nc',
path+f'b.e21.{sim_name}.f09_g17.LE2-{sim_num}.{counter_glade:03}.mosart.h1.RIVER_DISCHARGE_OVER_LAND_LIQ.19700102-19800101.nc',
path+f'b.e21.{sim_name}.f09_g17.LE2-{sim_num}.{counter_glade:03}.mosart.h1.RIVER_DISCHARGE_OVER_LAND_LIQ.19800102-19900101.nc',
path+f'b.e21.{sim_name}.f09_g17.LE2-{sim_num}.{counter_glade:03}.mosart.h1.RIVER_DISCHARGE_OVER_LAND_LIQ.19900102-20000101.nc',
path+f'b.e21.{sim_name}.f09_g17.LE2-{sim_num}.{counter_glade:03}.mosart.h1.RIVER_DISCHARGE_OVER_LAND_LIQ.20000102-20100101.nc',
path+f'b.e21.{sim_name}.f09_g17.LE2-{sim_num}.{counter_glade:03}.mosart.h1.RIVER_DISCHARGE_OVER_LAND_LIQ.20100102-20141231.nc'],
concat_dim='time')

print("Values of ds.time variable \n", ds.time.values)
print("Length of ds.time variable: \t", len(ds.time.values))
print("##########################################################")

## Resample to get annual maximum
ds_max=ds.resample(time='Y').max()
print("Values of ds_max.time variable \n", ds_max.time.values)
print("Length of ds_max.time variable: \t", len(ds_max.time.values))

The output for the above code is:

Values of ds.time variable
[cftime.DatetimeNoLeap(1850, 1, 2, 0, 0, 0, 0)
cftime.DatetimeNoLeap(1850, 1, 3, 0, 0, 0, 0)
cftime.DatetimeNoLeap(1850, 1, 4, 0, 0, 0, 0) ...
cftime.DatetimeNoLeap(2014, 12, 30, 0, 0, 0, 0)
cftime.DatetimeNoLeap(2014, 12, 31, 0, 0, 0, 0)
cftime.DatetimeNoLeap(2015, 1, 1, 0, 0, 0, 0)]
Length of ds.time variable: 60225
##########################################################
Values of ds_max.time variable
[cftime.DatetimeNoLeap(1850, 12, 31, 0, 0, 0, 0)
cftime.DatetimeNoLeap(1851, 12, 31, 0, 0, 0, 0)
cftime.DatetimeNoLeap(1852, 12, 31, 0, 0, 0, 0) ...
cftime.DatetimeNoLeap(2013, 12, 31, 0, 0, 0, 0)
cftime.DatetimeNoLeap(2014, 12, 31, 0, 0, 0, 0)
cftime.DatetimeNoLeap(2015, 12, 31, 0, 0, 0, 0)]
Length of ds_max.time variable: 166

The filename indicates that the variable ranges from 1850-01-01 to 2014-12-31. But, from the output of the above code, it can be seen that it ranges from 1850-01-02 to 2015-01-01. Is this an inconsistency? Please correct me if I am wrong in my understanding.

Also, when I try to resample it annually either by max (currently in my case), mean, or min, I have a value for 2015 which I think is not correct. I expected the length of resampled data to be 165, but the can be seen as 166 in the output. How should I go about this?

Thanks,
-Jibin

 

[slevis]

Please correct me if I'm wrong: to me it looks like your python code is reading files that you concatenated yourself in some way. If so, then we are unable to help, but you could check the contents of the files using ncdump.