This report may be related to this issue from 2013: https://bb.cgd.ucar.edu/output-date-error-monthly-history-files-shifted-1-monthMany people may have discovered independently that CESM timestamps are placed at the end of the month (or other averaging period) , and that external tools like CDO and xarray do not treat this correctly. It is kind of cumbersome to deal with this so a long term solution should be thought about. Perhaps the external tools developers can be contacted and urged to adhere to time_bounds better / at all.CDO
shows that CDO interprets the first time stamp as 1850-01-31 23:59:59, which is in principle okay.Newer version of CDO changed this:
reveals the first time stamp is decoded as 1850-02-01 00:00:00.In both cases, selecting e.g. a single year from the timeseries is not done correctly:
Time coordinate : 11 steps
RefTime = 1850-01-01 00:00:00 Units = days Calendar = 365_day Bounds = true
YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss
1850-01-31 23:59:59 1850-02-28 23:59:59 1850-03-31 23:59:59 1850-04-30 23:59:59
1850-05-31 23:59:59 1850-06-30 23:59:59 1850-07-31 23:59:59 1850-08-31 23:59:59
1850-09-30 23:59:59 1850-10-31 23:59:59 1850-11-30 23:59:59
A workaround is to introduce a time shift of one day:
cdo -L selyear,1850 -shifttime,-1d $FILE $SCRATCH/1850.nc
xarraySome example code showing that 11 values are returned when the year 1850 is selected:
The workaround is the same as with CDO, shifting the time coordinate by one day:
Code:
cdo -V # Climate Data Operators version 1.7.2 (http://mpimet.mpg.de/cdo)<br />FILE=/gpfs/fs1/collections/cdg/timeseries-cmip6/b.e21.BHIST.f09_g17.CMIP6-historical.001/lnd/proc/tseries/month_1/b.e21.BHIST.f09_g17.CMIP6-historical.001.clm2.h0.FSR.185001-201412.nc<br />cdo sinfo $FILE
Code:
module load cdo/1.9.4<br />cdo -V # Climate Data Operators version 1.9.4 (http://mpimet.mpg.de/cdo)<br />cdo sinfo $FILE
Code:
cdo selyear,1850 $FILE $SCRATCH/1850.nc<br />cdo sinfo $SCRATCH/1850.ncRefTime = 1850-01-01 00:00:00 Units = days Calendar = 365_day Bounds = true
YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss
1850-01-31 23:59:59 1850-02-28 23:59:59 1850-03-31 23:59:59 1850-04-30 23:59:59
1850-05-31 23:59:59 1850-06-30 23:59:59 1850-07-31 23:59:59 1850-08-31 23:59:59
1850-09-30 23:59:59 1850-10-31 23:59:59 1850-11-30 23:59:59
A workaround is to introduce a time shift of one day:
cdo -L selyear,1850 -shifttime,-1d $FILE $SCRATCH/1850.nc
xarraySome example code showing that 11 values are returned when the year 1850 is selected:
Code:
execdav # Python code base does not work on Cheyenne<br />export PATH=/glade/u/home/lvank/miniconda3/bin:$PATH<br />source activate base <br />ipython<br />> import xarray as xr<br />> file="/gpfs/fs1/collections/cdg/timeseries-cmip6/b.e21.BHIST.f09_g17.CMIP6-historical.001/lnd/proc/tseries/month_1/b.e21.BHIST.f09_g17.CMIP6-historical.001.clm2.h0.FSR.185001-201412.nc"<br />> ds=xr.open_dataset(file)<br />> ds.time<br />> ds1850 = ds.sel(time=slice('1850','1850'))<br />> ds1850.time<br /><xarray.DataArray 'time' (time: 11)><br />array(['1850-02-01T00:00:00.000000000', '1850-03-01T00:00:00.000000000',<br /> '1850-04-01T00:00:00.000000000', '1850-05-01T00:00:00.000000000',<br /> '1850-06-01T00:00:00.000000000', '1850-07-01T00:00:00.000000000',<br /> '1850-08-01T00:00:00.000000000', '1850-09-01T00:00:00.000000000',<br /> '1850-10-01T00:00:00.000000000', '1850-11-01T00:00:00.000000000',<br /> '1850-12-01T00:00:00.000000000'], dtype='datetime64[ns]')
Code:
<br />> from datetime import timedelta<br />> time_index_shifted = ds.time.get_index('time') - timedelta(days=1)<br />> ds['time'] = time_index_shifted