Welcome to the new DiscussCESM forum!
We are still working on the website migration, so you may experience downtime during this process.

Existing users, please reset your password before logging in here: https://xenforo.cgd.ucar.edu/cesm/index.php?lost-password/

postprocessing timeseries has probems with monthly history files

TriD

Rajashree Tri Datta
New Member
Dear CESM list,

I’m having some trouble with the creation of timeseries based on the postprocessing code and I’m hoping you can help. It appears to be working just fine for the 3-hourly and daily history files, but with very similar configurations for monthly history files, the logs suggest that it’s chugging away with creating a timeseries, but nothing is actually output to the folders.

Is there something in the xml that I’m missing here such that it’s pointing it to another directory?

Thanks in advance,

R. Tri Datta


Xml entry for 3 hourly timeseries (working):
<file_extension suffix=".h2.[0-9]">

<subdir>hist</subdir>

<tseries_create>TRUE</tseries_create>

<tseries_output_format>netcdf4c</tseries_output_format>

<tseries_tper>hour_3</tseries_tper>

<tseries_filecat_tper>years</tseries_filecat_tper>

<tseries_filecat_n>5</tseries_filecat_n>

</file_extension>


Xml entry for monthly timeseries (NOT working):

<file_extension suffix=".h0.[0-9]">

<subdir>hist</subdir>

<tseries_create> TRUE </tseries_create>

<tseries_output_format>netcdf4c</tseries_output_format>

<tseries_tper>month_1</tseries_tper>

<tseries_filecat_tper>years</tseries_filecat_tper>

<tseries_filecat_n>50</tseries_filecat_n>

</file_extension>
 

strandwg

Member
Dear CESM list,

I’m having some trouble with the creation of timeseries based on the postprocessing code and I’m hoping you can help. It appears to be working just fine for the 3-hourly and daily history files, but with very similar configurations for monthly history files, the logs suggest that it’s chugging away with creating a timeseries, but nothing is actually output to the folders.

Is there something in the xml that I’m missing here such that it’s pointing it to another directory?

Thanks in advance,

R. Tri Datta


Xml entry for 3 hourly timeseries (working):
<file_extension suffix=".h2.[0-9]">

<subdir>hist</subdir>

<tseries_create>TRUE</tseries_create>

<tseries_output_format>netcdf4c</tseries_output_format>

<tseries_tper>hour_3</tseries_tper>

<tseries_filecat_tper>years</tseries_filecat_tper>

<tseries_filecat_n>5</tseries_filecat_n>

</file_extension>


Xml entry for monthly timeseries (NOT working):

<file_extension suffix=".h0.[0-9]">

<subdir>hist</subdir>

<tseries_create> TRUE </tseries_create>

<tseries_output_format>netcdf4c</tseries_output_format>

<tseries_tper>month_1</tseries_tper>

<tseries_filecat_tper>years</tseries_filecat_tper>

<tseries_filecat_n>50</tseries_filecat_n>

</file_extension>

Are the spaces in around TRUE in "<tseries_create> TRUE </tseries_create>" a potential problem? I wouldn't think so, but that's the only difference I see between the two sets.
 

TriD

Rajashree Tri Datta
New Member
Hi Gary,

Apologies for the late response. I'm afraid that changing the spaces made no difference. I think that the timeseries postprocessing has some known issues, but the thing is that it IS working for 3_hourly data under the condition that the variable is 1-D (variables at multiple levels have problems). I don't know if this helps isolate the particular bug.
 

jedwards

CSEG and Liaisons
Staff member
Please try again using the code in /glade/u/home/jedwards/sandboxes/CESM_postprocessing
or checkout branch cheyenne_update_py3

and let me know if you still have the same issue.
 

TriD

Rajashree Tri Datta
New Member
Hi Jim, I tried both codebases (just to be sure) and I'm getting the following error. Is there a newer codebase?

Traceback (most recent call last):


File "./cesm_tseries_generator.py", line 406, in <module>
debugMsg('Running on {0} cores'.format(size), header=True)
File "/glade/work/jedwards/sandboxes/CESM_postprocessing/cesm-env2/lib/python3.7/site-packages/asaptools/vprinter.py", line 109, in __call__
if verbosity < self.verbosity:
TypeError: '<' not supported between instances of 'int' and 'list'
 

TriD

Rajashree Tri Datta
New Member
Hi Jim, I tried the new codebase and it looks like it's creating the folders, but not actually depositing any files there. Is there maybe something wrong with my setup? As an example snippet (full file in :

<file_extension suffix=".h1.[0-9]">
<subdir>hist</subdir>
<tseries_create>TRUE</tseries_create>
<tseries_output_format>netcdf4c</tseries_output_format>
<tseries_tper>month_1</tseries_tper>
<tseries_filecat_tper>years</tseries_filecat_tper>
<tseries_filecat_n>10</tseries_filecat_n>
</file_extension>
<file_extension suffix=".h2.[0-9]">
<subdir>hist</subdir>
<tseries_create>TRUE</tseries_create>
<tseries_output_format>netcdf4c</tseries_output_format>
<tseries_tper>hour_3</tseries_tper>
<tseries_filecat_tper>years</tseries_filecat_tper>
<tseries_filecat_n>1</tseries_filecat_n>
</file_extension>

From the logs:
42/144 opening /glade/scratch/rtdatta/archive/cesm2.2.0_FHIST_ne0np4.ANTSI.ne30x4_mt12_3600pes_210420_Nx4yR/lnd/hist/cesm2.2.0_FHIST_ne0np4.ANTSI.ne30x4_mt12_3600pes_210420_Nx4yR.clm2.h0.2005-08.nc
142/144 opening /glade/scratch/rtdatta/archive/cesm2.2.0_FHIST_ne0np4.ANTSI.ne30x4_mt12_3600pes_210420_Nx4yR/lnd/hist/cesm2.2.0_FHIST_ne0np4.ANTSI.ne30x4_mt12_3600pes_210420_Nx4yR.clm2.h2.1985-12-27-00000.nc
time_period_freq = hour_3
 

oleson

Keith Oleson
CSEG and Liaisons
Staff member
Jim asked me to look at this since I frequently run the timeseries postprocessing on land files.
I tried it using my own sandbox (/glade/work/oleson/postprocessing) and it worked fine.
My sandbox is fairly old but it should be fine for creating timeseries files, so you could point to it if you want.

I also tried JIm's sandbox (/glade/work/jedwards/sandboxes/CESM_postprocessing) and, as you described, it did find the history files and it created the subdirectories and is running, but didn't create any files.
The only possible error I see in the log file is:

Verbosity level must be an integer.
 
Top