Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

Plumber 2: missing surfdata files. How to create them?

ijaguirre

Ignacio Aguirre
New Member
Hi,

I'm attempting to run the Plumber 2 sites using CLM 5.3 (the latest version) using the ./run_tower capabilities. However, some towers (listed below) are missing the surfdata files, and therefore, their simulations cannot be conducted.

The stations missing surfdata files are:
  • BE-Lon
  • ES-ES2
  • IE-Ca1
  • IT-BCi
  • IT-CA2
  • DE-Geb
  • US-Ne1
  • DE-Kli
  • US-Ne1
  • US-Ne2
  • US-Ne3
  • DE-Seh
  • DK-Ris
  • US-Twt
  • US-ARM
  • US-Bo1
By default, these files are being searched in /glade/campaign/cesm/cesmdata/inputdata/lnd/clm2/surfdata_esmf/PLUMBER2/ctsm5.3.0/. Unfortunately, there are no files associated with these stations in that folder.

In this context, I am wondering: (a) Do you have the surfdata files associated with these stations?

If not, (b) I noticed that these files are available for ctsm5.2 (/glade/campaign/cesm/cesmdata/cseg/inputdata/lnd/clm2/surfdata_esmf/PLUMBER2/). Is there a way to automatically update them from version 5.2 to 5.3? I received a warning that the files are different, given the CLM version.

If not, (c) How can I create these files in a way that is consistent with the other stations? I noticed the existence of a script (CTSM/python/ctsm/subset_data.py at master · ESCOMP/CTSM), and I am wondering if you have any documentation on how the previous surfdata files were created to ensure consistency in the creation of the missing ones.

I tried to use the script, but it failed because the sample longitude value I entered was of the class Longitude, not a float. See below.

Code:
(ctsm_pylib) iaguirre@derecho7:~/glade_work/clm_version/develop_15062025/CTSM/python/ctsm> python subset_data.py point --create-surface --lat -33 --lon -70
WARNING: No dominant pft type is chosen. If you want to choose a dominant pft type, please use --dompft flag.
Traceback (most recent call last):
  File "subset_data.py", line 893, in <module>
    main()
  File "subset_data.py", line 888, in main
    subset_point(args, file_dict)
  File "subset_data.py", line 730, in subset_point
    specify_fsurf_out=file_dict["fsurf_out"],
  File "/glade/u/home/iaguirre/glade_work/clm_version/develop_15062025/CTSM/python/ctsm/site_and_regional/single_point_case.py", line 502, in create_surfdata_at_point
    f_tmp = f_in.sel(lsmlon=self.plon, lsmlat=self.plat, method="nearest")
  File "/glade/work/iaguirre/conda-envs/ctsm_pylib/lib/python3.7/site-packages/xarray/core/dataset.py", line 2231, in sel
    self, indexers=indexers, method=method, tolerance=tolerance
  File "/glade/work/iaguirre/conda-envs/ctsm_pylib/lib/python3.7/site-packages/xarray/core/coordinates.py", line 417, in remap_label_indexers
    obj, v_indexers, method=method, tolerance=tolerance
  File "/glade/work/iaguirre/conda-envs/ctsm_pylib/lib/python3.7/site-packages/xarray/core/indexing.py", line 269, in remap_label_indexers
    label = maybe_cast_to_coords_dtype(label, coords_dtype)
  File "/glade/work/iaguirre/conda-envs/ctsm_pylib/lib/python3.7/site-packages/xarray/core/utils.py", line 77, in maybe_cast_to_coords_dtype
    label = np.asarray(label, dtype=coords_dtype)
  File "/glade/work/iaguirre/conda-envs/ctsm_pylib/lib/python3.7/site-packages/numpy/core/_asarray.py", line 83, in asarray
    return array(a, dtype, copy=False, order=order)
TypeError: float() argument must be a string or a number, not 'Longitude'


Thank you very much in advance,



What version of the code are you using?
ctsm5.3.057
 

slevis

Moderator
Staff member
The default search seems wrong, because the files are here:
/glade/campaign/cesm/cesmdata/cseg/inputdata/lnd/clm2/surfdata_esmf/PLUMBER2
 

ijaguirre

Ignacio Aguirre
New Member
Hi

Thank you for the reply. I attempted to run CLM5.3 with the file from the aforementioned folder; however, I received a warning indicating a mismatch between the version associated with the file (CLM 5.2) and the version of my simulation (CLM 5.3). See below.

Code:
 Opened existing file
 /glade/campaign/cesm/cesmdata/cseg/inputdata/lnd/clm2/surfdata_esmf/PLUMBER2/su
 rfdata_1x1_PLUMBER2_BE-Lon_hist_2000_16pfts_c240326.nc         150
 Input surface dataset is:
 /glade/campaign/cesm/cesmdata/cseg/inputdata/lnd/clm2/surfdata_esmf/PLUMBER2/su
 rfdata_1x1_PLUMBER2_BE-Lon_hist_2000_16pfts_c240326.nc
This surface dataset is older and incompatible with this version of CTSM
Dataset version = 5.2 Version expected = 5.3
 ERROR in surfrdMod.F90 at line 183                                             
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                      
 ENDRUN:
 ERROR: Incompatible surface dataset

There is another folder,/glade/campaign/cesm/cesmdata/cseg/inputdata/lnd/clm2/surfdata_esmf/PLUMBER2/ctsm5.3.0; however, many stations (e.g., BE-Lon) are missing from this folder.

In this regard,
(a) Do you have the surfdata files associated with these stations?
(b) Is there a way to automatically update them from version 5.2 to 5.3?
(c) How can I create these files in a way that is consistent with the other stations?

Thank you in advance,
 

slevis

Moderator
Staff member
My mistake. So this directory
/glade/campaign/cesm/cesmdata/cseg/inputdata/lnd/clm2/surfdata_esmf/PLUMBER2
contains the 5.2 files
and this directory
/glade/campaign/cesm/cesmdata/cseg/inputdata/lnd/clm2/surfdata_esmf/PLUMBER2/ctsm5.3.0
contains the 5.3 files.

- I do not know why the 5.3 list does not include the same sites as 5.2.
- I am not aware of an automated way of updating files from 5.2 to 5.3.
- You could compare 5.3 versus 5.2 files for sites that have both. Then try to manually implement the same changes to the 5.2 files that you want to convert.
 

ijaguirre

Ignacio Aguirre
New Member
Hi
As an update, there is code to generate all the surfdata files located in /CTSM/tools/site_and_regional/plumber2_surf_wrapper
Once a Python environment is activated, this code will generate the surfdata files for all the Plumber 2 stations in the folder provided by the user or a subfolder created in /CTSM/tools/site_and_regional/.
The updated code is currently available at GitHub - ESCOMP/CTSM at b4b-dev and was updated by Sam Rabin (@samrabin) recently. I tested the code using CTSM 5.3, and the simulations ran as expected.
 

krocci

Katie Rocci
New Member
Hi Ignacio and Sam,

I am running into the same issue with longitude and the subset_data python script. This scripts works fine with my pre 5.3 runs but not when using CTSM 5.3.049. Ignacio, did you figure out this error? Or Sam, do you have insight on this error?

Thanks!
Katie

Error message from subset_data (same as Ignacio's):

(npl-2024b) krocci@derecho6:/glade/work/krocci/CTSM5.3.049/tools/site_and_regional> ./subset_data point --lat 45.402252 --lon 267.201915 --lon-type 360 --site ccc3_crujra --create-surface --create-datm -
-datm-syr 2000 --datm-eyr 2022 --dompft 13 --create-user-mods --outdir /glade/derecho/scratch/$USER/cc
c3_crujra
Traceback (most recent call last):
File "/glade/work/krocci/CTSM5.3.049/tools/site_and_regional/./subset_data", line 37, in <module>
main()
File "/glade/work/krocci/CTSM5.3.049/tools/site_and_regional/../../python/ctsm/subset_data.py", line 885, in main
subset_point(args, file_dict)
File "/glade/work/krocci/CTSM5.3.049/tools/site_and_regional/../../python/ctsm/subset_data.py", line 723, in subset_point
single_point.create_surfdata_at_point(
File "/glade/work/krocci/CTSM5.3.049/tools/site_and_regional/../../python/ctsm/site_and_regional/single_point_case.py", line 502, in create_surfdata_at_point
f_tmp = f_in.sel(lsmlon=self.plon, lsmlat=self.plat, method="nearest")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/glade/u/apps/opt/conda/envs/npl-2024b/lib/python3.11/site-packages/xarray/core/dataset.py", line 3126, in sel
query_results = map_index_queries(
^^^^^^^^^^^^^^^^^^
File "/glade/u/apps/opt/conda/envs/npl-2024b/lib/python3.11/site-packages/xarray/core/indexing.py", line 192, in map_index_queries
results.append(index.sel(labels, **options))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/glade/u/apps/opt/conda/envs/npl-2024b/lib/python3.11/site-packages/xarray/core/indexes.py", line 765, in sel
label_array = normalize_label(label, dtype=self.coord_dtype)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/glade/u/apps/opt/conda/envs/npl-2024b/lib/python3.11/site-packages/xarray/core/indexes.py", line 545, in normalize_label
value = np.asarray(value, dtype=dtype)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: float() argument must be a string or a real number, not 'Longitude'
 

samrabin

Sam Rabin
Member
Hi @krocci ,

The fixed code is available on the b4b-dev branch that Ignacio linked. It will also be merged into the master branch later this week as part of tag ctsm5.3.061. Please let me know if you still run into issues!
 

krocci

Katie Rocci
New Member
Thanks @samrabin !

That mostly worked. The only issue I ran into is that the subset script seemed to grab a weird value for longitude. When I tried to create a case using the new user_mods, it threw an error with the shell_commands of the user_mods. I was able to fix this by manually changing the longitude in the shell_commands but just a heads up in case this is a broader issue than just for me.

Here's the error form the create_newcase call:
krocci@derecho3:/glade/work/krocci/CTSM5.3.049/cime/scripts> ./create_newcase --case /glade/work/krocci/CedarCreek/MIMICS/CC_mimics_CLM5.3.049_100yr_fix --res CLM_USRDAT --compset I2000Clm60Bgc --run-unsupported --user-mods-dirs
/glade/derecho/scratch/krocci/ccc3_crujra/user_mods/
Compset longname is 2000_DATM%GSWP3v1_CLM60%BGC_SICE_SOCN_MOSART_SGLC_SWAV
Compset specification file is /glade/work/krocci/CTSM5.3.049/cime_config/config_compsets.xml
Automatically adding SESP to compset
Compset forcing is 1972-2004
ATM component is Data driven ATM GSWP3v1 data set
LND component is clm6.0:BGC (vert. resol. CN and methane):
ICE component is Stub ice component
OCN component is Stub ocn component
ROF component is MOSART: MOdel for Scale Adaptive River Transport
GLC component is Stub glacier (land ice) component
WAV component is Stub wave component
ESP component is Stub external system processing (ESP) component
Pes specification file is /glade/work/krocci/CTSM5.3.049/cime_config/config_pes.xml
Compset specific settings: name is RUN_STARTDATE and value is 2000-01-01
Machine is derecho
Pes setting: grid match is l%1x1|l%CLM_USRDAT
Pes setting: grid is a%CLM_USRDAT_l%CLM_USRDAT_oi%null_r%null_g%null_w%null_z%null_m%null
Pes setting: compset is 2000_DATM%GSWP3v1_CLM60%BGC_SICE_SOCN_MOSART_SGLC_SWAV_SESP
Pes setting: tasks is {'NTASKS_ATM': 1, 'NTASKS_LND': 1, 'NTASKS_ROF': 1, 'NTASKS_ICE': 1, 'NTASKS_OCN': 1, 'NTASKS_GLC': 1, 'NTASKS_WAV': 1, 'NTASKS_CPL': 1}
Pes setting: threads is {'NTHRDS_ATM': 1, 'NTHRDS_LND': 1, 'NTHRDS_ROF': 1, 'NTHRDS_ICE': 1, 'NTHRDS_OCN': 1, 'NTHRDS_GLC': 1, 'NTHRDS_WAV': 1, 'NTHRDS_CPL': 1}
Pes setting: rootpe is {'ROOTPE_ATM': 0, 'ROOTPE_LND': 0, 'ROOTPE_ROF': 0, 'ROOTPE_ICE': 0, 'ROOTPE_OCN': 0, 'ROOTPE_GLC': 0, 'ROOTPE_WAV': 0, 'ROOTPE_CPL': 0}
Pes setting: pstrid is {}
Pes other settings: {}
Pes other settings append: {}
Pes comments: none
setting additional fields from config_pes: {}, append {}
Compset is: 2000_DATM%GSWP3v1_CLM60%BGC_SICE_SOCN_MOSART_SGLC_SWAV_SESP
Grid is: a%CLM_USRDAT_l%CLM_USRDAT_oi%null_r%null_g%null_w%null_z%null_m%null
Components in compset are: ['datm', 'clm', 'sice', 'socn', 'mosart', 'sglc', 'swav', 'sesp']
Using project from .cesm_proj: P93300041
No charge_account info available, using value from PROJECT
cesm model version found: ctsm5.3.049
Batch_system_type is pbs
job is case.run USER_REQUESTED_WALLTIME None USER_REQUESTED_QUEUE None WALLTIME_FORMAT %H:%M:%S
job is case.st_archive USER_REQUESTED_WALLTIME None USER_REQUESTED_QUEUE None WALLTIME_FORMAT %H:%M:%S
job is case.cupid USER_REQUESTED_WALLTIME None USER_REQUESTED_QUEUE None WALLTIME_FORMAT %H:%M:%S
Creating Case directory /glade/work/krocci/CedarCreek/MIMICS/CC_mimics_CLM5.3.049_100yr_fix
Adding user mods directory /glade/derecho/scratch/krocci/ccc3_crujra/user_mods
RUN: /glade/work/krocci/CedarCreek/MIMICS/CC_mimics_CLM5.3.049_100yr_fix/shell_commands
FROM: /glade/work/krocci/CedarCreek/MIMICS/CC_mimics_CLM5.3.049_100yr_fix
stat: 2

errput: /glade/work/krocci/CedarCreek/MIMICS/CC_mimics_CLM5.3.049_100yr_fix/shell_commands: line 6: syntax error near unexpected token `newline'
/glade/work/krocci/CedarCreek/MIMICS/CC_mimics_CLM5.3.049_100yr_fix/shell_commands: line 6: `./xmlchange --force PTS_LON=<ctsm.longitude.Longitude object at 0x7f34cb836a90>'

Failed to setup case, removing /glade/work/krocci/CedarCreek/MIMICS/CC_mimics_CLM5.3.049_100yr_fix
Use --debug to force me to keep caseroot
ERROR: Command: '/glade/work/krocci/CedarCreek/MIMICS/CC_mimics_CLM5.3.049_100yr_fix/shell_commands' failed with error '/glade/work/krocci/CedarCreek/MIMICS/CC_mimics_CLM5.3.049_100yr_fix/shell_commands: line 6: syntax error near unexpected token `newline'
/glade/work/krocci/CedarCreek/MIMICS/CC_mimics_CLM5.3.049_100yr_fix/shell_commands: line 6: `./xmlchange --force PTS_LON=<ctsm.longitude.Longitude object at 0x7f34cb836a90>'' from dir '/glade/work/krocci/CedarCreek/MIMICS/CC_mimics_CLM5.3.049_100yr_fix'
 

ijaguirre

Ignacio Aguirre
New Member
Just as a reference, this is my call to the function:
Code:
(ctsm_pylib) iaguirre@derecho3:/glade/work/iaguirre/clm_version/develop_15062025/CTSM/tools/site_and_regional> ./subset_data point --lat 50.55159 --lon 4.74613 --lon-type 180 --site 1x1_PLUMBER2_BE-Lon --dompft 15 0 --pctpft 100.0 0 --cth 0.5 0 --cbh 0.01 0 --create-surface --uniform-snowpack --cap-saturation --verbose --overwrite

I also provided this ./xmlchange --force PTS_LON manually:
Code:
./xmlchange --force PTS_LON=11.3175
 

krocci

Katie Rocci
New Member
For clarification, here is the subset_data call that requires the manual workaround: ./subset_data point --lat 45.402252 --lon 267.201915 --site ccc3_crujra --create-surface --create-datm --datm-syr 2000 --datm-eyr 2022 --dompft 13 --create-user-mods --outdir /glade/derecho/scratch/$USER/ccc3_crujra

Using this subsetted data in a single point case using CLM 5.3.049 is throwing a CNBalanceCheckMod error (see below) for either a CLM-BGC AD case with the default soil model or a CLM-BGC cold start case with the MIMICS soil model. Using the same setup with an older version of CLM (ctsm5.2.028) different subsetted data for the same site did not have a CNBalanceCheckMod error. Not sure if this is related or not.

Here are the case calls I used for that:
./create_newcase --case /glade/work/krocci/CedarCreek/MIMICS/CC_mimics_CLM5.3.049_100yr_fix --res CLM_USRDAT --compset I2000Clm60Bgc --run-unsupported --user-mods-dirs /glade/derecho/scratch/krocci/ccc3_crujra/user_mods/
./create_newcase --case /glade/work/krocci/CedarCreek/base_cases/clmcentury_ccc3_AD_CLM5.3.049_fix --res CLM_USRDAT --compset I2000Clm60Bgc --run-unsupported --user-mods-dirs /glade/derecho/scratch/krocci/ccc3_crujra/user_mods/


CNBalanceCheckMod error:
cbalance warning at c = 1 -1.311777934193126E-007
1088.18627377548
column cbalance error = -1.311777934193126E-007 1
is fates column? = F
Latdeg,Londeg= 45.4022520000000 267.201915000000
begcb = 1088.21015726758
endcb = 1088.18627377548
delta store = -2.388349209627449E-002
--- Inputs ---
gpp = 1.689101673261114E-004
--- Outputs ---
er = 2.404915492863770E-002
col_fire_closs = 3.378512728450417E-006
col_hrv_xsmrpool_to_atm = 0.000000000000000E+000
col_xsmrpool_to_atm = 0.000000000000000E+000
wood_harvestc = 0.000000000000000E+000
crop_harvestc_to_cropprodc = 0.000000000000000E+000
-1*som_c_leached = -2.787123374655006E-014
iam = 0: local column index = 1
iam = 0: global column index = 1
iam = 0: global landunit index = 1
iam = 0: global gridcell index = 1
iam = 0: gridcell longitude = 267.2019150
iam = 0: gridcell latitude = 45.4022520
iam = 0: column type = 1
iam = 0: landunit type = 1
ENDRUN:
ERROR in CNBalanceCheckMod.F90 at line 384
 

krocci

Katie Rocci
New Member
An update on the CNBalanceCheckMod issue: this was unrealted to the subsetting. The older version of CLM (5.2.028) had a much higher error tolerance and when I match that error tolerance in the newer version of CLM (5.3.049), my cases run. This might be related to changes needed to do MIMICS spinup - I will check with Will when he is back.
 
Top