Issue in reading urbantv_streams for high-resolution regional simulation

[yifanc17]

What version of the code are you using?
CTSM5.2.015

Have you made any changes to files in the source tree?
No

Describe every step you took leading up to the problem:

# create and setup case
CASEDIR="/glade/work/yifanc17/cases/GSWP3.CTSM5.2.015.ExplicitAC.California.1km.CTRL.c241023"
/glade/work/yifanc17/code/ctsm5.2.015/cime/scripts/create_newcase --case "${CASEDIR}" --compset 2000_DATM%GSWP3v1_CLM50%SP_SICE_SOCN_MOSART_SGLC_SWAV --res CLM_USRDAT --run-unsupported
./xmlchange LND_DOMAIN_MESH=/glade/work/yifanc17/02_data/cesmdata/meshdata/CONUS_1kmx1km/California_1km_lnd_mesh_c241015.nc
./xmlchange ATM_DOMAIN_MESH=/glade/work/yifanc17/02_data/cesmdata/meshdata/CONUS_1kmx1km/California_1km_lnd_mesh_c241015.nc
./xmlchange MASK_MESH=/glade/work/yifanc17/02_data/cesmdata/meshdata/CONUS_1kmx1km/California_1km_mask_mesh_c241015.nc
./xmlchange DATM_YR_START=2005,DATM_YR_END=2005,DATM_YR_ALIGN=2005
./xmlchange RUN_STARTDATE=2005-01-01,STOP_N=5,STOP_OPTION=ndays #unit test

# modify user_nl_clm (please see below detailed description)
fsurdat = '/glade/work/yifanc17/02_data/cesmdata/surfdata/CONUS_1km/surfdata_CA_1km_hist_2005_16pfts_c241015.nc'
stream_fldfilename_urbantv = '/glade/work/yifanc17/02_data/cesmdata/meshdata/CONUS_1kmx1km/CLM50_urbantv_California_1km_c241022.nc'
stream_meshfile_urbantv = '/glade/work/yifanc17/02_data/cesmdata/meshdata/CONUS_1kmx1km/California_1km_lnd_mesh_c241015.nc'
use_init_interp = .true.
n_dom_landunits = 0
n_dom_pfts = 0

calc_human_stress_indices = 'ALL'
collapse_urban         = .false.
soil_layerstruct_predefined    = '20SL_8.5m'

hist_nhtfrq            = -24
hist_mfilt             = 1
hist_dov2xy            = .true.

If this is a port to a new machine: Please attach any files you added or changed for the machine port (e.g., config_compilers.xml, config_machines.xml, and config_batch.xml) and tell us the compiler version you are using on this machine.
Please attach any log files showing error messages or other useful information.
No

Describe your problem or question:
I'm trying to run a 1km-resolution land-only simulation over California region (latxlon=1113x1113). I have followed the instruction here to create required mesh file and surfdata.

However, I kept getting the same error as below in lnd.log:

ERROR: no valid urban data for g=        32050
 landunit type:              9
 urban_valid:      T
 t_building_max:    0.000000000000000E+000
 p_ac:                                 NaN
iam = 0: local  landunit index = 478371
iam = 0: global landunit index = 38221
iam = 0: global gridcell index = 32050
iam = 0: gridcell longitude    =  242.9540100
iam = 0: gridcell latitude     =   32.2540092
iam = 0: landunit type         = 9
 ENDRUN:
 ERROR: ERROR in UrbanTimeVarType.F90 at line 334

The error was raised because t_building_max: 0.000000000000000E+000 and it couldn't pass the valid urban data check unless it is a positive number.

I have tried three sets of urbantv_streams:
1) default, 0.9x1.25:

stream_fldfilename_urbantv = '/glade/campaign/cesm/cesmdata/inputdata/lnd/clm2/urbandata/CLM50_tbuildmax_Oleson_2016_0.9x1.25_simyr1849-2106_c160923.nc'
stream_meshfile_urbantv = '/glade/campaign/cesm/cesmdata/inputdata/lnd/clm2/urbandata/CLM50_tbuildmax_Oleson_2016_0.9x1_ESMFmesh_cdf5_100621.nc'

I didn't modify urbantv_streams in the first run, after getting above error, I thought it was due to potential mismatch bewteen the 0.9x1.25 T_BUILDING_MAX data and the 1km mesh. However, it still didn't work after I changed to:
2) customized over California, 1kmx1km:

stream_fldfilename_urbantv = '/glade/work/yifanc17/02_data/cesmdata/meshdata/CONUS_1kmx1km/CLM50_urbantv_California_1km_c241022.nc'
stream_meshfile_urbantv = '/glade/work/yifanc17/02_data/cesmdata/meshdata/CONUS_1kmx1km/California_1km_lnd_mesh_c241015.nc'

This set was regridded to match with the 1km mesh, and the red point in below figure is the gridcell that raised error, where it actually has T_BUILDING_MAX=300K instead of 0 (for TBD, HD and MD).

According to the 1km mask_mesh.nc below, this gridcell seemed to be ocean, which shouldn't go through the urban valid check process.

I have also tried to fill in the default T_BUILDING_MAX with constant value 300K everywhere regardless of land/ocean and it still didn't work:
3) default filled with constant values, 0.9x1.25

stream_fldfilename_urbantv = '/glade/work/yifanc17/02_data/cesmdata/CLM50_tbuildmax_Oleson_2016_0.9x1.25_simyr1849-2106_gapfilled_c241023.nc'
stream_meshfile_urbantv = '/glade/campaign/cesm/cesmdata/inputdata/lnd/clm2/urbandata/CLM50_tbuildmax_Oleson_2016_0.9x1_ESMFmesh_cdf5_100621.nc'

Among above three test cases, urban input data were read successfully so it shouldn't have anything to do with the mesh/surfdata. I think the model was probably having issue in reading the correct urbantv_streams, otherwise it should have found the positive value (T_BUILDING_MAX=300K). However, the CaseDocs/lnd_in showed the correct path for both files. Do you have any suggestions for this error? Any advice would be greatly appreciated! Thank you!

 

[oleson]

I'm looking at this but the only idea I have so far is to look at whether elementMask in the filled 0.9x1.25 dataset mesh file you are using /glade/campaign/cesm/cesmdata/inputdata/lnd/clm2/urbandata/CLM50_tbuildmax_Oleson_2016_0.9x1_ESMFmesh_cdf5_100621.nc is 1 everywhere. I see a lot of zeroes in it? Are you still using the original one?

 

[yifanc17]

I'm looking at this but the only idea I have so far is to look at whether elementMask in the filled 0.9x1.25 dataset mesh file you are using /glade/campaign/cesm/cesmdata/inputdata/lnd/clm2/urbandata/CLM50_tbuildmax_Oleson_2016_0.9x1_ESMFmesh_cdf5_100621.nc is 1 everywhere. I see a lot of zeroes in it? Are you still using the original one?

Hi Keith, thanks for the reply! Yes I was using the original one (/glade/campaign/cesm/cesmdata/inputdata/lnd/clm2/urbandata/CLM50_tbuildmax_Oleson_2016_0.9x1_ESMFmesh_cdf5_100621.nc) where land is 1 and ocean is 0. I have tried to set elementMask to 1 everywhere and re-run the simulation but still got the same error.

 

[oleson]

I have a case here that at least ran past the read of urbantv without tbuilding_max errors:

/glade/work/oleson/ctsm5.2.015/cime/scripts/GSWP3.CTSM5.2.015.ExplicitAC.California.1km.CTRL.c241023

I used your 1km urbantv files:

stream_fldfilename_urbantv = '/glade/work/yifanc17/02_data/cesmdata/meshdata/CONUS_1kmx1km/CLM50_urbantv_California_1km_c241022.nc'
stream_meshfile_urbantv = '/glade/work/yifanc17/02_data/cesmdata/meshdata/CONUS_1kmx1km/California_1km_lnd_mesh_c241015.nc'

and set urbantvmapalgo = 'redist', which is something @bdobbins found was necessary for these high-res simulations.
To get the model to accept this you need to make the following changes:

--- a/bld/namelist_files/namelist_definition_ctsm.xml
+++ b/bld/namelist_files/namelist_definition_ctsm.xml
@@ -2029,7 +2029,7 @@ Time interpolation method to use with urban time varying streams
</entry>

<entry id="urbantvmapalgo" type="char*256" category="datasets"
- group="urbantv_streams" valid_values="bilinear,nn,nnoni,nnonj,spval,copy" >
+ group="urbantv_streams" valid_values="bilinear,nn,nnoni,nnonj,spval,copy,redist" >


--- a/src/cpl/share_esmf/UrbanTimeVarType.F90
+++ b/src/cpl/share_esmf/UrbanTimeVarType.F90
@@ -174,6 +174,7 @@ contains
call shr_mpi_bcast(stream_year_first_urbantv , mpicom)
call shr_mpi_bcast(stream_year_last_urbantv , mpicom)
call shr_mpi_bcast(model_year_align_urbantv , mpicom)
+ call shr_mpi_bcast(urbantvmapalgo , mpicom)
call shr_mpi_bcast(stream_fldFileName_urbantv , mpicom)

Unfortunately, the model died after a couple of time steps with a negative or zero downward longwave. So it seems to be having trouble interpolating the 0.5deg forcing data to the 1km grid, perhaps similar to this issue here: malloc(): invalid size (unsorted) when reading in my own forcing data

The user fixed this by interpolating the forcing data to the land grid. There is probably a better fix than this, but not sure what it might be at this point.