high resolution (0.1 degree) CLM5-BGC-crop regional simulation

[oleson]

Yes, sorry, I regenerated the domain file using:

TO create mapping file for gen_domain (in /glade/work/oleson/release-cesm2.2.0/cime/tools/mapping/gen_mapping_files/gen_ESMF_mapping_file)

qcmd -- ./create_ESMF_map.sh --maptype aave --filesrc /glade/p/cesmdata/cseg/inputdata/share/scripgrids/gx1v7_151008.nc --filedst /glade/work/oleson/release-cesm2.1.3/components/clm/tools/mkmapgrids/SCRIPgrid_3600x1800pt_Global_nomask_c221030.nc --namesrc gx1v7 --namedst 0.1x0.1



OUTPUT IS map_gx1v7_TO_0.1x0.1_aave.231018.nc



USING /glade/work/oleson/release-cesm2.2.0

TO create domain file (in /glade/work/oleson/release-cesm2.2.0/cime/tools/mapping/gen_domain_files)

---------------------

./gen_domain -m /glade/work/oleson/release-cesm2.2.0/cime/tools/mapping/gen_mapping_files/gen_ESMF_mapping_file/map_gx1v7_TO_0.1x0.1_aave.231018.nc -o gx1v7 -l 0.1x0.1



OUTPUT IS domain.lnd.0.1x0.1_gx1v7.231019.nc

 

[xgao304]

@oleson,

Thanks for the information. Just one more question: for the new scrip grid file:

The one you listed in the above command is

SCRIPgrid_3600x1800pt_Global_nomask_c221030.nc

The one in your pub directory is

SCRIPgrid_3600x1800pt_Global_nomask_c231018.nc

They are created on different dates. Are they the same? Since I don't have access to the one created on 221030, so it is hard to tell.

Xiang

 

[oleson]

They are the same.

 

[xgao304]

@oleson:

I have a further question regarding my high-resolution (0.1x0.1) regional case. I can run the case with GSWP3 forcing at 0.5 degree and generated domain and surface dataset at 0.1 degree , but got the error message with the forcing generated from a regional climate model. The error seems related to wrong forcing grid size (see the attached log files).

create_newcase --case BgcCrop2000_BANGhigh --compset 2000_DATM%GSWP3v1_CLM50%BGC-CROP_SICE_SOCN_SROF_SGLC_SWAV \
--res CLM_USRDAT --user-mods-dir $MYDATA_DIR --machine svante --compiler intel \
--run-unsupported

My regional domain:
longitude (83.05E~95.95E, at 0.1 interval)
Latitude (15.15N ~ 26.85N, at 0.1 interval)

GSWP3 (global)
longitude (0.25~359.75, at 0.5 interval)
Latitude (-89.75 ~ 89.75, at 0.5 interval)

forcing from regional climate model:
lon=146;
lat = 141;
lon = 82.17, 82.27, 82.37, ..., 96.67 ;
lat = 14.68, 14.77, 14.86, 14.95, 15.04, 15.13, 15.22, 15.31, 15.4, 15.49,
15.58, 15.67, 15.76, 15.85, 15.94, 16.03, 16.12, 16.21, 16.3, 16.39,
16.48, 16.57, 16.66, 16.75, 16.84, 16.93, 17.02, 17.11, 17.2, 17.29,
17.38, 17.47, 17.56, 17.65, 17.74, 17.83, 17.92, 18.01, 18.1, 18.19,
18.28, 18.37, 18.46, 18.55, 18.64, 18.73, 18.82, 18.91, 19, 19.09, 19.18,
19.27, 19.36, 19.45, 19.54, 19.63, 19.72, 19.81, 19.9, 19.99, 20.08,
20.17, 20.26, 20.35, 20.44, 20.53, 20.62, 20.71, 20.8, 20.89, 20.98,
21.07, 21.16, 21.25, 21.34, 21.43, 21.52, 21.61, 21.7, 21.79, 21.88,
21.97, 22.06, 22.15, 22.24, 22.33, 22.42, 22.51, 22.6, 22.69, 22.78,
22.87, 22.96, 23.05, 23.14, 23.23, 23.32, 23.41, 23.5, 23.59, 23.68,
23.77, 23.86, 23.95, 24.04, 24.13, 24.22, 24.31, 24.4, 24.49, 24.58,
24.67, 24.76, 24.85, 24.94, 25.03, 25.12, 25.21, 25.3, 25.39, 25.48,
25.57, 25.66, 25.75, 25.84, 25.93, 26.02, 26.11, 26.2, 26.29, 26.38,
26.47, 26.56, 26.65, 26.74, 26.83, 26.92, 27.01, 27.1, 27.19, 27.28 ;

My question is: does the forcing grid have to be exactly aligned with those of domain/surface data? I thought as long as the forcing spatial extent
is larger than the domain of interest, the CTSM will do automatic interpolation if the grid is not well aligned. Is that the case?

Thanks,

Xiang

 

[slevis]

Your understanding sounds correct to me.

If you are not restricted to a model version that uses mct, and you can use a newer version that uses nuopc, then I would suggest familiarizing yourself with this discussion Setting up (high-res sparse) regional-grid CTSM simulations · ESCOMP/CTSM · Discussion #1919

 

[xgao304]

At this point, we need a quick turnaround for our project, so it is probably better for us to stick to the current model version cesm2.1.3/CLM5. I have reviewed the step-by-step instruction for running CTSM-FATES on a sparse regional grid with WRF datm driver files. For my case, I have already created the correct surface dataset and domain files at a high resolution over the domain of my interest (I have tested these datasets using the default CLM forcing data.) I am only stuck with the step A (similar to "Generate datm files from wrf output"). I think all the other steps (B-F) have been taken care of.

For step A, since I don't know R very well, it would be a bit challenging for me to modify and use the R script written by Xiulin Gao. However, I have already did some reprocessing in the forcing data format, including 1) separate the data into three directories (Precip, Solar, and TPHWL) with the consistent variable names and units; and 2) 3-hourly forcing data is grouped for each year and each month (without leap year).

I think the only issue is the grid mismatch between DATM and surface dataset.

An example head/metadata file for my forcing is as follows:

netcdf clmforc.MRCM-ERA5.Prec.1994-12 {
dimensions:
time = UNLIMITED ; // (248 currently)
lat = 141 ;
lon = 146 ;
variables:
double PRECTmms(time, lat, lon) ;
PRECTmms:_FillValue = 9.99999961690316e+35 ;
PRECTmms:cmip_phase = "renalysis" ;
PRECTmms:description = "MRCM driven by ERA5 under historical scenario" ;
PRECTmms:long_name = "PRECTmms total precipitation" ;
PRECTmms:missing_value = 1.e+36f ;
PRECTmms:remap = "remapped via ESMF_regrid_with_weights: Bilinear" ;
PRECTmms:units = "mm H2O / sec" ;
PRECTmms:variable = "precipitation" ;
float lon(lon) ;
lon:axis = "X" ;
float lat(lat) ;
lat:axis = "Y" ;
double time(time) ;
time:standard_name = "time" ;
time:long_name = "Time" ;
time:units = "hours since 1900-1-1 00:00:00" ;
time:calendar = "standard" ;
time:axis = "T" ;

// global attributes:

data:

lon = 82.17, 82.27, 82.37, 82.47, 82.57, 82.67, 82.77, 82.87, 82.97, 83.07,
83.17, 83.27, 83.37, 83.47, 83.57, 83.67, 83.77, 83.87, 83.97, 84.07,
84.17, 84.27, 84.37, 84.47, 84.57, 84.67, 84.77, 84.87, 84.97, 85.07,
85.17, 85.27, 85.37, 85.47, 85.57, 85.67, 85.77, 85.87, 85.97, 86.07,
86.17, 86.27, 86.37, 86.47, 86.57, 86.67, 86.77, 86.87, 86.97, 87.07,
87.17, 87.27, 87.37, 87.47, 87.57, 87.67, 87.77, 87.87, 87.97, 88.07,
88.17, 88.27, 88.37, 88.47, 88.57, 88.67, 88.77, 88.87, 88.97, 89.07,
89.17, 89.27, 89.37, 89.47, 89.57, 89.67, 89.77, 89.87, 89.97, 90.07,
90.17, 90.27, 90.37, 90.47, 90.57, 90.67, 90.77, 90.87, 90.97, 91.07,
91.17, 91.27, 91.37, 91.47, 91.57, 91.67, 91.77, 91.87, 91.97, 92.07,
92.17, 92.27, 92.37, 92.47, 92.57, 92.67, 92.77, 92.87, 92.97, 93.07,
93.17, 93.27, 93.37, 93.47, 93.57, 93.67, 93.77, 93.87, 93.97, 94.07,
94.17, 94.27, 94.37, 94.47, 94.57, 94.67, 94.77, 94.87, 94.97, 95.07,
95.17, 95.27, 95.37, 95.47, 95.57, 95.67, 95.77, 95.87, 95.97, 96.07,
96.17, 96.27, 96.37, 96.47, 96.57, 96.67 ;

lat = 14.68, 14.77, 14.86, 14.95, 15.04, 15.13, 15.22, 15.31, 15.4, 15.49,
15.58, 15.67, 15.76, 15.85, 15.94, 16.03, 16.12, 16.21, 16.3, 16.39,
16.48, 16.57, 16.66, 16.75, 16.84, 16.93, 17.02, 17.11, 17.2, 17.29,
17.38, 17.47, 17.56, 17.65, 17.74, 17.83, 17.92, 18.01, 18.1, 18.19,
18.28, 18.37, 18.46, 18.55, 18.64, 18.73, 18.82, 18.91, 19, 19.09, 19.18,
19.27, 19.36, 19.45, 19.54, 19.63, 19.72, 19.81, 19.9, 19.99, 20.08,
20.17, 20.26, 20.35, 20.44, 20.53, 20.62, 20.71, 20.8, 20.89, 20.98,
21.07, 21.16, 21.25, 21.34, 21.43, 21.52, 21.61, 21.7, 21.79, 21.88,
21.97, 22.06, 22.15, 22.24, 22.33, 22.42, 22.51, 22.6, 22.69, 22.78,
22.87, 22.96, 23.05, 23.14, 23.23, 23.32, 23.41, 23.5, 23.59, 23.68,
23.77, 23.86, 23.95, 24.04, 24.13, 24.22, 24.31, 24.4, 24.49, 24.58,
24.67, 24.76, 24.85, 24.94, 25.03, 25.12, 25.21, 25.3, 25.39, 25.48,
25.57, 25.66, 25.75, 25.84, 25.93, 26.02, 26.11, 26.2, 26.29, 26.38,
26.47, 26.56, 26.65, 26.74, 26.83, 26.92, 27.01, 27.1, 27.19, 27.28 ;

My surface dataset (slightly smaller than forcing domain) has the following metadata in terms of grid specification:

LONGXY =
83.05, 83.15, 83.25, 83.35, 83.45, 83.55, 83.65, 83.75, 83.85, 83.95,
84.05, 84.15, 84.25, 84.35, 84.45, 84.55, 84.65, 84.75, 84.85, 84.95,
85.05, 85.15, 85.25, 85.35, 85.45, 85.55, 85.65, 85.75, 85.85, 85.95,
86.05, 86.15, 86.25, 86.35, 86.45, 86.55, 86.65, 86.75, 86.85, 86.95,
87.05, 87.15, 87.25, 87.35, 87.45, 87.55, 87.65, 87.75, 87.85, 87.95,
88.05, 88.15, 88.25, 88.35, 88.45, 88.55, 88.65, 88.75, 88.85, 88.95,
89.05, 89.15, 89.25, 89.35, 89.45, 89.55, 89.65, 89.75, 89.85, 89.95,
90.05, 90.15, 90.25, 90.35, 90.45, 90.55, 90.65, 90.75, 90.85, 90.95,
91.05, 91.15, 91.25, 91.35, 91.45, 91.55, 91.65, 91.75, 91.85, 91.95,
92.05, 92.15, 92.25, 92.35, 92.45, 92.55, 92.65, 92.75, 92.85, 92.95,
93.05, 93.15, 93.25, 93.35, 93.45, 93.55, 93.65, 93.75, 93.85, 93.95,
94.05, 94.15, 94.25, 94.35, 94.45, 94.55, 94.65, 94.75, 94.85, 94.95,
95.05, 95.15, 95.25, 95.35, 95.45, 95.55, 95.65, 95.75, 95.85, 95.95,

LATIXY: 15.15, 15.25,......, 26.85

I have two specific questions:

1. Are 2-D LONGXY and LATIXY required in the forcing dataset? Currently I only include 1-D lon() and lat().
2. Have the grid of the forcing dataset to be exactly aligned with that of surface dataset? for example, in
my case,

lon in the re-processed forcing data should be: 82.15, 82.25, 82.35, ....., 96.65
lat in the re-processed forcing data should be: 14.65, 14.75, 14.85, ....., 27.25

Thanks,

Xiang

 

[slevis]

Q1. If the DATM files that we provide with the model show 2D, then I would recommend 2D just to be sure.
Q2. The two grids can differ.

 

[xgao304]

Thanks for the information. I could try to formuate the lon and lat as 2D (longxy and latixy) to give it a
try, although my intuition is that is not the issue to cause the error.

Do you mind taking a look at the attached log file to see what is main cause for the run error (before I formulate
lat and lon)? If the two grids can differ, I am not sure what is the problem (I have tested the use of GSWP3 forcing
with all the other settings remaining the same and the run can go without any problem). Thanks.

Xiang

 

[oleson]

It doesn't look like your datm domain file is the same size as your datm data, 130x118 compared to 146X141.

 

[xgao304]

@oleson:
I assume this should not be the issue, as you could use the global GSWP3 forcing (720x360) to drive the region of interest (85x78) - I tested with GSWP3 forcing and the run went OK. My understanding is that the datm domain has to be larger than the region of interest (for my case: datm is 146x141 versus domain 130x118). Please correct me if I am wrong. Thanks.

 

[oleson]

Your datm domain file:

/net/fs05/d1/xgao/cesm2.1.3/cases/sim_setup/Bangladesh/BANGdatacrews/domain.lnd.0.1x0.1_gx1v7_BANGcrews_c231029.nc

needs to have the same dimensions as your datm data, e.g.,

/home/xgao/CREWSnet/forcing/MRCM-ERA5/Solar/clmforc.MRCM-ERA5.Solr.1995-12.nc

 

[xgao304]

My confusion is why I can use the global GSWP3 data to drive the same datm domain
/net/fs05/d1/xgao/cesm2.1.3/cases/sim_setup/Bangladesh/BANGdatacrews/domain.lnd.0.1x0.1_gx1v7_BANGcrews_c231029.nc.

These two have different dimensions as well.

clmforc.GSWP3.c2011.0.5x0.5.Prec.1951-01.nc (720x360)

 

[oleson]

I suspect you aren't using that file for your datm domain in your GSWP3 simulation. For example, what does your datm.streams.txt.CLMGSWP3v1.Precip file show for the domain file in your GSWP3 simulation. Normally, it has this:

<domainInfo>
<variableNames>
time time
xc lon
yc lat
area area
mask mask
</variableNames>
<filePath>
/glade/p/cgd/tss/people/oleson/atm_forcing.datm7.GSWP3.0.5d.v1.c200929
</filePath>
<fileNames>
domain.lnd.360x720_gswp3.0v1.c170606.nc
</fileNames>
</domainInfo>


That file has the same dimensions as the GSWP3 forcing data.
If it doesn't then I don't see how you could run with GSWP3 successfully.

 

[xgao304]

@oleson,

Now I got what you meant. Yes, for the GSWP3 forcing case, I got the exactly same as what you showed above. The reason is - since I am using the default GSWP3 forcing data, the datm domain is automatically specified when creating datm.streams.txt.CLMGSWP3v1.Precip.

But for the RCM forcing case, I have to generate user_datm.streams.txt.* files in which I have to specify the domain file to be used. I simply used the one generated by subsetting the global domain and surface data sets at 0.1x0.1 degree created through those multi-step procedure.

(getregional_datasets.pl -ne 26.95,95.95 -sw 15.05,83.05 -i sample_inlist -o sample_outlist).

Here is my question:

The global domain and surface dataset have the following dimensions:
(3600x1800)

LONGXY = 0.05, 0.15, ...., 359.85, 359.95
LATIXY: = -89.95, -89.85, ..... 89.85, 89.95

My forcing data dimension: (146x141)

lon = 82.17, 82.27, 82.37, 82.47, 82.57, 82.67, 82.77, 82.87, 82.97, 83.07,
83.17, 83.27, 83.37, 83.47, 83.57, 83.67, 83.77, 83.87, 83.97, 84.07,
84.17, 84.27, 84.37, 84.47, 84.57, 84.67, 84.77, 84.87, 84.97, 85.07,
85.17, 85.27, 85.37, 85.47, 85.57, 85.67, 85.77, 85.87, 85.97, 86.07,
86.17, 86.27, 86.37, 86.47, 86.57, 86.67, 86.77, 86.87, 86.97, 87.07,
87.17, 87.27, 87.37, 87.47, 87.57, 87.67, 87.77, 87.87, 87.97, 88.07,
88.17, 88.27, 88.37, 88.47, 88.57, 88.67, 88.77, 88.87, 88.97, 89.07,
89.17, 89.27, 89.37, 89.47, 89.57, 89.67, 89.77, 89.87, 89.97, 90.07,
90.17, 90.27, 90.37, 90.47, 90.57, 90.67, 90.77, 90.87, 90.97, 91.07,
91.17, 91.27, 91.37, 91.47, 91.57, 91.67, 91.77, 91.87, 91.97, 92.07,
92.17, 92.27, 92.37, 92.47, 92.57, 92.67, 92.77, 92.87, 92.97, 93.07,
93.17, 93.27, 93.37, 93.47, 93.57, 93.67, 93.77, 93.87, 93.97, 94.07,
94.17, 94.27, 94.37, 94.47, 94.57, 94.67, 94.77, 94.87, 94.97, 95.07,
95.17, 95.27, 95.37, 95.47, 95.57, 95.67, 95.77, 95.87, 95.97, 96.07,
96.17, 96.27, 96.37, 96.47, 96.57, 96.67 ;
lat = 14.68, 14.77, 14.86, 14.95, 15.04, 15.13, 15.22, 15.31, 15.4, 15.49,
15.58, 15.67, 15.76, 15.85, 15.94, 16.03, 16.12, 16.21, 16.3, 16.39,
16.48, 16.57, 16.66, 16.75, 16.84, 16.93, 17.02, 17.11, 17.2, 17.29,
17.38, 17.47, 17.56, 17.65, 17.74, 17.83, 17.92, 18.01, 18.1, 18.19,
18.28, 18.37, 18.46, 18.55, 18.64, 18.73, 18.82, 18.91, 19, 19.09, 19.18,
19.27, 19.36, 19.45, 19.54, 19.63, 19.72, 19.81, 19.9, 19.99, 20.08,
20.17, 20.26, 20.35, 20.44, 20.53, 20.62, 20.71, 20.8, 20.89, 20.98,
21.07, 21.16, 21.25, 21.34, 21.43, 21.52, 21.61, 21.7, 21.79, 21.88,
21.97, 22.06, 22.15, 22.24, 22.33, 22.42, 22.51, 22.6, 22.69, 22.78,
22.87, 22.96, 23.05, 23.14, 23.23, 23.32, 23.41, 23.5, 23.59, 23.68,
23.77, 23.86, 23.95, 24.04, 24.13, 24.22, 24.31, 24.4, 24.49, 24.58,
24.67, 24.76, 24.85, 24.94, 25.03, 25.12, 25.21, 25.3, 25.39, 25.48,
25.57, 25.66, 25.75, 25.84, 25.93, 26.02, 26.11, 26.2, 26.29, 26.38,
26.47, 26.56, 26.65, 26.74, 26.83, 26.92, 27.01, 27.1, 27.19, 27.28 ;

My question is : how to formulate my subsettting command to make the datm domain match with my datm data - Should I simply use the following command
(although the global domain and surface data set is centered on an interval of 0.05):

getregional_datasets.pl -ne 27.28,96.67 -sw 14.68,82.17 -i sample_inlist -o sample_outlist)

Thanks,

Xiang

Thanks, Xiang

 

[oleson]

Since your datm forcing data appears to be on a grid that's a bit odd (0.10 in longitude and 0.09 in latitude?) and your global domain file for the surface data is not the same, I think I would just create a datm domain file manually. It should be pretty straighforward, just follow the GSWP3 domain file example.

 

[xgao304]

@oleson: I have three questions about the datm domain file.

1. I am wondering what fields are required in the file? For example, the general domain file has the following fields.

xc (lon of grid center)
xv (lon of grid vertices)
yc (lat of grid center)
yv (lat of grid vertices)
area
mask

The user_datm.streams.txt* seems to only list four of them as follows:

<domainInfo>
<variableNames>
time time
xc lon
yc lat
area area
mask mask
</variableNames>
<filePath>

Which fields are required?

2. The mask is not a land/sea mask, but has a constant value of 1, is that correct?

3.The original forcing at ~0.1 degree that are generated from a regional climate model to drive my model is in lambert conformal projection -
only has the information for LONGXY, LATIXY, and land-sea mask. It is hard to get the area information. The way to get around it may be to regrid it
into regular lat/lon, then calculate the area (if it is required), but then the land/sea mask may be messed up (unless it is a constant value). Is my understanding correct?

So the answers to the questions above may help generate a correct domain file manually.

Thanks,

Xiang

 

[slevis]

1) You are most likely to generate a successful file if you use the existing one as a template. You are welcome to experiment with fewer variables, in case you find that some variables are optional, and then add more variables if you find that they are required.
2) I think it will work with mask = 1 everywhere, especially if the datm files have valid data everywhere.
3) Sounds like this may be an issue if mask is not 1 everywhere, so let's see how it goes.

 

[xgao304]

I made the datm forcing domain file consistent with that of surface data set. After I submit the test run, I got some new error message
"ERROR: (shr_strdata_advance) ERROR dt limit for stream
(shr_strdata_advance) ERROR: for stream 1

see attached cesm log file for details)

I found the similar thread ERROR: (shr_strdata_advance) ERROR dt limit for stream

However, it seems in my case the error occurs to the middle of the month, not the beginning or the end.

should I set ndep_taxmode = 'cycle' to "extend" in lnd_in?

Any information is appreciated.

 

[slevis]

I made the datm forcing domain file consistent with that of surface data set. After I submit the test run, I got some new error message
"ERROR: (shr_strdata_advance) ERROR dt limit for stream
(shr_strdata_advance) ERROR: for stream 1

see attached cesm log file for details)

I found the similar thread ERROR: (shr_strdata_advance) ERROR dt limit for stream

However, it seems in my case the error occurs to the middle of the month, not the beginning or the end.

should I set ndep_taxmode = 'cycle' to "extend" in lnd_in?

Any information is appreciated.

Getting errors is frustrating, but often moving to a new error is a good sign, because it suggests that the previous one is resolved.

Much of what we do in model development is experimenting using the trial and error method. Since you had the intuition to try a new ndep_taxmode value, it may be worth pursuing. From looking at the error in your cesm.log file, there's another possibility:

(shr_strdata_advance) ERROR: for stream            1
 (shr_strdata_advance) ERROR: dt limit1    20.2500000000000     
  0.125000000000000        1.50000000000000

I am unsure, but I get the sense that you may need to reset dtlimit.
I searched the whole Forum site for "dtlimit" and found the following:
In this post, I see that default dtlimit = 1.5, so maybe you need to reset dtlimit to 0.125?

 

[xgao304]

Thanks. Can I use ./xmlchange to reset dtlimit?