Running CLM with ERA5 forcings

[adrienD]

Hi everyone, I am trying to run a regional case using ERA5 forcings with CTSM on the supercomputer mistral (I've been able to make a default run on it). I know it's a very specific case, but I hope someone can help.

I tried to follow what Keith has recommended here. I have downloaded ERA5 following forcings: incident solar (FSDS), precipitation (PRECTmms), specific humidity at the lowest atm level, temperature at the lowest atm level (TBOT) and wind at the lowest atm level (WIND). And I separate them into 4 subdirectories in my inputdata/atm/datm7/ERA5/ folder, e.g.:

Solar/1979-01.nc
Precip/1979-01.nc
TPW/1979-01.nc
SpeHum/1979-01-01.nc (different from others because there is one file per day instead of one file per month)

However, I don't know what I have to do next. Should I change namelist files? Should I create a new compset? Or should I just change settings in a new case run? I have already some ideas but there are so many options there, I don't know what is best to do.

Thanks in advance!

 

[oleson]

No need to create a new compset at this point. I would just create a new case and then modify the datm namelist files to point to your new data.

 

[adrienD]

No need to create a new compset at this point. I would just create a new case and then modify the datm namelist files to point to your new data.

Thanks for you fast answer! Do you have an idea where I can get/create a file for the domain of ERA5?

 

[adrienD]

Thanks for you fast answer! Do you have an idea where I can get/create a file for the domain of ERA5?

Solved! Seems to work fine for now

 

[ucas_qs]

Hello, I am also facing the same problem now. The force used in my default case is CORE2-NYF. I modified the datm stream file, but there still seems to be a problem. How did you change it?

 

[adrienD]

Hello, I am also facing the same problem now. The force used in my default case is CORE2-NYF. I modified the datm stream file, but there still seems to be a problem. How did you change it?

In your case directory, your have to change those 3 files (you can find them in CaseDocs):

datm.streams.txt.CLMGSWP3v1.TPQW
datm.streams.txt.CLMGSWP3v1.Solar
datm.streams.txt.CLMGSWP3v1.Precip

1) Make a copy and change them to include your forcings
2) Move them to your case directory of your choice and rename them with user_ like this:

user_datm.streams.txt.CLMGSWP3v1.TPQW
user_datm.streams.txt.CLMGSWP3v1.Solar
user_datm.streams.txt.CLMGSWP3v1.Precip

3) Launch ./preview_namelists to see if your model included the changes

I take the opportunity to ask another question to @oleson. I would like to include a new forcings variable (e.g. specific humidity) with different configurations (e.g. netcdf are regrouped in days rather than months). I have created a user_datm.streams.txt.CLMGSWP3v1.SpeHum, however CTSM doesn't use my file. What should I do?

 

[erik]

I take the opportunity to ask another question to @oleson. I would like to include a new forcings variable (e.g. specific humidity) with different configurations (e.g. netcdf are regrouped in days rather than months). I have created a user_datm.streams.txt.CLMGSWP3v1.SpeHum, however CTSM doesn't use my file. What should I do?

The user_datm.streams.txt file needs to be the file that's referenced in your datm_in file. So you need to use the name user_datm.streams.txt.CLMGSWP3v1.TPQW and it will use that file. Technically it's possible to use a different file-name, but it takes coordinating the new file, and the datm_in file as well as copying the file into your $RUNDIR. So it's best to just use the same filename that it's already setup for.

 

[ucas_qs]

Thank you for your reply. I used CESM2.1.3 to run a C compset(ocean-only), and I also followed the above process to replace U_10 and V_10 in COREv2 NYF with 0. However, in the file output by POP, Taux and Tauy seem not to be zero, showing spatial variation. But it's two orders of magnitude smaller than the control run. I don't know what's going on here. Maybe there's nothing wrong with the process of replacing the wind field, but POP2 has a special treatment for calculating the wind stress?

 

[oleson]

Since this is an ocean-only compset, maybe you should post in the ocean forum.
Maybe there is some minimum wind speed specified/required in the ocean model?

 

[adrienD]

I have another issue using CLM with ERA5 forcing with very few log errors to work with. Here are some error messages I got from the log files I attached:

From cesm.log

1: NetCDF: Variable not found
1: pio_support::pio_die:: myrank= -1 : ERROR: nf_mod.F90: 730 :
1: NetCDF: Variable not found

From atm.log

(datm_comp_init) allocate AVs
(datm_comp_init) flds_strm = strm_swdn:strm_precn:strm_tbot:strm_wind:strm_pbot:strm_shum
(shr_dmodel_readstrm) open : /work/aa0049/a271098/inputdata/atm/datm7/atm_forcing.datm7.ERA5/Solar/E5sf12_01_1999-12_Solar.nc
(shr_dmodel_readstrm) file lb: /work/aa0049/a271098/inputdata/atm/datm7/atm_forcing.datm7.ERA5/Solar/E5sf12_01_1999-12_Solar.nc 738
(shr_dmodel_readstrm) file ub: /work/aa0049/a271098/inputdata/atm/datm7/atm_forcing.datm7.ERA5/Solar/E5sf12_01_1999-12_Solar.nc 739
(shr_dmodel_readstrm) open : /work/aa0049/a271098/inputdata/atm/datm7/atm_forcing.datm7.ERA5/Precip/E5sf12_01_1999-12_Precip.nc
(shr_dmodel_readstrm) file lb: /work/aa0049/a271098/inputdata/atm/datm7/atm_forcing.datm7.ERA5/Precip/E5sf12_01_1999-12_Precip.nc 738
(shr_dmodel_readstrm) file ub: /work/aa0049/a271098/inputdata/atm/datm7/atm_forcing.datm7.ERA5/Precip/E5sf12_01_1999-12_Precip.nc 739
(shr_dmodel_readstrm) open : /work/aa0049/a271098/inputdata/atm/datm7/atm_forcing.datm7.ERA5/TPHWL/E5sf00_1H_2000-01_TPHWL.nc
(shr_dmodel_readstrm) file lb: /work/aa0049/a271098/inputdata/atm/datm7/atm_forcing.datm7.ERA5/TPHWL/E5sf00_1H_2000-01_TPHWL.nc 1

It seems to me that the error comes from TPHWL forcing but I don't understand what these "lb" "ub" variables are and why NetCDF doesn't find them.

I try to make a simulation with only the Precip or the Solar ERA5 forcings and it's working fine.

Any suggestions? @oleson @erik

Thanks in advance!

 

[oleson]

What version of CLM or CESM are you using (git describe)? I need that information to look at the proper line in the traceback in the cesm log.
The lb and ub are lower-bounds and upper bounds of the time dimension on the forcing file that the datm is reading.
It's possible that the datm is not finding one of the variables in the forcing file that is specified in your datm.streams.txt.CLMGSWP3v1.TPQW file .
There is probably something similar to this in your streams file:

TBOT tbot
WIND wind
QBOT shum
PSRF pbot
FLDS lwdn

Do all of the variables (e.g., TBOT, WIND) exist in your forcing file?

 

[adrienD]

What version of CLM or CESM are you using (git describe)? I need that information to look at the proper line in the traceback in the cesm log.
The lb and ub are lower-bounds and upper bounds of the time dimension on the forcing file that the datm is reading.
It's possible that the datm is not finding one of the variables in the forcing file that is specified in your datm.streams.txt.CLMGSWP3v1.TPQW file .
There is probably something similar to this in your streams file:

TBOT tbot
WIND wind
QBOT shum
PSRF pbot
FLDS lwdn

Do all of the variables (e.g., TBOT, WIND) exist in your forcing file?

Thanks for your quick answer! Sorry I forgot I didn't include my git describe in this post. I am using ctsm5.1.dev023.

I am actually using only 4 variables:

TBOT tbot
WIND wind
PSRF pbot
var133 shum

I was not using the incident longwave radiation because of an old CLM4 doc I used, but I guess it's my issue here?

 

[oleson]

It's fine not to specify/use the incident longwave as the model will still use an empirical equation if it's not provided (as in CLM4).
The problem is associated with this line in shr_dmodel_mod.F90:

call shr_mpi_bcast(sfldName,mpicom,'sfldName')

I think that sfldName is the variable name specified in your stream file and I think it's not finding one of those variables names in your forcing file (possibly).
So, are "TBOT", "WIND", "PSRF", and "var133" variables in your forcing files?

Could you attach your other two streams files (just in case it's not the TPQW file) and output from an ncdump header dump of the first three forcing files that are being read in?

ncdump -h /work/aa0049/a271098/inputdata/atm/datm7/atm_forcing.datm7.ERA5/Solar/E5sf12_01_1999-12_Solar.nc >&! E5sf12_01_1999-12_Solar.ncdump

ncdump -h /work/aa0049/a271098/inputdata/atm/datm7/atm_forcing.datm7.ERA5/Precip/E5sf12_01_1999-12_Precip.nc >&! E5sf12_01_1999-12_Precip.ncdump

ncdump -h /work/aa0049/a271098/inputdata/atm/datm7/atm_forcing.datm7.ERA5/TPHWL/E5sf00_1H_2000-01_TPHWL.nc >&! E5sf00_1H_2000-01_TPHWL.ncdump

 

[oleson]

Actually, this line:

rcode = pio_inq_varid(pioid,trim(sfldName),varid)

which is inquiring whether the variable is on the file.

 

[adrienD]

Yes, "TBOT", "WIND", "PSRF", and "var133" are the variables names in my TPQW ERA5 forcings.

I have attached the files you have asked. Thanks again!

 

[oleson]

Ok, I think you've specified WIND in your streams file, but you have "wind" in your forcing file. I think the capitalization has to be the same.

 

[adrienD]

Ok, I think you've specified WIND in your streams file, but you have "wind" in your forcing file. I think the capitalization has to be the same.

Problem solved! Thanks a lot.

 

[adrienD]

Hi, I have another issue with the ERA5 forcings, but I am hopeful it will be the last one!

Precipitation and TPQW forcings are working fine but I have some issues when using the solar forcings form ERA5. Here is an extract of the log files attached:

6: errsoi incorporated into sensible heat in LakeTemperature: c, (W/m^2):
6: 74227 4.394531250000000E-003
6: Inconsistency in SnowDivision_Lake! c, remaindersdztot = 74227
6: 4.656612873077393E-010
6: calling getglobalwrite with decomp_index= 74227 and clmlevel= column
6: local column index = 74227
6: global column index = 119988
6: global landunit index = 56745
6: global gridcell index = 33060
6: gridcell longitude = 201.681701660156
6: gridcell latitude = 63.3588218688965
6: column type = 5
6: landunit type = 5
6: ENDRUN:
6: ERROR: ERROR in SnowHydrologyMod.F90 at line 2816
4: WARNING: snow balance error
4: nstep= 2 local indexc= 51662 col%itype= 5
4: lun%itype= 5 errh2osno= -2.235174179077148E-008
0: ERROR: ERROR: One or more of the output from CLM to the coupler are NaN
13: # of NaNs = 1
13: Which are NaNs = F F F F F F T F F F F F F F F F F F F F F F F F F F F F F F F
13: F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F
13: NaN found in field at gridcell index
13: 55608
13: ERROR: ERROR: One or more of the output from CLM to the coupler are NaN

Is it possible I am using a wrong variable to replace the "incident solar (FSDS)" variable from GWSP? I didn't find a lot of information about which solar variable forcing to use. I am using this one: ECMWF | Parameter details

Thanks in advance!

 

[oleson]

The incident solar (FSDS) should be the total (visible + near-IR + direct + diffuse) incoming solar radiation at the surface (in W/m2).
It's hard to diagnose the cause of the error you are getting. The field name that has the NaN seems to be missing.

NaN found in field at gridcell index

The statement that prints that out is:

write(iulog,*) "NaN found in field ", trim(fname), ' at gridcell index ',begg+i-1

So, "fname" seems to be blank.

I would recommend checking your atm forcing data for unphysical values (e.g., negative shortwave/longwave radiation, negative precipitation, etc.).
Also, check the "units" attribute on your forcing variables and make sure they are correct.

 

[adrienD]

Thanks for your answer Keith. I have checked my forcings, I don't have any negative or non-physical values. Initially, the units were in J/s but I have converted them to W/m2.

Depending on how many nodes I am using, I have different error messages. Sometimes, there is a snow balance error, or a longwave energy balance error (like the logs attached)... Is it normal?

I have also attached the ncdump -h and the stream file of my solar forcings.