Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

Running CLM with ERA5 forcings

adrienD

Adrien Damseaux
Member
Hi everyone, I am trying to run a regional case using ERA5 forcings with CTSM on the supercomputer mistral (I've been able to make a default run on it). I know it's a very specific case, but I hope someone can help.

I tried to follow what Keith has recommended here. I have downloaded ERA5 following forcings: incident solar (FSDS), precipitation (PRECTmms), specific humidity at the lowest atm level, temperature at the lowest atm level (TBOT) and wind at the lowest atm level (WIND). And I separate them into 4 subdirectories in my inputdata/atm/datm7/ERA5/ folder, e.g.:

Solar/1979-01.nc
Precip/1979-01.nc
TPW/1979-01.nc
SpeHum/1979-01-01.nc (different from others because there is one file per day instead of one file per month)

However, I don't know what I have to do next. Should I change namelist files? Should I create a new compset? Or should I just change settings in a new case run? I have already some ideas but there are so many options there, I don't know what is best to do.

Thanks in advance!
 

oleson

Keith Oleson
CSEG and Liaisons
Staff member
No need to create a new compset at this point. I would just create a new case and then modify the datm namelist files to point to your new data.
 

adrienD

Adrien Damseaux
Member
No need to create a new compset at this point. I would just create a new case and then modify the datm namelist files to point to your new data.
Thanks for you fast answer! Do you have an idea where I can get/create a file for the domain of ERA5?
 

ucas_qs

qiushi Zhang
Member
Hello, I am also facing the same problem now. The force used in my default case is CORE2-NYF. I modified the datm stream file, but there still seems to be a problem. How did you change it?
 

adrienD

Adrien Damseaux
Member
Hello, I am also facing the same problem now. The force used in my default case is CORE2-NYF. I modified the datm stream file, but there still seems to be a problem. How did you change it?
In your case directory, your have to change those 3 files (you can find them in CaseDocs):

datm.streams.txt.CLMGSWP3v1.TPQW
datm.streams.txt.CLMGSWP3v1.Solar
datm.streams.txt.CLMGSWP3v1.Precip

1) Make a copy and change them to include your forcings
2) Move them to your case directory of your choice and rename them with user_ like this:

user_datm.streams.txt.CLMGSWP3v1.TPQW
user_datm.streams.txt.CLMGSWP3v1.Solar
user_datm.streams.txt.CLMGSWP3v1.Precip

3) Launch ./preview_namelists to see if your model included the changes

I take the opportunity to ask another question to @oleson. I would like to include a new forcings variable (e.g. specific humidity) with different configurations (e.g. netcdf are regrouped in days rather than months). I have created a user_datm.streams.txt.CLMGSWP3v1.SpeHum, however CTSM doesn't use my file. What should I do?
 

erik

Erik Kluzek
CSEG and Liaisons
Staff member
I take the opportunity to ask another question to @oleson. I would like to include a new forcings variable (e.g. specific humidity) with different configurations (e.g. netcdf are regrouped in days rather than months). I have created a user_datm.streams.txt.CLMGSWP3v1.SpeHum, however CTSM doesn't use my file. What should I do?

The user_datm.streams.txt file needs to be the file that's referenced in your datm_in file. So you need to use the name user_datm.streams.txt.CLMGSWP3v1.TPQW and it will use that file. Technically it's possible to use a different file-name, but it takes coordinating the new file, and the datm_in file as well as copying the file into your $RUNDIR. So it's best to just use the same filename that it's already setup for.
 

ucas_qs

qiushi Zhang
Member
Thank you for your reply. I used CESM2.1.3 to run a C compset(ocean-only), and I also followed the above process to replace U_10 and V_10 in COREv2 NYF with 0. However, in the file output by POP, Taux and Tauy seem not to be zero, showing spatial variation. But it's two orders of magnitude smaller than the control run. I don't know what's going on here. Maybe there's nothing wrong with the process of replacing the wind field, but POP2 has a special treatment for calculating the wind stress?
 

oleson

Keith Oleson
CSEG and Liaisons
Staff member
Since this is an ocean-only compset, maybe you should post in the ocean forum.
Maybe there is some minimum wind speed specified/required in the ocean model?
 

adrienD

Adrien Damseaux
Member
I have another issue using CLM with ERA5 forcing with very few log errors to work with. Here are some error messages I got from the log files I attached:

From cesm.log
1: NetCDF: Variable not found
1: pio_support::pio_die:: myrank= -1 : ERROR: nf_mod.F90: 730 :
1: NetCDF: Variable not found
From atm.log
(datm_comp_init) allocate AVs
(datm_comp_init) flds_strm = strm_swdn:strm_precn:strm_tbot:strm_wind:strm_pbot:strm_shum
(shr_dmodel_readstrm) open : /work/aa0049/a271098/inputdata/atm/datm7/atm_forcing.datm7.ERA5/Solar/E5sf12_01_1999-12_Solar.nc
(shr_dmodel_readstrm) file lb: /work/aa0049/a271098/inputdata/atm/datm7/atm_forcing.datm7.ERA5/Solar/E5sf12_01_1999-12_Solar.nc 738
(shr_dmodel_readstrm) file ub: /work/aa0049/a271098/inputdata/atm/datm7/atm_forcing.datm7.ERA5/Solar/E5sf12_01_1999-12_Solar.nc 739
(shr_dmodel_readstrm) open : /work/aa0049/a271098/inputdata/atm/datm7/atm_forcing.datm7.ERA5/Precip/E5sf12_01_1999-12_Precip.nc
(shr_dmodel_readstrm) file lb: /work/aa0049/a271098/inputdata/atm/datm7/atm_forcing.datm7.ERA5/Precip/E5sf12_01_1999-12_Precip.nc 738
(shr_dmodel_readstrm) file ub: /work/aa0049/a271098/inputdata/atm/datm7/atm_forcing.datm7.ERA5/Precip/E5sf12_01_1999-12_Precip.nc 739
(shr_dmodel_readstrm) open : /work/aa0049/a271098/inputdata/atm/datm7/atm_forcing.datm7.ERA5/TPHWL/E5sf00_1H_2000-01_TPHWL.nc
(shr_dmodel_readstrm) file lb: /work/aa0049/a271098/inputdata/atm/datm7/atm_forcing.datm7.ERA5/TPHWL/E5sf00_1H_2000-01_TPHWL.nc 1
It seems to me that the error comes from TPHWL forcing but I don't understand what these "lb" "ub" variables are and why NetCDF doesn't find them.

I try to make a simulation with only the Precip or the Solar ERA5 forcings and it's working fine.

Any suggestions? @oleson @erik

Thanks in advance!
 

Attachments

  • atm.log.31699302.210824-113552.txt
    11.5 KB · Views: 5
  • cesm.log.31699302.210824-113552.txt
    26.3 KB · Views: 3

oleson

Keith Oleson
CSEG and Liaisons
Staff member
What version of CLM or CESM are you using (git describe)? I need that information to look at the proper line in the traceback in the cesm log.
The lb and ub are lower-bounds and upper bounds of the time dimension on the forcing file that the datm is reading.
It's possible that the datm is not finding one of the variables in the forcing file that is specified in your datm.streams.txt.CLMGSWP3v1.TPQW file .
There is probably something similar to this in your streams file:

TBOT tbot
WIND wind
QBOT shum
PSRF pbot
FLDS lwdn

Do all of the variables (e.g., TBOT, WIND) exist in your forcing file?
 

adrienD

Adrien Damseaux
Member
What version of CLM or CESM are you using (git describe)? I need that information to look at the proper line in the traceback in the cesm log.
The lb and ub are lower-bounds and upper bounds of the time dimension on the forcing file that the datm is reading.
It's possible that the datm is not finding one of the variables in the forcing file that is specified in your datm.streams.txt.CLMGSWP3v1.TPQW file .
There is probably something similar to this in your streams file:

TBOT tbot
WIND wind
QBOT shum
PSRF pbot
FLDS lwdn

Do all of the variables (e.g., TBOT, WIND) exist in your forcing file?
Thanks for your quick answer! Sorry I forgot I didn't include my git describe in this post. I am using ctsm5.1.dev023.

I am actually using only 4 variables:

TBOT tbot
WIND wind
PSRF pbot
var133 shum

I was not using the incident longwave radiation because of an old CLM4 doc I used, but I guess it's my issue here?
 

Attachments

  • git describe.txt
    1.8 KB · Views: 7
  • user_datm.streams.txt.CLMGSWP3v1.TPQW.txt
    14.2 KB · Views: 11

oleson

Keith Oleson
CSEG and Liaisons
Staff member
It's fine not to specify/use the incident longwave as the model will still use an empirical equation if it's not provided (as in CLM4).
The problem is associated with this line in shr_dmodel_mod.F90:

call shr_mpi_bcast(sfldName,mpicom,'sfldName')

I think that sfldName is the variable name specified in your stream file and I think it's not finding one of those variables names in your forcing file (possibly).
So, are "TBOT", "WIND", "PSRF", and "var133" variables in your forcing files?

Could you attach your other two streams files (just in case it's not the TPQW file) and output from an ncdump header dump of the first three forcing files that are being read in?

ncdump -h /work/aa0049/a271098/inputdata/atm/datm7/atm_forcing.datm7.ERA5/Solar/E5sf12_01_1999-12_Solar.nc >&! E5sf12_01_1999-12_Solar.ncdump

ncdump -h /work/aa0049/a271098/inputdata/atm/datm7/atm_forcing.datm7.ERA5/Precip/E5sf12_01_1999-12_Precip.nc >&! E5sf12_01_1999-12_Precip.ncdump

ncdump -h /work/aa0049/a271098/inputdata/atm/datm7/atm_forcing.datm7.ERA5/TPHWL/E5sf00_1H_2000-01_TPHWL.nc >&! E5sf00_1H_2000-01_TPHWL.ncdump
 

oleson

Keith Oleson
CSEG and Liaisons
Staff member
Actually, this line:

rcode = pio_inq_varid(pioid,trim(sfldName),varid)

which is inquiring whether the variable is on the file.
 

adrienD

Adrien Damseaux
Member
Yes, "TBOT", "WIND", "PSRF", and "var133" are the variables names in my TPQW ERA5 forcings.

I have attached the files you have asked. Thanks again!
 

Attachments

  • E5sf00_1H_2000-01_TPHWL.ncdump.txt
    3.2 KB · Views: 12
  • E5sf12_01_1999-12_Precip.ncdump.txt
    1.3 KB · Views: 6
  • E5sf12_01_1999-12_Solar.ncdump.txt
    1.2 KB · Views: 4
  • user_datm.streams.txt.CLMGSWP3v1.TPQW.txt
    14.2 KB · Views: 10
  • user_datm.streams.txt.CLMGSWP3v1.Solar.txt
    14.2 KB · Views: 4
  • user_datm.streams.txt.CLMGSWP3v1.Precip.txt
    14.6 KB · Views: 5

oleson

Keith Oleson
CSEG and Liaisons
Staff member
Ok, I think you've specified WIND in your streams file, but you have "wind" in your forcing file. I think the capitalization has to be the same.
 

adrienD

Adrien Damseaux
Member
Hi, I have another issue with the ERA5 forcings, but I am hopeful it will be the last one!

Precipitation and TPQW forcings are working fine but I have some issues when using the solar forcings form ERA5. Here is an extract of the log files attached:

6: errsoi incorporated into sensible heat in LakeTemperature: c, (W/m^2):
6: 74227 4.394531250000000E-003
6: Inconsistency in SnowDivision_Lake! c, remaindersdztot = 74227
6: 4.656612873077393E-010
6: calling getglobalwrite with decomp_index= 74227 and clmlevel= column
6: local column index = 74227
6: global column index = 119988
6: global landunit index = 56745
6: global gridcell index = 33060
6: gridcell longitude = 201.681701660156
6: gridcell latitude = 63.3588218688965
6: column type = 5
6: landunit type = 5
6: ENDRUN:
6: ERROR: ERROR in SnowHydrologyMod.F90 at line 2816
4: WARNING: snow balance error
4: nstep= 2 local indexc= 51662 col%itype= 5
4: lun%itype= 5 errh2osno= -2.235174179077148E-008
0: ERROR: ERROR: One or more of the output from CLM to the coupler are NaN
13: # of NaNs = 1
13: Which are NaNs = F F F F F F T F F F F F F F F F F F F F F F F F F F F F F F F
13: F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F
13: NaN found in field at gridcell index
13: 55608
13: ERROR: ERROR: One or more of the output from CLM to the coupler are NaN

Is it possible I am using a wrong variable to replace the "incident solar (FSDS)" variable from GWSP? I didn't find a lot of information about which solar variable forcing to use. I am using this one: ECMWF | Parameter details

Thanks in advance!
 

Attachments

  • atm.log.32172246.210907-111052.txt
    12.8 KB · Views: 3
  • cesm.log.32172246.210907-111052.txt
    41.2 KB · Views: 4

oleson

Keith Oleson
CSEG and Liaisons
Staff member
The incident solar (FSDS) should be the total (visible + near-IR + direct + diffuse) incoming solar radiation at the surface (in W/m2).
It's hard to diagnose the cause of the error you are getting. The field name that has the NaN seems to be missing.

NaN found in field at gridcell index

The statement that prints that out is:

write(iulog,*) "NaN found in field ", trim(fname), ' at gridcell index ',begg+i-1

So, "fname" seems to be blank.

I would recommend checking your atm forcing data for unphysical values (e.g., negative shortwave/longwave radiation, negative precipitation, etc.).
Also, check the "units" attribute on your forcing variables and make sure they are correct.
 

adrienD

Adrien Damseaux
Member
Thanks for your answer Keith. I have checked my forcings, I don't have any negative or non-physical values. Initially, the units were in J/s but I have converted them to W/m2.

Depending on how many nodes I am using, I have different error messages. Sometimes, there is a snow balance error, or a longwave energy balance error (like the logs attached)... Is it normal?

I have also attached the ncdump -h and the stream file of my solar forcings.
 

Attachments

  • cesm.log.32312889.210914-114910.txt
    21.5 KB · Views: 4
  • ncdump.solar.txt
    1.6 KB · Views: 10
  • user_datm.streams.txt.CLMGSWP3v1.Solar.txt
    14.2 KB · Views: 7
Top