mksurfdata_ESMF done around glcmec

[Yuan Sun]

Thx a lot! Now it's working. However, I found severl weird points:
1. I double-checked the .input_data_list, some data directory seems to keep as the default path (e.g. mksrf_fsoicol = /glade/campaign/cesm/cesmdata/inputdata/lnd/clm2/rawdata/CTSM53RawData/globalctsm53histMKSRFDeg025_240709/mksrf_landuse_ctsm53_soilcolor_CLIM.c240709.nc). This would generate a warning when downloading:
Model missing file mksrf_fsoicol = '/glade/campaign/cesm/cesmdata/inputdata/lnd/clm2/rawdata/CTSM53RawData/globalctsm53histMKSRFDeg025_240709/mksrf_landuse_ctsm53_soilcolor_CLIM.c240709.nc'
WARNING: Cannot download file since it lives outside of the input_data_root '/scratch/c/cgf/cytian/CTSM_test/input'

2. When downloading the input data, wget generated several warnings, for example:
(a)
WARNING: wget failed with output: and errput --2024-11-14 10:18:21-- ftp://ftp.cgd.ucar.edu/cesm/inputdata/lnd/clm2/mappingdata/grids/UNSTRUCTgrid_0.25x0.25_nomask_cdf5_c200129.nc
=> ‘/scratch/c/cgf/cytian/CTSM_test/input/lnd/clm2/mappingdata/grids/UNSTRUCTgrid_0.25x0.25_nomask_cdf5_c200129.nc’
Resolving ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)... 128.117.13.41
Connecting to ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)|128.117.13.41|:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done. ==> PWD ... done.
==> TYPE I ... done. ==> CWD (1) /cesm/inputdata/lnd/clm2/mappingdata/grids ... done.
==> SIZE UNSTRUCTgrid_0.25x0.25_nomask_cdf5_c200129.nc ... done.
==> PASV ... done. ==> RETR UNSTRUCTgrid_0.25x0.25_nomask_cdf5_c200129.nc ...
No such file ‘UNSTRUCTgrid_0.25x0.25_nomask_cdf5_c200129.nc’.

(b)
WARNING: wget failed with output: and errput --2024-11-14 10:18:22-- ftp://ftp.cgd.ucar.edu/cesm/inputdata/lnd/clm2/rawdata/lake_area/mksurf_lake_0.05x0.05_hist_clm5_hydrolakes_2000.cdf5.c20220325.nc
=> ‘/scratch/c/cgf/cytian/CTSM_test/input/lnd/clm2/rawdata/lake_area/mksurf_lake_0.05x0.05_hist_clm5_hydrolakes_2000.cdf5.c20220325.nc’
Resolving ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)... 128.117.13.41
Connecting to ftp.cgd.ucar.edu (ftp.cgd.ucar.edu)|128.117.13.41|:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done. ==> PWD ... done.
==> TYPE I ... done. ==> CWD (1) /cesm/inputdata/lnd/clm2/rawdata/lake_area ...
No such directory ‘cesm/inputdata/lnd/clm2/rawdata/lake_area’.

I am wondering whether you met similar warnings. Additionally, I want to confirm with you that if I already ran a global simulation before (--compset IHistClm50Bgc), which means the needed input data were already downloaded in the {INPUT_PATH}. Now I want to compare the simulation over a certain region with that under high-resolution scale. Under this context, should I still do the fourth step (i.e., downloading the raw data) when attempting to create a regional surface dataset? Apologyize if the question does not make much sense, as I am pretty new in land-surface simulation.

Hi,

Too much information for me. I am not sure if I fully understand your questions.

First, the soil color in my surfdata.namelist is : mksrf_fsoicol = '/work/n02/n02/yuansun/cesm/cesm_inputdata/lnd/clm2/rawdata/pftcftdynharv.0.25x0.25.LUH2.histsimyr1850-2015.c20230226/mksrf_soilcolor_ctsm52_histLUH2_2005.c20230226.nc' I am not sure how yours is directed to ctsm5.3. Please check through the website - Revision 69147: /trunk/inputdata to see if the targeted data existed on NCAR's server.

Second, if you met warnings and were not sure the data was downloaded successfully, I recommend you to check the data by using xarray or other libraries. For example, you can use xr.open_dataset('your_downloaded.nc') to see if it can be read. The download process seldomly failed using 'SVN'. The FTP channel is sometimes outdated in my understanding so you may miss some new data used for ctsm5.2.

Third, running global simulations requires established data, which means NCAR already generated global data from raw data. However, regional simulations require raw data. So you are downloading high-resolution raw data. These came from different datasets.

Best,
Yuan

 

[Zh Chen]

Based on other examples I see in the mksurfdata code, it does seem like that array needs to be allocated. And based on those other examples, it doesn't look like it needs to be deallocated. I'll file an issue on this, thanks.

Hi, While using CTSM5.3.009, I noticed that absolute path references in <data_filename> of tools/mksurfdata_esmf/gen_mksurfdata_namelist.xml trigger warning messages during fsurf namelist generation in my HPC. I would like to verify if this behavior is intended.

 

[slevis]

@Zh Chen please share the triggered warnings, and I will try to answer your question.