Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

Run CLM with my own atmosphere forcing data

LIN

watermelon
New Member
Hello, everyone! I used the local atmospheric forcing dataset CMFD(China meteorological forcing dataset; 1978-2018; 0.1°×0.1°;3 hours) to replace GSWP3v1 in the CLM model (1901-2014; 0.5° x 0.5°) data. But I meet some questions.
My test was successful if I ues GSWP3v1 data, the code as follows:
Code:
{ create_newcase --case ~/SSP126_SP_QTP_0.125_04172 --compset SSP126_DATM%GSWP3v1_CLM50%SP_SICE_SOCN_SROF_SGLC_SWAV --res CLM_USRDAT --run-unsupported
./xmlchange DIN_LOC_ROOT=/home/user/cesmlab
./xmlchange CLM_USRDAT_NAME=QTP
./xmlchange ATM_DOMAIN_FILE=domain.lnd.0.125x0.125_tx0.QTP.1v2.140704.nc,LND_DOMAIN_FILE=domain.lnd.0.125x0.125_tx0.QTP.1v2.140704.nc
./xmlchange STOP_N=1,STOP_OPTION=nyears
./xmlchange DATM_CLMNCEP_YR_START=2001,DATM_CLMNCEP_YR_END=2001 
./xmlchange RUN_STARTDATE=2015-01-01 
./case.setup

vim user_nl_clm
fsurdat='/home/user/cesmlab/lnd/clm2/surfdata_map/surfdata_0.125x0.125_78pfts_simyr2000_QTP_addglc_revisedLAI_c151014.nc'
flanduse_timeseries='/home/user/cesmlab/lnd/clm2/surfdata_map/release-clm5.0.18/landuse.timeseries_0.125x0.125_SSP1-2.6_78pfts_CMIP6_simyr1850-2100_QTP.nc'
finidat='/home/user/cesmlab/archive/BGC_QTP_SROF_1.1prec_3.9/rest/2015-01-01-00000/BGC_QTP_SROF_1.1prec_3.9.clm2.r.2015-01-01-00000.nc' use_init_interp = .true check_dynpft_consistency = .false hist_fincl1='TLAI'
./case.build 
./case.submit}
My test was failed when I replace GSWP3v1 CMFD, the code as follows:
Code:
{create_newcase --case ~/SSP126_SP_QTP_0.125_04173 --compset SSP126_DATM%GSWP3v1_CLM50%SP_SICE_SOCN_SROF_SGLC_SWAV --res CLM_USRDAT --run-unsupported
./xmlchange DIN_LOC_ROOT=/home/user/cesmlab
./xmlchange DIN_LOC_ROOT_CLMFORC=home/user/cesmlab/CMFD_modify    #CMFD_modify have the files of "Precip"、"Solar"、“TPQWL”, each folder has nc files.
./xmlchange CLM_USRDAT_NAME=QTP
./xmlchange DATM_MODE="CLM1PT"
./xmlchange ATM_DOMAIN_FILE=domain.lnd.0.125x0.125_tx0.QTP.1v2.140704.nc,LND_DOMAIN_FILE=domain.lnd.0.125x0.125_tx0.QTP.1v2.140704.nc
./xmlchange STOP_N=1,STOP_OPTION=nyears
./xmlchange DATM_CLMNCEP_YR_START=2001,DATM_CLMNCEP_YR_END=2001 
./xmlchange RUN_STARTDATE=2015-01-01 
./case.setup

vim user_nl_clm
fsurdat='/home/user/cesmlab/lnd/clm2/surfdata_map/surfdata_0.125x0.125_78pfts_simyr2000_QTP_addglc_revisedLAI_c151014.nc'
flanduse_timeseries='/home/user/cesmlab/lnd/clm2/surfdata_map/release-clm5.0.18/landuse.timeseries_0.125x0.125_SSP1-2.6_78pfts_CMIP6_simyr1850-2100_QTP.nc'
finidat='/home/user/cesmlab/archive/BGC_QTP_SROF_1.1prec_3.9/rest/2015-01-01-00000/BGC_QTP_SROF_1.1prec_3.9.clm2.r.2015-01-01-00000.nc' use_init_interp = .true check_dynpft_consistency = .false hist_fincl1='TLAI'
./preview_namelists
#1.For the first time below the CaseDoc generated three stream file object (datm. Streams. TXT. CLM1PT. XXXX, a topo, one presaero)
#2.To model the datm. Streams. TXT. CLM1PT. 360 x720cru file as a template, create 3 stream. TXT. File:datm.streams.txt.CLM1PT.Precip; datm.streams.txt.CLM1PT.Solar; datm.streams.txt.CLM1PT.TPQWL
#3.
cp ~/SSP126_SP_QTP_0.125_04173/CaseDocs/datm.streams.txt.CLM1PT.Precip user_datm.streams.txt.CLM1PT.Precip cp ~/SSP126_SP_QTP_0.125_04173/CaseDocs/datm.streams.txt.CLM1PT.Solar user_datm.streams.txt.CLM1PT.Solar cp ~/SSP126_SP_QTP_0.125_04173/CaseDocs/datm.streams.txt.CLM1PT.TPQWL user_datm.streams.txt.CLM1PT.TPQWL 

cp ~/SSP126_SP_QTP_0.125_04173/CaseDocs/datm.streams.txt.CLM1PT.Precip ~/SSP126_SP_QTP_0.125_04173/Buildconf/datmconf/datm.streams.txt.CLM1PT.Precip 
cp ~/SSP126_SP_QTP_0.125_04173/CaseDocs/datm.streams.txt.CLM1PT.Solar ~/SSP126_SP_QTP_0.125_04173/Buildconf/datmconf/datm.streams.txt.CLM1PT.Solar 
cp ~/SSP126_SP_QTP_0.125_04173/CaseDocs/datm.streams.txt.CLM1PT.TPQWL ~/SSP126_SP_QTP_0.125_04173/Buildconf/datmconf/datm.streams.txt.CLM1PT.TPQWL
#4.
vim user_nl_datm
&datm_nml  
   decomp ="1d"
   factorfn = "null" 
   force_prognostic_true = .false.  
   iradsw = 1  
   presaero = .true.  
   restfilm = "undefined" 
   restfils = "undefined"  
   wiso_datm = .false. 
/ 
&shr_strdata_nml  
   datamode = "CLMNCEP"  
   domainfile = "/home/user/cesmlab/share/domains/domain.lnd.0.125x0.125_tx0.QTP.1v2.140704.nc"  
   dtlimit = 1.5, 1.5, 1.5, 1.5, 1.5, 1.5  fillalgo = "nn", "nn", "nn", "nn", "nn", "nn"  
   fillmask = "nomask", "nomask", "nomask", "nomask", "nomask", "nomask"  
   fillread = "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET"  
   fillwrite = "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET"  
   mapalgo = "nn", "nn", "nn", "nn", "nn", "nn"  
   mapmask = "nomask", "nomask", "nomask", "nomask", "nomask", "nomask"  
   mapread = "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET"  
   mapwrite = "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET"  
   readmode = "single", "single", "single", "single", "single", "single"  
   streams = "datm.streams.txt.CLM1PT.Precip 2001 2001 2001",     
           "datm.streams.txt.CLM1PT.Solar 2001 2001 2001",    
           "datm.streams.txt.CLM1PT.TPHWL 2001 2001 2001",     
           "datm.streams.txt.presaero.SSP1-2.6 2015 2015 2101",    
           "datm.streams.txt.topo.observed 1 1 1",     
           "datm.streams.txt.co2tseries.SSP1-2.6 2015 2015 2500" 
   taxmode = "extend", "extend", "extend", "extend", "extend", "extend" 
   tintalgo = "nearest", "linear", "lower", "linear", "lower", "linear"  vectors = "null"
/
#5.The./preview_namelists command is executed a second time, then the model updates datm_in and namelist_infile under Buildconf/datmconf according to what is in the user_nl_datm file, But the contents of the datm.input_data_list file under Buildconf/ will not be updated. so I changed the datm.input_data_list next step
./preview_namelists
#6. changed datm.input_data_list, you can see the attach files
#7.I commented out the source code so that the contents of the datm.input_data_list file under Buildconf/ would not be overwritten
vim /opt/ncar/cesm2/cime/src/components/data_comps_mct/datm/cime_config/buildnml}

1681741414175.png

./check_input_data #no problem
./case.build
./case.submit
After that, the datm.input_data_list was deleted but Idon't know why!
1681741579883.png
Also,./case.submit failed and the problems as follows :

1681741926886.png
I can't find the file in /home/user/cesmlab/run/SSP126_SP_QTP_0.125_04173/cesm.log.230417-142916 under the path. And I vim this file was blank.

The questions I want to ask are as follows:
Q1: Is my method of replacing data feasible, and if so, what adjustments should I make to keep my code running smoothly?
Q2: When I failed after ./case.submit, always let me see log file for details by cesm.log.XXX. But I always find it and don't know why!
Q3. what's means of "./case.st_archive" ? the following answer appears when I run ./case.st_archive.
Q4. It seems like problem and how can I deal with it?
1681742567888.png
Q5: If that doesn't work, I try to write all the variables into one nc file base datm.streams.txt.CLM1PT.CLM_USRDAT. But the variables were different from my data. Is it feasible for me to modify the file of "datm. Streams. TXT. CLM1PT. CLM_USRDAT" after writing my own xxx.CLM1PT. CLM_USRDAT.nc? Will it still be replaced by the default file when I run ./case.submit
Q6:Do I need to modify any other files if I write all the variables into one nc file and changed datm. Streams. TXT. CLM1PT. CLM_USRDAT?

1681743514821.png
 

Attachments

  • datm.streams.txt
    1.2 KB · Views: 7
  • 1681741969447.png
    1681741969447.png
    23.1 KB · Views: 8
  • datm.input_data_list.txt
    4 KB · Views: 3
  • 1681742960670.png
    1681742960670.png
    47 KB · Views: 12

oleson

Keith Oleson
CSEG and Liaisons
Staff member
Once you've done ./xmlchange DATM_MODE="CLM1PT", the model is going to expect all of the forcing in one stream instead of three separate streams. So don't change DATM_MODE and then retry all of your steps from the beginning. You should also not be changing anything in Buildconf, e.g., copy files into there.
 

LIN

watermelon
New Member
Thanks for your reply!I made a new attempt today. In this test, I didn't change DATM_MODE and then retry all of your steps from the beginning.

Code:
create_newcase --case ~/SSP126_SP_QTP_0.125_0419 --compset SSP126_DATM%GSWP3v1_CLM50%SP_SICE_SOCN_SROF_SGLC_SWAV --res CLM_USRDAT --run-unsupported 
./xmlchange DIN_LOC_ROOT=/home/user/cesmlab 
./xmlchange CLM_USRDAT_NAME=QTP
./xmlchange ATM_DOMAIN_FILE=domain.lnd.0.125x0.125_tx0.QTP.1v2.140704.nc,LND_DOMAIN_FILE=domain.lnd.0.125x0.125_tx0.QTP.1v2.140704.nc ./xmlchange STOP_N=1,STOP_OPTION=nyears 
./xmlchange DATM_CLMNCEP_YR_START=2001,DATM_CLMNCEP_YR_END=2001 
./xmlchange RUN_STARTDATE=2015-01-01 
./xmlchange RUNDIR=/home/user/cesmlab/run/SSP126_SP_QTP_0.125_0419 
./xmlchange DOUT_S_ROOT=/home/user/cesmlab/archive/SSP126_SP_QTP_0.125_0419 
./case.setup 
./preview_namelists

vim user_nl_clm
 #insert
fsurdat='/home/user/cesmlab/lnd/clm2/surfdata_map/surfdata_0.125x0.125_78pfts_simyr2000_QTP_addglc_revisedLAI_c151014.nc' flanduse_timeseries='/home/user/cesmlab/lnd/clm2/surfdata_map/release-clm5.0.18/landuse.timeseries_0.125x0.125_SSP1-2.6_78pfts_CMIP6_simyr1850-2100_QTP.nc' 
finidat='/home/user/cesmlab/archive/BGC_QTP_SROF_1.1prec_3.9/rest/2015-01-01-00000/BGC_QTP_SROF_1.1prec_3.9.clm2.r.2015-01-01-00000.nc' use_init_interp = .true 
check_dynpft_consistency = .false
hist_fincl1='TLAI' 

cp ~/SSP126_SP_QTP_0.125_0419/CaseDocs/datm.streams.txt.CLMGSWP3v1.Precip user_datm.streams.txt.CLMGSWP3v1.Precip 
cp ~/SSP126_SP_QTP_0.125_0419/CaseDocs/datm.streams.txt.CLMGSWP3v1.Solar user_datm.streams.txt.CLMGSWP3v1.Solar 
cp ~/SSP126_SP_QTP_0.125_0419/CaseDocs/datm.streams.txt.CLMGSWP3v1.TPQW user_datm.streams.txt.CLMGSWP3v1.TPQW
#after copy I revised the user_datm.streams.txt.CLMGSWP3v1.XXX,for example as follows

1681874676561.png
and then, I modified the user_nl_datm data.
Code:
vim user_nl_datm
&datm_nml  
   decomp ="1d"
   factorfn = "null" 
   force_prognostic_true = .false.  
   iradsw = 1  
   presaero = .true.  
   restfilm = "undefined" 
   restfils = "undefined"  
   wiso_datm = .false. 
/ 
&shr_strdata_nml  
   datamode = "CLMNCEP"  
   domainfile = "/home/user/cesmlab/share/domains/domain.lnd.0.125x0.125_tx0.QTP.1v2.140704.nc"  
   dtlimit = 1.5, 1.5, 1.5, 1.5, 1.5, 1.5  
   0fillalgo = "nn", "nn", "nn", "nn", "nn", "nn"  
   fillmask = "nomask", "nomask", "nomask", "nomask", "nomask", "nomask"  
   fillread = "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET"  
   fillwrite = "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET"  
   mapalgo = "nn", "nn", "nn", "nn", "nn", "nn"  
   mapmask = "nomask", "nomask", "nomask", "nomask", "nomask", "nomask"  
   mapread = "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET"  
   mapwrite = "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET"  
   readmode = "single", "single", "single", "single", "single", "single"  
   streams = "datm.streams.txt.CLMGSWP3v1.Precip 2001 2001 2001",     
           "datm.streams.txt.CLMGSWP3v1.Solar 2001 2001 2001",    
           "datm.streams.txt.CLMGSWP3v1.TPHWL 2001 2001 2001",     
           "datm.streams.txt.presaero.SSP1-2.6 2015 2015 2101",    
           "datm.streams.txt.topo.observed 1 1 1",     
           "datm.streams.txt.co2tseries.SSP1-2.6 2015 2015 2500" 
   taxmode = "extend", "extend", "extend", "extend", "extend", "extend" 
   tintalgo = "nearest", "linear", "lower", "linear", "lower", "linear"  vectors = "null"
/

./preview_namelists
./check_input_data
./case.build
./case.submit
}
However, the error is still reported as follows:
1681880595275.png

Is the cause of the error related to not changing the domain file? This test still uses the default domain data.
1681880871455.png
Below is the header file of my atmospheric forcing data, do you see the need to change the domain file? If I want to make changes, how to make a new domain file that fits my atmospheric forcing data.
1681881066126.png

@oleson
 

oleson

Keith Oleson
CSEG and Liaisons
Staff member
Yes, you definitely need to change the domain file so that it matches the resolution of your forcing data.
 

LIN

watermelon
New Member
Dear oleson,
I met some difficulties in the production of domain file today, I hope you can give me some answers.

First, I read the READ ME and INSTALL file under path of ~/cesm2/cime/tools/mapping/gen_domain_files
And I run the code in the INSTALL :
Code:
cd src
../../../configure --macros-format Makefile --mpilib mpi-serial
. ./.env_mach_specific.sh
gmake
Then the gen domain file appears in the folder of ~/cesm2/cime/tools/mapping/gen_domain_files:
Code:
#./gen_domain -m $MAPFILE -o $PTNAME -l $PTNAME
./gen_domain -m ~/cesmlab/CMFD_modify/Precip/prec_ITPCAS-CMFD_V0106_B-01_03hr_010deg_200003.nc -o 45_g37 -l f45_g37
I thought the "gridocn = output ocean grid name (NOT A FILE NAME!) gridlnd = output land grid name (NOT A FILE NAME!)" was meant the grid resolution?
My forcing data resolution was 0.1°×0.1°. And I found that f45_g37 was about 0.1° x 0.1°, so I ran it, but there was still an error
1681984152032.png
How can I make a file that fits my atmospheric forcing data and would like your help, thanks again!
 

oleson

Keith Oleson
CSEG and Liaisons
Staff member
You will need to create a mapping file to use with gen_domain (instead of using ~/cesmlab/CMFD_modify/Precip/prec_ITPCAS-CMFD_V0106_B-01_03hr_010deg_200003.nc).
You would use ./mknoocnmap.pl to do that (see ./mknoocnmap.pl --help for the available options). mknoocnmap.pl is located in tools/mkmapdata.
An example for a regional grid:

./mknoocnmap.pl -p 30.5,114.4 -n 80x60_wh -nx 80 -ny 60 -dx 4 -dy 3

This will produce a mapping file in tools/mkmapgrids, e.g.,:

map_80x60_wh_noocean_to_80x60_wh_nomask_aave_da_230420.nc

and then:

setenv MAPFILE map_80x60_wh_noocean_to_80x60_wh_nomask_aave_da_230420.nc
setenv OCNDOM 80x60_wh
setenv ATMDOM 80x60_wh
./gen_domain -m $MAPFILE -o $OCNDOM -l $ATMDOM

That will produce a domain file that you can use in your streams files, e.g.,:

domain.lnd.80x60_wh_80x60_wh.230420.nc

I think that will work. The file will have a "mask" variable that will be 1 everywhere, so as long as you have forcing data at every grid cell you should be ok.
Alternatively, you could create one manually with the grid information you have on the forcing file.
 

LIN

watermelon
New Member
Thanks your kind reply, and today I have successfully created the data of -m$MAPFILE -o $OCNDOM -l $ATMDOM. But I still met some questions, it's a little complex, I hope you can understand what I said.
I used the docker to run the CLM, and when I creat the new docker, I run the code of "cp -r /opt/ncar/cesm2 /home/user/cesm2" to help me check the code. And yesterday I just modify the code in docker I copied.
Code:
cd ~/cesm2/components/clm/tools/mkmapdata/
 ./mknoocnmap.pl -p 35,105 -n china_CMFD_grid -nx 700 -ny 400 -dx 70 -dy 40
There has the error of "env: ‘ncl’: No such file or directory". So I run these code in my server next:
Code:
conda activate ncl_stable
cd /data/.conda/envs/xiamu/components/clm/tools/mkmapdata/
./mknoocnmap.pl -p 35,105 -n china_CMFD_grid_421 -nx 700 -ny 400 -dx 70 -dy 40
cd /data/.conda/envs/xiamu/components/clm/tools/mkmapgrids/
ls
map_china_CMFD_grid_noocean_to_china_CMFD_grid_nomask_aave_da_230421.nc SCRIPgrid_china_CMFD_grid_nomask_c230421.nc SCRIPgrid_china_CMFD_grid_noocean_c230421.nc
export MAPFILE=map_china_CMFD_grid_noocean_to_china_CMFD_grid_nomask_aave_da_230421.nc
export OCNDOM=SCRIPgrid_china_CMFD_grid_nomask_c230421.nc
export ATMDOM=SCRIPgrid_china_CMFD_grid_noocean_c230421.nc
cd /data/jiakun/.conda/envs/xiamu/cime/tools/mapping/gen_domain_files/
cd src
../../../configure --macros-format Makefile --mpilib mpi-serial --machine linux-generic
. ./.env_mach_specific.sh
gmake
1682087495491.png

And in the docker, it's successful.
1682087874507.png

And I try to move my results by tools/mkmapgrids into docker.
1682088313224.png
and the file that appears with the process was core.68833 ,it's garbled.

I actually lost confidence a little because I felt every step is facing problems. Thank you for your patience all the time, hope to hear back from you soon, and I hope you can have a enjoyable weekends!
 

Attachments

  • 1682087839604.png
    1682087839604.png
    36.2 KB · Views: 9

LIN

watermelon
New Member
Hello, oleson. I have deal the problem of 'Can't open include file 'netcdf.inc' by reinstalling the netcdf library, because I found that my netcdf library may have been deleted by mistake by my classmates who worked together on the server.
1682342791989.png
Code:
(xiamu) [jiakun@gpu-node-01 src]$ gmake
gfortran  -o ../gen_domain gen_domain.o -L/usr/local/lib -lnetcdff -L/usr/local/lib -lnetcdf  -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,-rpath,/data/jiakun/.conda/envs/xiamu/lib -Wl,-rpath-link,/data/jiakun/.conda/envs/xiamu/lin/.conda/envs/xiamu/lib
/data/jiakun/.conda/envs/xiamu/bin/ld: warning: libhdf5_hl.so.8, needed by /usr/local/lib/libnetcdf.so, not found (try using -rpath or -rpath
/data/jiakun/.conda/envs/xiamu/bin/ld: warning: libhdf5.so.8, needed by /usr/local/lib/libnetcdf.so, not found (try using -rpath or -rpath-li
/data/jiakun/.conda/envs/xiamu/bin/ld: gen_domain.o: in function `MAIN__':
gen_domain.F90:(.text+0x4342): undefined reference to `iargc_'
/data/jiakun/.conda/envs/xiamu/bin/ld: /usr/local/lib/libnetcdf.so: undefined reference to `H5Gopen1'
/data/jiakun/.conda/envs/xiamu/bin/ld: /usr/local/lib/libnetcdf.so: undefined reference to `H5T_STD_U16LE_g'
/data/jiakun/.conda/envs/xiamu/bin/ld: /usr/local/lib/libnetcdf.so: undefined reference to `H5Dset_extent'
/data/jiakun/.conda/envs/xiamu/bin/ld: /usr/local/lib/libnetcdf.so: undefined reference to `H5Pcreate'
/data/jiakun/.conda/envs/xiamu/bin/ld: /usr/local/lib/libnetcdf.so: undefined reference to `H5Tget_native_type'
/data/jiakun/.conda/envs/xiamu/bin/ld: /usr/local/lib/libnetcdf.so: undefined reference to `H5Aexists'
........
/data/jiakun/.conda/envs/xiamu/bin/ld: /usr/local/lib/libnetcdf.so: undefined reference to `H5Tenum_create'
/data/jiakun/.conda/envs/xiamu/bin/ld: /usr/local/lib/libnetcdf.so: undefined reference to `H5T_NATIVE_USHORT_g'
/data/jiakun/.conda/envs/xiamu/bin/ld: /usr/local/lib/libnetcdf.so: undefined reference to `H5Dwrite'
/data/jiakun/.conda/envs/xiamu/bin/ld: /usr/local/lib/libnetcdf.so: undefined reference to `H5Dclose'
/data/jiakun/.conda/envs/xiamu/bin/ld: /usr/local/lib/libnetcdf.so: undefined reference to `H5T_IEEE_F32BE_g'
/data/jiakun/.conda/envs/xiamu/bin/ld: /usr/local/lib/libnetcdf.so: undefined reference to `H5Tget_class'
/data/jiakun/.conda/envs/xiamu/bin/ld: /usr/local/lib/libnetcdf.so: undefined reference to `H5Tcreate'
/data/jiakun/.conda/envs/xiamu/bin/ld: /usr/local/lib/libnetcdf.so: undefined reference to `H5Fclose'
/data/jiakun/.conda/envs/xiamu/bin/ld: /usr/local/lib/libnetcdf.so: undefined reference to `H5Gget_num_objs'
/data/jiakun/.conda/envs/xiamu/bin/ld: /usr/local/lib/libnetcdf.so: undefined reference to `H5Gclose'
/data/jiakun/.conda/envs/xiamu/bin/ld: /usr/local/lib/libnetcdf.so: undefined reference to `H5Dcreate2'
/data/jiakun/.conda/envs/xiamu/bin/ld: /usr/local/lib/libnetcdf.so: undefined reference to `H5T_STD_I64BE_g'
/data/jiakun/.conda/envs/xiamu/bin/ld: /usr/local/lib/libnetcdf.so: undefined reference to `H5Aget_name'
collect2: error: ld returned 1 exit status
gmake: *** [../gen_domain] Error 1
And then there was another question that I didn't understand, and I saw a similar question being answered in the forum maybe the version of the netcdf library is too low? I don't know if this is the reason. Please give me some ideas to solve it. I really appreciate it! @oleson
 

erik

Erik Kluzek
CSEG and Liaisons
Staff member
Glad you saw this issue with netcdf.inc. Yes, having NETCDF deleted by someone else would be a problem!

On the latest error. The problem is with linking to the NetCDF library as it also needs to link in HDF5 in order to use NetCDF. This needs to be added in at the link step.

You might do some testing of your NetCDF library build using "nf-config" which tells you how to compile with link with the library.

nf-config --flibs

will tell you what libraries need to be linked in

nf-config --has-nc4

will let you know if HDF5 was built with your version of NetCDF.

I think it will tell you that you need to add some additional libraries for HDF5 to the link step. Once you get those in I would expect this to link.

This also should come in at the "configure" step where you get your machine setup to build gen_domain. You need to run that step for your machine and you need to get your machine working to build CESM/CTSM. But, you can also get this working by editing the Makefile that the configure step created.
 

erik

Erik Kluzek
CSEG and Liaisons
Staff member
By the way I wanted to confirm that the workaround you used when you saw the error about "env: ‘ncl’: No such file or directory" is correct you needed to get an environment where you could use ncl the NCAR Command Language. It looks like you were able to activate that with a conda environment.

Also note that if NCL causes a problem you could comment out the line that runs the NCL script. In this context the NCL script was doing something not strictly required. It removes duplicated weights from the mapping file.
 

zhangguanhao

zhangguanhao
New Member
Glad you saw this issue with netcdf.inc. Yes, having NETCDF deleted by someone else would be a problem!

On the latest error. The problem is with linking to the NetCDF library as it also needs to link in HDF5 in order to use NetCDF. This needs to be added in at the link step.

You might do some testing of your NetCDF library build using "nf-config" which tells you how to compile with link with the library.

nf-config --flibs

will tell you what libraries need to be linked in

nf-config --has-nc4

will let you know if HDF5 was built with your version of NetCDF.

I think it will tell you that you need to add some additional libraries for HDF5 to the link step. Once you get those in I would expect this to link.

This also should come in at the "configure" step where you get your machine setup to build gen_domain. You need to run that step for your machine and you need to get your machine working to build CESM/CTSM. But, you can also get this working by editing the Makefile that the configure step created.
Dear professor:
I met the same problem, could you please give some details about how can I add the link of HDF5.
The problem is with linking to the NetCDF library as it also needs to link in HDF5 in order to use NetCDF. This needs to be added in at the link step.
Thanks a lot!
 

LiwenWu

LiwenWu
Member
Hello, everyone! I used the local atmospheric forcing dataset CMFD(China meteorological forcing dataset; 1978-2018; 0.1°×0.1°;3 hours) to replace GSWP3v1 in the CLM model (1901-2014; 0.5° x 0.5°) data. But I meet some questions.
My test was successful if I ues GSWP3v1 data, the code as follows:
Code:
{ create_newcase --case ~/SSP126_SP_QTP_0.125_04172 --compset SSP126_DATM%GSWP3v1_CLM50%SP_SICE_SOCN_SROF_SGLC_SWAV --res CLM_USRDAT --run-unsupported
./xmlchange DIN_LOC_ROOT=/home/user/cesmlab
./xmlchange CLM_USRDAT_NAME=QTP
./xmlchange ATM_DOMAIN_FILE=domain.lnd.0.125x0.125_tx0.QTP.1v2.140704.nc,LND_DOMAIN_FILE=domain.lnd.0.125x0.125_tx0.QTP.1v2.140704.nc
./xmlchange STOP_N=1,STOP_OPTION=nyears
./xmlchange DATM_CLMNCEP_YR_START=2001,DATM_CLMNCEP_YR_END=2001
./xmlchange RUN_STARTDATE=2015-01-01
./case.setup

vim user_nl_clm
fsurdat='/home/user/cesmlab/lnd/clm2/surfdata_map/surfdata_0.125x0.125_78pfts_simyr2000_QTP_addglc_revisedLAI_c151014.nc'
flanduse_timeseries='/home/user/cesmlab/lnd/clm2/surfdata_map/release-clm5.0.18/landuse.timeseries_0.125x0.125_SSP1-2.6_78pfts_CMIP6_simyr1850-2100_QTP.nc'
finidat='/home/user/cesmlab/archive/BGC_QTP_SROF_1.1prec_3.9/rest/2015-01-01-00000/BGC_QTP_SROF_1.1prec_3.9.clm2.r.2015-01-01-00000.nc' use_init_interp = .true check_dynpft_consistency = .false hist_fincl1='TLAI'
./case.build
./case.submit}
My test was failed when I replace GSWP3v1 CMFD, the code as follows:
Code:
{create_newcase --case ~/SSP126_SP_QTP_0.125_04173 --compset SSP126_DATM%GSWP3v1_CLM50%SP_SICE_SOCN_SROF_SGLC_SWAV --res CLM_USRDAT --run-unsupported
./xmlchange DIN_LOC_ROOT=/home/user/cesmlab
./xmlchange DIN_LOC_ROOT_CLMFORC=home/user/cesmlab/CMFD_modify    #CMFD_modify have the files of "Precip"、"Solar"、“TPQWL”, each folder has nc files.
./xmlchange CLM_USRDAT_NAME=QTP
./xmlchange DATM_MODE="CLM1PT"
./xmlchange ATM_DOMAIN_FILE=domain.lnd.0.125x0.125_tx0.QTP.1v2.140704.nc,LND_DOMAIN_FILE=domain.lnd.0.125x0.125_tx0.QTP.1v2.140704.nc
./xmlchange STOP_N=1,STOP_OPTION=nyears
./xmlchange DATM_CLMNCEP_YR_START=2001,DATM_CLMNCEP_YR_END=2001
./xmlchange RUN_STARTDATE=2015-01-01
./case.setup

vim user_nl_clm
fsurdat='/home/user/cesmlab/lnd/clm2/surfdata_map/surfdata_0.125x0.125_78pfts_simyr2000_QTP_addglc_revisedLAI_c151014.nc'
flanduse_timeseries='/home/user/cesmlab/lnd/clm2/surfdata_map/release-clm5.0.18/landuse.timeseries_0.125x0.125_SSP1-2.6_78pfts_CMIP6_simyr1850-2100_QTP.nc'
finidat='/home/user/cesmlab/archive/BGC_QTP_SROF_1.1prec_3.9/rest/2015-01-01-00000/BGC_QTP_SROF_1.1prec_3.9.clm2.r.2015-01-01-00000.nc' use_init_interp = .true check_dynpft_consistency = .false hist_fincl1='TLAI'
./preview_namelists
#1.For the first time below the CaseDoc generated three stream file object (datm. Streams. TXT. CLM1PT. XXXX, a topo, one presaero)
#2.To model the datm. Streams. TXT. CLM1PT. 360 x720cru file as a template, create 3 stream. TXT. File:datm.streams.txt.CLM1PT.Precip; datm.streams.txt.CLM1PT.Solar; datm.streams.txt.CLM1PT.TPQWL
#3.
cp ~/SSP126_SP_QTP_0.125_04173/CaseDocs/datm.streams.txt.CLM1PT.Precip user_datm.streams.txt.CLM1PT.Precip cp ~/SSP126_SP_QTP_0.125_04173/CaseDocs/datm.streams.txt.CLM1PT.Solar user_datm.streams.txt.CLM1PT.Solar cp ~/SSP126_SP_QTP_0.125_04173/CaseDocs/datm.streams.txt.CLM1PT.TPQWL user_datm.streams.txt.CLM1PT.TPQWL

cp ~/SSP126_SP_QTP_0.125_04173/CaseDocs/datm.streams.txt.CLM1PT.Precip ~/SSP126_SP_QTP_0.125_04173/Buildconf/datmconf/datm.streams.txt.CLM1PT.Precip
cp ~/SSP126_SP_QTP_0.125_04173/CaseDocs/datm.streams.txt.CLM1PT.Solar ~/SSP126_SP_QTP_0.125_04173/Buildconf/datmconf/datm.streams.txt.CLM1PT.Solar
cp ~/SSP126_SP_QTP_0.125_04173/CaseDocs/datm.streams.txt.CLM1PT.TPQWL ~/SSP126_SP_QTP_0.125_04173/Buildconf/datmconf/datm.streams.txt.CLM1PT.TPQWL
#4.
vim user_nl_datm
&datm_nml 
   decomp ="1d"
   factorfn = "null"
   force_prognostic_true = .false. 
   iradsw = 1 
   presaero = .true. 
   restfilm = "undefined"
   restfils = "undefined" 
   wiso_datm = .false.
/
&shr_strdata_nml 
   datamode = "CLMNCEP" 
   domainfile = "/home/user/cesmlab/share/domains/domain.lnd.0.125x0.125_tx0.QTP.1v2.140704.nc" 
   dtlimit = 1.5, 1.5, 1.5, 1.5, 1.5, 1.5  fillalgo = "nn", "nn", "nn", "nn", "nn", "nn" 
   fillmask = "nomask", "nomask", "nomask", "nomask", "nomask", "nomask" 
   fillread = "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET" 
   fillwrite = "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET" 
   mapalgo = "nn", "nn", "nn", "nn", "nn", "nn" 
   mapmask = "nomask", "nomask", "nomask", "nomask", "nomask", "nomask" 
   mapread = "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET" 
   mapwrite = "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET", "NOT_SET" 
   readmode = "single", "single", "single", "single", "single", "single" 
   streams = "datm.streams.txt.CLM1PT.Precip 2001 2001 2001",    
           "datm.streams.txt.CLM1PT.Solar 2001 2001 2001",   
           "datm.streams.txt.CLM1PT.TPHWL 2001 2001 2001",    
           "datm.streams.txt.presaero.SSP1-2.6 2015 2015 2101",   
           "datm.streams.txt.topo.observed 1 1 1",    
           "datm.streams.txt.co2tseries.SSP1-2.6 2015 2015 2500"
   taxmode = "extend", "extend", "extend", "extend", "extend", "extend"
   tintalgo = "nearest", "linear", "lower", "linear", "lower", "linear"  vectors = "null"
/
#5.The./preview_namelists command is executed a second time, then the model updates datm_in and namelist_infile under Buildconf/datmconf according to what is in the user_nl_datm file, But the contents of the datm.input_data_list file under Buildconf/ will not be updated. so I changed the datm.input_data_list next step
./preview_namelists
#6. changed datm.input_data_list, you can see the attach files
#7.I commented out the source code so that the contents of the datm.input_data_list file under Buildconf/ would not be overwritten
vim /opt/ncar/cesm2/cime/src/components/data_comps_mct/datm/cime_config/buildnml}

View attachment 3516

./check_input_data #no problem
./case.build
./case.submit
After that, the datm.input_data_list was deleted but Idon't know why!
View attachment 3517
Also,./case.submit failed and the problems as follows :

View attachment 3518
I can't find the file in /home/user/cesmlab/run/SSP126_SP_QTP_0.125_04173/cesm.log.230417-142916 under the path. And I vim this file was blank.

The questions I want to ask are as follows:
Q1: Is my method of replacing data feasible, and if so, what adjustments should I make to keep my code running smoothly?
Q2: When I failed after ./case.submit, always let me see log file for details by cesm.log.XXX. But I always find it and don't know why!
Q3. what's means of "./case.st_archive" ? the following answer appears when I run ./case.st_archive.
Q4. It seems like problem and how can I deal with it?
View attachment 3520
Q5: If that doesn't work, I try to write all the variables into one nc file base datm.streams.txt.CLM1PT.CLM_USRDAT. But the variables were different from my data. Is it feasible for me to modify the file of "datm. Streams. TXT. CLM1PT. CLM_USRDAT" after writing my own xxx.CLM1PT. CLM_USRDAT.nc? Will it still be replaced by the default file when I run ./case.submit
Q6:Do I need to modify any other files if I write all the variables into one nc file and changed datm. Streams. TXT. CLM1PT. CLM_USRDAT?

View attachment 3522
Hi Lin, seeing your post above, I found that you are using CMFD as your forcing data. Recently I have been searching for relevant code to process CMFD into the forcing data required by CLM for a long time since I am not familiar with matlab. Could you please show me a piece of your code as a reference?My email is wuliwen@stumail.nwu.edu.cn. Of cource I know my request may be offensive. Begging for your understanding. Any suggestions would be greatly appreciated! Thank you!
 
Top