seeking surface dataset and domain data file with 1°×1° resolution

[dayon]

Has anyone produced the surface dataset and domain data file with 1°×1° resolution on a global scale? Can you share them with me? thank you very much! When I created the surface dataset and domain data file with the tools included in CESM2.1.3, it was always failed
please call me, yuhaishen2020@gmail.com

 

[katec]

Hi, the 1 degree grid that is the default for CMIP6 is called f09_g17. If you make a new experiment with this grid resolution, CESM will download all of the needed surface dataset files automatically. If it does not do that, then please send your specific error or missing file so we can address that issue.

 

[dayon]

Hi，katec, thanks a lot. I want to generate global grid with a strict square grid (1°×1°) by using the tools in the cesm2.2.0.
However,When I follow the process to create mapping files, type the command：
./mkmapdata.sh -f SCRIPgrid_360x180pt_ECMWF_nomask_c201125.nc -r 360x180pt_ECMWF

all the previous process can get information "Completed weight generation successfully.", However, the process failed when generating "the weight files" about the "/datahuge/cesm_inputdata/lnd/clm2/mappingdata/grids/UGRID_1km-merge-10min_HYDRO1K-merge-nomask_c130402.nc".

Besides, when the process reaches this step, the usage of the computer memory display is close to 100%. what make me confused is that my computer's memory is 94.38GB, should it be enough? I don't know, so, if you can help, I will be very grateful. Looking forward to your reply.

The output display after typing "./mkmapdata.sh -f SCRIPgrid_360x180pt_ECMWF_nomask_c201125.nc -r 360x180pt_ECMWF" is attachment.

 

[katec]

Yes, the mapping tools require a huge amount of fast memory. There are several types of memory in a computer. The 94GB number you have there is probably the hard disk memory which is slow and used for long-term storage. The RAM or fast access memory is usually 8-32GB per processor. These map generation tools are typically run on our supercomputers which have large amounts of fast-access memory per processor.

Most recent CTSM tags should have a script called regridbatch.sh that will submit the regridding job to the supercomputer queue so it can be done in parallel and share the memory requirements across processors. You could give that script a try. You could also comment out the grid that is giving you trouble in the appropriate "grids=(" section of mkmapdata.sh . Not all of these grids are needed for land surface datasets, and if you want to run at 1-degree, you might not need the high resolution (10 minute) files that are giving you trouble here.

 

[dayon]

Thanks, katec. The machine I used to run the cesm is a server which has the RAM of 94.38GB, and the hard disk memory of 5.5TB. I found that it can success when I generated the small regional area surface dataset rather than the global scale. Besides, thanks for your suggestion of regridding job to the supercomputer queue.

You could also comment out the grid that is giving you trouble in the appropriate "grids=(" section of mkmapdata.sh . Not all of these grids are needed for land surface datasets, and if you want to run at 1-degree, you might not need the high resolution (10 minute) files that are giving you trouble here.

Yes, it's good suggestiton. but there may be two question:
1) I usually run the cesm with the Clm50Sp or Clm50SpGs compset, and I want to consult you that which grids are NOT needed for land surface datasets? 2) I think that both Clm50Sp and Clm50SpGs compset have the river model, and if the surface dataset I created doesn't contain the river information (about the information deduced from the UGRID_1km-merge-10min_HYDRO1K-merge-nomask_c130402.nc) , then, Is there any problem with the surface dataset?

thanks a lot. waiting for your reply.

 

[katec]

These are pretty land-model specific questions that are a bit out of my area of expertise. I'm going to transfer this thread to their forums to see if a Land Model scientist can give you the best answers.

 

[oleson]

I believe the UGRID_1km-merge-10min_HYDRO1K-merge-nomask_c130402.nc file is needed because it is eventually used to provide "SLOPE" and "STD_ELEV" on the surface dataset which are used for snow cover fraction and surface water parameterizations.
I've run regridbatch.sh for your 1degx1deg grid on our large memory nodes. I've put the resulting mapping file in our anonymous ftp site. Hopefully, it will work for you and you can move on to creating a domain file and surface dataset.

ftp ftp.cgd.ucar.edu
cd pub/oleson/mapping_files
get map_1km-merge-10min_HYDRO1K-merge-nomask_to_360x180pt_ECMWF_nomask_aave_da_c201202.nc

 

[dayon]

These are pretty land-model specific questions that are a bit out of my area of expertise. I'm going to transfer this thread to their forums to see if a Land Model scientist can give you the best answers.

Hi katec, thanks for your transfer. it helped me a lot.

 

[dayon]

I believe the UGRID_1km-merge-10min_HYDRO1K-merge-nomask_c130402.nc file is needed because it is eventually used to provide "SLOPE" and "STD_ELEV" on the surface dataset which are used for snow cover fraction and surface water parameterizations.
I've run regridbatch.sh for your 1degx1deg grid on our large memory nodes. I've put the resulting mapping file in our anonymous ftp site. Hopefully, it will work for you and you can move on to creating a domain file and surface dataset.

ftp ftp.cgd.ucar.edu
cd pub/oleson/mapping_files
get map_1km-merge-10min_HYDRO1K-merge-nomask_to_360x180pt_ECMWF_nomask_aave_da_c201202.nc

Hi, Oleson. Thanks for your help. I will go to download the file immediately following the path you gave, thank you very much for your help, I have benefited a lot.

 

[dayon]

I believe the UGRID_1km-merge-10min_HYDRO1K-merge-nomask_c130402.nc file is needed because it is eventually used to provide "SLOPE" and "STD_ELEV" on the surface dataset which are used for snow cover fraction and surface water parameterizations.
I've run regridbatch.sh for your 1degx1deg grid on our large memory nodes. I've put the resulting mapping file in our anonymous ftp site. Hopefully, it will work for you and you can move on to creating a domain file and surface dataset.

ftp ftp.cgd.ucar.edu
cd pub/oleson/mapping_files
get map_1km-merge-10min_HYDRO1K-merge-nomask_to_360x180pt_ECMWF_nomask_aave_da_c201202.nc

Hi, thanks for your help very much. I have downloaded the 1degx1deg grid nc file named "map_1km-merge-10min_HYDRO1K-merge-nomask_to_360x180pt_ECMWF_nomask_aave_da_c201202.nc", it is great helpful.

Woud you like to help me product the 2degx2deg map file by the same way? The map file can be named "map_1km-merge-10min_HYDRO1K-merge-nomask_to_180x90pt_ECMWF_nomask_aave_da_c201202.nc"
Due to the limitation of the RAM of my own computer, I am not able to get it. however, it is also very important to me.

If you can help me, I would greatly appreciate your help. Thanks a lot, waiting for your reply.

 

[oleson]

I have put both of these mapping files on our ftp site.
In the future, I think that katec's suggestion of adapting regridbatch.sh to your computer system is a good one. It would help to share the memory requirements across more nodes.

 

[dayon]

Thanks for your help ans suggestions very much.