Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

Error when excute mkmapdata.sh :unable to detect the source grid file type

majun

Member
Dear Scientist,
Recently I want to run a regional case (centerpoint is 114.1°E,30.4°N,grid number is 10×10, and the grid resolution is 0.1°×0.1° ) using CRUNCEP as forcing data and Jackson as surface datasets. However, there are two questions bothering me:

1. The correct step is using ./mknoocnmap.pl to create scripgrid file ,then using mkmapdata.sh to create mapping file, then using gen_domain to create domain file, finally using mksurfdata.pl to create surface data. Am I right?

2. when using mkmapdata.sh, something wrong occurs.
First,
> cd ~/majun/clm5.0/tools/mkmapdata
> ./mknoocnmap.pl -p 114.1,30.4 -n testwuhan -dx 10 -dy 10
It succeed and produce SCRIP file "map_testwuhan_noocean_to_testwuhan_nomask_aave_da_200327.nc"
Then,
> ./mkmapdata.sh -f map_testwuhan_noocean_to_testwuhan_nomask_aave_da_200327.nc -r 10×10 -t regional
error occurs, it shows:

Script to create mapping files required by mksurfdata_map
query command is ./../../bld/queryDefaultNamelist.pl -silent -namelist clmexp -justvalue -options sim_year=2000 -csmdata /home/shenhf/majun/inputdata

Using user specified scrip grid file: map_wuhantest_noocean_to_wuhantest_nomask_aave_da_200327.nc
Output grid resolution is 10×10
Hostname = centos6
Machine centos6 NOT recognized
Creating mapping file: map_0.5x0.5_AVHRR_to_10×10_nomask_aave_da_c200327.nc
From input grid: /home/shenhf/majun/inputdata/lnd/clm2/mappingdata/grids/SCRIPgrid_0.5x0.5_AVHRR_c110228.nc
For output grid: map_wuhantest_noocean_to_wuhantest_nomask_aave_da_200327.nc

WARNING: deprecated switch -src_type will be ignored. The file type will be detected automatically

WARNING: deprecated switch -dst_type will be ignored. The file type will be detected automatically

ERROR: Unable to detect the source grid file type.
Error status returned from mkmapdata script

I want to know what's the problem, is the problem of inputdata? my inpudata path is export CSMDATA=/home/shenhf/majun/inputdata, what should I do to create surface data successfully? any help or suggestion is highly appreciated.
 

erik

Erik Kluzek
CSEG and Liaisons
Staff member
Out of the box mkmapdata.sh is only setup and tested to run on cheyenne. To get it work on another machine will require installing ESMF and doing some changes to get it to work there. But, another requirement is that because one of the grids used in it is as a 1km global scale, it also requires a ton of memory on whatever machine you want to run it on. You need over 400 GBytes of memory on your machine.
 

majun

Member
Out of the box mkmapdata.sh is only setup and tested to run on cheyenne. To get it work on another machine will require installing ESMF and doing some changes to get it to work there. But, another requirement is that because one of the grids used in it is as a 1km global scale, it also requires a ton of memory on whatever machine you want to run it on. You need over 400 GBytes of memory on your machine.
Thanks a lot , I'm a novice on runngng clm5.0,please don't be annoyed. I have installed ESMF, could you please tell me how to do some change to get it to work ? What's more, I just want to successfully run a regional case in my study area (whatever the grid resolution is, 1° is also fine),The correct step is using ./mknoocnmap.pl to create scripgrid file ,then using mkmapdata.sh to create mapping file, then using gen_domain to create domain file, finally using mksurfdata.pl to create surface data. Am I right?
 

majun

Member
Out of the box mkmapdata.sh is only setup and tested to run on cheyenne. To get it work on another machine will require installing ESMF and doing some changes to get it to work there. But, another requirement is that because one of the grids used in it is as a 1km global scale, it also requires a ton of memory on whatever machine you want to run it on. You need over 400 GBytes of memory on your machine.
Dear professor, Still the same problem, when I am using mkmapdata.sh , it shows
ERROR: Unable to detect the source grid file type.
Error status returned from mkmapdata script do you know what's the problem? thanks
ncl verison is 6.3.0,esmf version is 8.0.0, netcdf-c is 4.6.2 netcdf-fortran is 4.5.2 ,related configurations are attached below:
 

Attachments

  • esmf.png
    esmf.png
    157.8 KB · Views: 39
  • netcdf-c.png
    netcdf-c.png
    80.4 KB · Views: 36
  • netcdf-fortran.png
    netcdf-fortran.png
    77.1 KB · Views: 34

majun

Member
Out of the box mkmapdata.sh is only setup and tested to run on cheyenne. To get it work on another machine will require installing ESMF and doing some changes to get it to work there. But, another requirement is that because one of the grids used in it is as a 1km global scale, it also requires a ton of memory on whatever machine you want to run it on. You need over 400 GBytes of memory on your machine.
hi,erik
Dose ''You need over 400 GBytes of memory on your machine" refers to 1km with global grids or 1km with the number of grids is 100 or no matter how large the study area is?
If each grid resolution is 100mx100m (0.001°x0.001°) or 30mx30m(0.0003°), with dx is 4°,dy is 3°,how much memory and CPU cores are needed?
 

erik

Erik Kluzek
CSEG and Liaisons
Staff member
Because the input grids are global, it doesn't much matter what the output grid size is. Memory is the issue, and not number of processors. If you can run with MPI to get more memory though -- that does help. So on cheyenne we run with 4 processors using 109 GBytes of memory each.
 

majun

Member
Because the input grids are global, it doesn't much matter what the output grid size is. Memory is the issue, and not number of processors. If you can run with MPI to get more memory though -- that does help. So on cheyenne we run with 4 processors using 109 GBytes of memory each.
thanks a lot . Does ''it doesn't much matter what the output grid size is'' refers to size of total grids or each grid resolution ?
 
Top