Run CTSM with own domain: doesn't work since I have updated

[adrienD]

Hi,

I was able to run CTSM (ctsm5.1.dev023) with a domain I have created over the Arctic.

I have updated CTSM with a more recent version (ctsm5.1.dev086) and I can't run CTSM with the same domain anymore. I got the error message

0: ERROR: (dshr_mod:dshr_mesh_init) ERROR: model_meshfile UNSET does not exist

You can find the logs attached.


When I am looking to the env_run.xml file, things have changed. I was used to modify ATM_DOMAIN_FILE and LND_DOMAIN_FILE. See below:

    <entry id="ATM_DOMAIN_FILE" value="domain.ocn.lARC_oARC.220330.nc">
      <type>char</type>
      <desc>atm domain file</desc>
    </entry>
    <entry id="ATM_DOMAIN_PATH" value="$DIN_LOC_ROOT/share/domains">
      <type>char</type>
      <desc>path of atm domain file</desc>
    </entry>
    <entry id="LND_DOMAIN_FILE" value="domain.ocn.lARC_oARC.220330.nc">
      <type>char</type>
      <desc>lnd domain file</desc>
    </entry>
    <entry id="LND_DOMAIN_PATH" value="$DIN_LOC_ROOT/share/domains">
      <type>char</type>
      <desc>path of lnd domain file</desc>
    </entry>

Now this part of the file have changed to:

<entry id="PTS_DOMAINFILE" value="UNSET">
      <type>char</type>
      <desc>used only if if PTS_LAT and PTS_LON are greater than or
          equal to 0.  If this is the case then if PTS_DOMAINFILE is not
          equal to UNSET a nearest neighbor search of PTS_DOMAINFILE using
          PTS_LAT and PTS_LON will be done and the component mesh will have
          this nearest neighbor value. </desc>
    </entry>
    <entry id="LND_DOMAIN_FILE" value="domain.ocn.lARC_oARC.220330.nc">
      <type>char</type>
      <desc>lnd domain file</desc>
    </entry>
    <entry id="LND_DOMAIN_PATH" value="$DIN_LOC_ROOT/share/domains">
      <type>char</type>
      <desc>path of lnd domain file</desc>
    </entry>
    <entry id="ATM_DOMAIN_MESH" value="UNSET">
      <type>char</type>
      <desc>atm mesh file (full pathname)</desc>
    </entry>
    <entry id="LND_DOMAIN_MESH" value="UNSET">
      <type>char</type>
      <desc>lnd mesh file (full pathname)</desc>
    </entry>

I have tried to also change ATM_DOMAIN_MESH and LND_DOMAIN_MESH with my domain domain.ocn.lARC_oARC.220330.nc but it was unsuccessful.

Someone can help me? Is there new documentation about this? For example, this documentation 1.6.3. Running Single Point Configurations — ctsm CTSM master documentation is using variables like ATM_DOMAIN_FILE who are not in the code anymore.


Thanks!

 

[erik]

Hi Adrien

I think the problem is just that NUOPC is now the default coupler, and your setup to work with the MCT coupler. I do recommend updating your setup to with with NUOPC, but as a first step you should try it with recreating your case, but telling create_newcase to use the MCT driver rather than NUOPC (use
--driver mct). Try that out and see if it works for you.

 

[adrienD]

Hi Adrien

I think the problem is just that NUOPC is now the default coupler, and your setup to work with the MCT coupler. I do recommend updating your setup to with with NUOPC, but as a first step you should try it with recreating your case, but telling create_newcase to use the MCT driver rather than NUOPC (use
--driver mct). Try that out and see if it works for you.

Hi Eric,

Thanks for your answer. It did solve the issue but I have another one now.

The MOSART model initialization failed. It didn't happen to me with dev023. I guess I have to change something in env_run.xml, but I don't really know what.

(rof_init_mct) :MOSART model initialization
  mosart npes =           24
  mosart iam  =            0
  inst_name = ROF
 Read in mosart_inparm namelist from: mosart_in
 define run:
    run type              = initial
    coupling_period       =        10800
    delt_mosart           =         3600
    decomp option         = roundrobin
    bypass_routing option = direct_in_place
    qgwl runoff option    = threshold
    smat option           = Xonly
    MOSART river data       = UNSET
  ******** RTM Time Manager Configuration ********
   Calendar type:            NO_LEAP
   Timestep size (seconds):         10800
   Start date (yr mon day tod):             1991           1           1
           0
   Stop date (yr mon day tod):              1992           1           1
           0
   Reference date (yr mon day tod):         1991           1           1
           0
   Current step number:                 0
   Current date (yr mon day tod):           1991           1           1
           0
  ************************************************
 MOSART tracers =            2 LIQ:ICE
 (GETFIL): attempting to find local file UNSET
 (GETFIL): failed getting file from full path:
 UNSET                                                                         
                                                                                
                                                                                
                    
 ERROR: GETFIL: FAILED to get UNSET

 

[adrienD]

I have just realized I posted this in the wrong forum, my apologies. Can a moderator move it to CTSM, CLM, MOSART, RTM

 

[oleson]

I'll move this, thanks.

 

[oleson]

It is trying to find the MOSART routing file which should be showing up in CaseDocs/mosart_in:

frivinp_rtm = "/glade/p/cesmdata/cseg/inputdata/rof/mosart/MOSART_routing_Global_0.5x0.5_c170601.nc"

where the path should be specific to your machine.
What does your mosart_in file look like?

 

[oleson]

Although I guess you have your own routing file for this regional simulation?

 

[adrienD]

Thanks @oleson for you help.

I am not sure what you mean by routing file? I have attached my env_run.xml and user_nl_clm file.

This is how my mosart_in look like:

&mosart_inparm
  bypass_routing_option = "direct_in_place"
  coupling_period = 10800
  decomp_option = "roundrobin"
  delt_mosart = 3600
  do_rtmflood = .false.
  finidat_rtm = " "
  frivinp_rtm = "UNSET"
  ice_runoff = .true.
  qgwl_runoff_option = "threshold"
  rtmhist_fexcl1 = ""
  rtmhist_fexcl2 = ""
  rtmhist_fexcl3 = ""
  rtmhist_fincl1 = ""
  rtmhist_fincl2 = ""
  rtmhist_fincl3 = ""
  rtmhist_mfilt = 1
  rtmhist_ndens = 1
  rtmhist_nhtfrq = 0
  smat_option = "Xonly"
/

And this is how my mosart_in look like in my previous successful simulations:

&mosart_inparm
  bypass_routing_option = "direct_in_place"
  coupling_period = 10800
  decomp_option = "roundrobin"
  delt_mosart = 3600
  do_rtm = .false.
  do_rtmflood = .false.
  finidat_rtm = " "
  frivinp_rtm = "/work/aa0049/a271098/inputdata/"
  ice_runoff = .true.
  qgwl_runoff_option = "threshold"
  rtmhist_fexcl1 = ""
  rtmhist_fexcl2 = ""
  rtmhist_fexcl3 = ""
  rtmhist_fincl1 = ""
  rtmhist_fincl2 = ""
  rtmhist_fincl3 = ""
  rtmhist_mfilt = 1
  rtmhist_ndens = 1
  rtmhist_nhtfrq = 0
  smat_option = "Xonly"
/

What do you recommend me to do?

 

[adrienD]

I added the line

frivinp_rtm="/work/aa0049/a271098/inputdata/rof/mosart/MOSART_routing_Global_0.5x0.5_c170601.nc"

to mosart_nl_mosart, and the problem was solved. But I don't know if it's ideal.

I have another issue right now "ERROR: (shr_mct_sMatReaddnc)No such file or directory". I guess it's a mapping matrix, is it related to the previous issue?

I have attached my log files.

 

[oleson]

Right, your frivinp_rtm was set to UNSET previously which is why you got the first error. Changing that to the default routing path and file fixed that.
The new error seems to be related to this line in prep_lnd_mod.F90:

call seq_map_init_rcfile(mapper_Fr2l, rof(1), lnd(1), &
'seq_maps.rc','rof2lnd_fmapname:','rof2lnd_fmaptype:',samegrid_lr, &
string='mapper_Fr2l initialization',esmf_map=esmf_map_flag)

So it is looking for a file that maps the rof (mosart) grid to the lnd grid. Normally, this is set in env_run.xml by variable ROF2LND_FMAPNAME.
I guess I would look at what you have set there compared to what it was set to previously.
However, one question is why did this work before. Are you sure you were using MOSART in the previous simulation that worked? If so, then I think you would have needed to generate a mapping file appropriate for a regional simulation.
Overall, I suggest comparing the new and old xml files and namelist files to see what might be different.
I'm not an expert in setting up regional simulations. You might also need a regional routing file. @erik or @swensosc , do you have any suggestions?

 

[erik]

Yes, so the first question is if you really want to run the river routing model over your regional grid. To do so you'll need to subset the rivers for the global frivinp_rtm file as well as get the mapping files in place that @oleson pointed out. So it's quite a bit of work to get it setup. Someone else asked a question about running MOSART on a regional grid, you can look at that discussion. So are you going to be looking at river data for your simulation? If not, you might as well turn MOSART off for this. If you do want to, you'll need to go through the work of getting the mapping files and frivinp_rtm file for your region.

 

[adrienD]

Yes, so the first question is if you really want to run the river routing model over your regional grid. To do so you'll need to subset the rivers for the global frivinp_rtm file as well as get the mapping files in place that @oleson pointed out. So it's quite a bit of work to get it setup. Someone else asked a question about running MOSART on a regional grid, you can look at that discussion. So are you going to be looking at river data for your simulation? If not, you might as well turn MOSART off for this. If you do want to, you'll need to go through the work of getting the mapping files and frivinp_rtm file for your region.

Thanks @oleson and @erik for your help.

I believe that in my previous simulations I was not using a river routing model, but how can I know that?

My guess was that in dev026, the default mosart_in include a parameter do_rtm = .false. (line 6), which is not used anymore in dev086. Therefore, I was not using a river routing model before.

The question is how I can turn off river routing model now?

 

[oleson]

The most basic way to turn it off would be to specify the stub runoff model when you create your case. E.g., for an I2000Clm50BgcCrop compset, the longname is:

2000_DATM%GSWP3v1_CLM50%BGC-CROP_SICE_SOCN_MOSART_SGLC_SWAV_SESP

To specify the stub runoff model, you would substitute SROF for MOSART,

2000_DATM%GSWP3v1_CLM50%BGC-CROP_SICE_SOCN_SROF_SGLC_SWAV_SESP

Alternatively, if your case is already set up you should be able to turn it off using:

./xmlchange MOSART_MODE=NULL

MOSART_MODE is a variable in env_build.xml
You might need to setup your case again, e.g., ./case.setup or possibly ./case.setup --reset

 

[adrienD]

I have used the longname compset and it's working like before, thank you very much!

However, ./xmlchange MOSART_MODE=NULL was not working in my case. Maybe, it's a variable that can be used only with the NUOPC coupler?

@erik Do you have some documentation to update my setup with the coupler NUOPC in the future?

 

[oleson]

The xmlchange command won't work on a case where you've specified the stub runoff model because that variable doesn't exist in env_build.xml