MOM6 open boundaries fail

[ThomasNeumann]

I have set a regional model in the Baltic Sea area with MOM6. However, OBCs seem not to work properly:
Has anybody ever set up an open boundary other than with I,J = 0 or N? That is, at the boundary of the model domain.
My model works fine with the OBC at the boundary of the model domain when I or J is N or 0. But it will not work if the OBC is inside the model domain e.g.
OBC_SEGMENT_001 = "J=585,I=128:50,FLATHER,ORLANSKI".
The model crashes imediately after a couple of time steps
FATAL from PE 72: MOM_regridding: adjust_interface_motion() - implied h<0 is larger than roundoff!

Curiously, it works with SIMPLE, and SALT and TEMP set to zero
OBC_SEGMENT_001 = "J=188,I=410:408,SIMPLE"
OBC_SEGMENT_001_DATA = "SSH=value:0.3,U=value:-0.0,V=value:-0.1,TEMP=value:0.0,SALT=value:0.0"
If I set an infinitesimal small value (e.g. 1e-10) for SALT or TEMP, the model crashes. It is the same with the velocity normal to the OBC.

If I zero out the fluxes (flux_x/y) at the OBC in MOM_tracer_advect.F90 , the model does not crash. Of course, this is not what we want.

Has anybody experienced this OBC behavior or has some hints for a workaround? Thanks.

Best, Thomas.

 

[kshedstrom@alaska_edu]

Is it failing inside or outside of the OBC? The model has to have all land mask outside the OBC. There's a MASK_OUTSIDE_OBCS you can try setting to True.

 

[ThomasNeumann]

It fails inside. I also made a simple test: I have successfully running a model for the entire Baltic Sea with a West OBC towards the North Sea at I=0. Then I set the OBC to I=1 and MASK_OUTSIDE_OBCS = True. The Bathymetry is homogenious abput 10 grid points at the OBC. The model with OBC I=1 fails after a few time steps. The error message is not always the same, moreover it depends on the settings. This time it was: e.g. FATAL from PE 74: lookup_es: saturation vapor pressure table overflow, nbad= 7
It is the same behavior for all OBC directions, I tried it out.
I assume that there goes something wrong with prescribing the tranports at the OBC in case it it is not located at the outermost grid point.
I also guess it starts in the barotropic mode. Looking at diagnostics from every time step, at the first appearance already a couple of grid cells are affected (NaN in salt, temp, u, etc.). And it starts at the OBC inside the model.
I will try to dig deeper into this problem, but also hope that somebody has some advice or similar experiences.

 

[kshedstrom@alaska_edu]

I tried a little Bering Sea test with the boundary at i=2. There are two problems with it:

1. The ocean land mask got set for i=2, but i=1 is still water.
2. The ice land mask is still water for both points.

My prior tests were all ocean-only. I would say the MASK_OUTSIDE_OBCS needs fixing and that the ice model needs more work on the OBCs before we can fully support this. Thanks for reporting this!

 

[ThomasNeumann]

Thanks for trying out. Yes, I read that OBC are not working with SIS. I thought that drifting sea ice is not handled at the OBC, but I am not sure that this will cause a model crash. Below is my model bathymetry. It is a coastal region in the Baltic Sea and the horizontal resolution is 200m. OBC 1 and 2 work fine, but OBC 3-6 do not work. If I move OBC 1 or 2 inwards the model domain, they also do not work. The second figure is wet from sea_ice_geometry.nc. Thus it seems that SIS knows the land sea mask without MASK_OUTSIDE_OBCS. I also think that MASK_OUTSIDE_OBCS is not needed because the bathymetry is masked out outside the OBCs. That is, I doubt a bit that SIS is the problem, however I am not the expert.

Thanks for helping me!

PS: I have a similar setup running with MOM5 but I like much more the MOM6 now ;-)

 

[kshedstrom@alaska_edu]

Do you have IGNORE_FLUXES_OVER_LAND? It is default False, but I have True. I submitted a PR to fix my original land mask trouble, but if your land mask is correct, then that doesn't account for your troubles.

 

[ThomasNeumann]

IGNORE_FLUXES_OVER_LAND is specific for ePBL; I am using KPP. However, I turned on ePBL and IGNORE_FLUXES_OVER_LAND, but it didn't changed anything. I think your land mask trouble (1. in your post) is for me the case if MASK_OUTSIDE_OBCS = False, otherwise the ocean is masked out outside the domain.
I applied only OBC 2 and OBC 3, from my figure above, with the same setting except that for OBC 2 J=N and for OBC 3 J=408. T,S,U,V look very similar for the first time steps. Even the high frequency eta outout from the BT solver looks similar for both OBCs. However, after about 100 BT time steps, eta becomes NaN at the OBC 3 for a couple of grid points.
I am using a git clone from NOAA-GFDL/MOM6:dev/gfdl from about three months ago.

What is PR? Probably, I should also submit a PR because I am almost on my wits' end.

 

[kshedstrom@alaska_edu]

Once I fixed the ocean's view of the land mask then my troubles went away (issue #1). I could look into it if I could reproduce your trouble. Issue #2 isn't troubling me, but the mis-match in ocean-ice land masks is where there is no ice in this particular test. I could try moving a boundary where there's ice, but not today.

A PR is a github pull request, a way to merge in updates. I can propose changes, but someone else has to evaluate and merge them. You should feel free to create an issue instead.