Main menu

Navigation

error: ice_transport_remap

11 posts / 0 new
Last post
s.l.bradley-1@...
error: ice_transport_remap

Hello,

I am running CESM2.1.0 - B case for an LGM settings (f09_g17) 

compset : 1850_CAM60_CLM50%BGC_CICE_POP2_RTM_CISM2%NOEVOLVE_SWAV

I am trying to start the model using restarts from 

(a)  CAM/CLM/CICE: PI run 

b.e20.B1850.f09_g17.pi_control.all.299

(b) POP (LGM CCSM4) 

/gpfs/fs1/scratch/erik/b40.lgm21ka.1deg.003/rest/1846-01-01-00000

The 'B' case runs succesfully when not using the restart files. 

However we get an error related to the ice_transport_remap [cesm.log.4983760.chadmin1.190411-070243]

88173 716: Warning: Departure points out of bounds in remap

88174 716: my_task, i, j =         104          10           8

88175 716: dpx, dpy =   18853.1040333448        4399.03128628862

88176 716: HTN(i,j), HTN(i+1,j) =   15418.6525795818        15277.6516127053

88177 716: HTE(i,j), HTE(i,j+1) =   55727.0484591510        55918.9173556769

88178 716: istep1, my_task, iblk =           3         104           2

88179 716: Global block:         624

88180 716: Global i and j:          57         379

88181 716: remap transport: bad departure points

88182 716: ERROR: remap transport: bad departure points

88183 674:Image              PC                Routine            Line        Source

88184 674:cesm.exe           000000000343DB6D  Unknown               Unknown  Unknown

88185 674:cesm.exe           0000000002BAE022  shr_abort_mod_mp_         114  shr_abort_mod.F90

88186 674:cesm.exe           0000000001754524  ice_exit_mp_abort          46  ice_exit.F90

88187 674:cesm.exe           000000000199A252  ice_transport_rem         560  ice_transport_remap.F90

88188 674:cesm.exe           000000000198D6F4  ice_transport_dri         474  ice_transport_driver.F90

88189 674:cesm.exe           0000000001977BFB  ice_step_mod_mp_s        1208  ice_step_mod.F90

88190 674:cesm.exe           000000000183F212  cice_runmod_mp_ci         205  CICE_RunMod.F90

88191 674:cesm.exe           0000000001745CBA  ice_comp_mct_mp_i         563  ice_comp_mct.F90

88192 674:cesm.exe           0000000000425864  component_mod_mp_         728  component_mod.F90

88193 674:cesm.exe           000000000040AEBE  cime_comp_mod_mp_        2699  cime_comp_mod.F90

88194 674:cesm.exe           000000000042550C  MAIN__                    125  cime_driver.F90

88195 674:cesm.exe           0000000000408C9E  Unknown               Unknown  Unknown

88196 674:libc-2.19.so       00002AAAB08F9B25  __libc_start_main     Unknown  Unknown

88197 674:cesm.exe           0000000000408BA9  Unknown               Unknown  Unknown

 

88198 672:Image              PC                Routine            Line        Source

 

Following looking at the discussion board, I can see that suggestions for this error are: 

(a) increase ndtd =2

(b) ATM_NCPL = 144

This led to error with ATM: with the FV subcycling: 

atm.log.4983830.chadmin1.190411-072126

When using the revised FV setting: 

(C) user_nl_atm:  

fv_nsplit   =            4

fv_nspltrac =            2

fv_nspltvrm =            2


Trying all these suggesions, leads to a new error (see log with *

 

4983867.chadmin1.190411-073836)

 component_mod:check_fields NaN found in ATM instance:    1 field Sa_z 1d global

88951 207:  index:    25441

88952 513: ERROR:

88953 513: component_mod:check_fields NaN found in ATM instance:    1 field Sa_z 1d global

88954 513:  index:    25463

88955 9: ERROR:

88956 9: component_mod:check_fields NaN found in ATM instance:    1 field Sa_z 1d global

88957 9:  index:    25753

88958 8: ERROR:

88959 8: component_mod:check_fields NaN found in ATM instance:    1 field Sa_z 1d global

88960 8:  index:    25751

88961 257: ERROR:

 

88962 257: component_mod:check_fields NaN found in ATM instance:    1 field Sa_z 1d global


I am then assuming that the steps we followed to resolve our CICE error where not the correct approach


Do you have any suggestions?

 I have uploaded all the log files



dbailey

Is your land/ocean mask the same as the pre-industrial case? It sounds like you have a mismatch here.

Dave

s.l.bradley-1@...

Hi Dave

 

Sorry, I did not add this to the questions. 

Yes our run has different land-ocean mask. 

However when I run with all PI - restarts:

 

a)  CAM/CLM/CICE/POP: PI run 

b.e20.B1850.f09_g17.pi_control.all.299

The model runs - for 5 days - with no problems. This gave me confidence that the change in the land/ocean mask between PI restart files and our new LGM set up did not seem to be a problem.

So this was why I was a little confused why when changing the POP to be an LGM restart file - issue arose. 

 

Thanks

 

Sarah 

dbailey

I'm still confused, but it sounds like your POP restart/initial file is not providing data in an area where your run needs it. This would cause the system to go haywire. Can you get hourly or daily history files before the crash? This would be a way to see where the problem is.

Dave

s.l.bradley-1@...

Hi David

First - an update. I revised the FV settings for CAM and then added your great suggestions of setting hourly output we now have an error which is related to POP.

This is great news - as we can try to work with this: 

I have attached the new cesm.log file

 

:POP aborting...

88979 863: ERROR: k.e. > 100

88980 863:

dbailey

I don't see the attachment. How many timesteps does it run before this error? It sounds like it is just blowing up from the initialization. You can try dramatically increasing dt_count in POP to 500 or something for a year or so.

Dave

s.l.bradley-1@...

Hi David

 

Is the cesm.log not attached - I have attached again 

it does not run at all - not even one time step

We tried making it dt_count=96 and we thought that was already large. Okay - I will try 500! 

 

Let see

 

Sarah 

dbailey

If it is crashing on the first step, you might have to initialize differently. Sounds like you might not be able to use the velocities from the POP restart. In this case, you can try a T/S initialization.

 

Dave

s.l.bradley-1@...

Hi David

 

I tried again with dt_count = 500 and unfortunately, that did not resolve the ke> 100 error.

 

In the case notes from the older ccsm4 LGM run - it suggests that the POP restart files can cause this error: k.e > 100 

To resovle you should set the velocities.surface pressure etc all to zero and this will resolve  it.

Unfortunately that still did not resolve this issue.

 

I am unsure what other method of initialising the T/S: we are using the restart from the CCSM4 LGM run so that we can reduce the amount of spin up time for the ocean for our B run, which we expect will have to be ~ 300 - 500 years.

Perhaps we should be changing something else when we read in the T/S from the restart files? 

 

Do you have any other suggestions? 

Thanks 

dbailey

Is this a hybrid run? That is, is POP doing a "startup_spunup"? This might be the issue. I'm really not familiar enough with POP to help here.

Dave

s.l.bradley-1@...

Hi dave

 

Yes it is: thanks for all your help with this: it did resolve the issues with the CICE which was a big hurdle. 

I have contacted, with Erik Kluzek assiatance someone from the PWG group. As you say - they are more familar with the CCSM4 Pop issues

 

Sarah 

Log in or register to post comments

Who's new

  • afroberts@...
  • richard.allard@...
  • tracy@...
  • 498749303@...
  • lijundesio@...