CESM2 Compset F2000climo res f09_g16 running error

[23lukia@tongji_edu_cn]

Hi, I tried to run a case with compset of F2000climo (Is it similar to F2000 in CESM 1.2.2?) and the resolution of f09_g16.
The partial error message of cesm.log is as follow (The whole file is also attached):
I just create case then setup, build and run, have not modified any config files yet.
Could you help me?


00: Writing to file f2000climo_1.cism.initial_hist.0001-01-01-00000.nc at time
00: 0.000000000000000E+000
00: MCT::m_Router::initp_: GSMap indices not increasing...Will correct
00: MCT::m_Router::initp_: RGSMap indices not increasing...Will correct
00: MCT::m_Router::initp_: RGSMap indices not increasing...Will correct
00: MCT::m_Router::initp_: GSMap indices not increasing...Will correct
19: inconsistency between land fraction and sea ice fraction
19: n= 1 fracl= 0.000000000000000E+000 fraci=
19: 0.709719049622650 sum= 0.709719049622650
20: inconsistency between land fraction and sea ice fraction
20: n= 5 fracl= 0.000000000000000E+000 fraci=
20: 0.540082232485649 sum= 0.540082232485649
20: ERROR:
20: (seq_domain_check) inconsistency between land fraction and sea ice fraction
21: inconsistency between land fraction and sea ice fraction
21: n= 13 fracl= 0.000000000000000E+000 fraci=
21: 0.588340027069572 sum= 0.588340027069572
21: ERROR:
21: (seq_domain_check) inconsistency between land fraction and sea ice fraction
22: inconsistency between land fraction and sea ice fraction
22: n= 47 fracl= 0.000000000000000E+000 fraci=
22: 0.921239338239427 sum= 0.921239338239427
22: ERROR:
22: (seq_domain_check) inconsistency between land fraction and sea ice fraction
23: inconsistency between land fraction and sea ice fraction
18: inconsistency between land fraction and sea ice fraction
18: n= 2255 fracl= 0.000000000000000E+000 fraci=
18: 0.000000000000000E+000 sum= 0.000000000000000E+000
18: ERROR:
18: (seq_domain_check) inconsistency between land fraction and sea ice fraction
19: ERROR:
19: (seq_domain_check) inconsistency between land fraction and sea ice fraction
23: n= 225 fracl= 0.000000000000000E+000 fraci=
23: 0.971003402798802 sum= 0.971003402798802
23: ERROR:
23: (seq_domain_check) inconsistency between land fraction and sea ice fraction
19: Image PC Routine Line Source
19: cesm.exe 000000000259E08A Unknown Unknown Unknown
19: cesm.exe 000000000230BDAD shr_abort_mod_mp_ 114 shr_abort_mod.F90
19: cesm.exe 000000000230BE5A shr_abort_mod_mp_ 61 shr_abort_mod.F90
19: cesm.exe 000000000049ABBA seq_domain_mct_mp 421 seq_domain_mct.F90
19: cesm.exe 000000000043660A cime_comp_mod_mp_ 1732 cime_comp_mod.F90
19: cesm.exe 000000000043D5E7 MAIN__ 114 cime_driver.F90
19: cesm.exe 000000000041E7DE Unknown Unknown Unknown
19: libc-2.17.so 00002AFF135F7B35 __libc_start_main Unknown Unknown
19: cesm.exe 000000000041E6E9 Unknown Unknown Unknown
20: Image PC Routine Line Source
20: cesm.exe 000000000259E08A Unknown Unknown Unknown
20: cesm.exe 000000000230BDAD shr_abort_mod_mp_ 114 shr_abort_mod.F90
20: cesm.exe 000000000230BE5A shr_abort_mod_mp_ 61 shr_abort_mod.F90
20: cesm.exe 000000000049ABBA seq_domain_mct_mp 421 seq_domain_mct.F90
20: cesm.exe 000000000043660A cime_comp_mod_mp_ 1732 cime_comp_mod.F90
20: cesm.exe 000000000043D5E7 MAIN__ 114 cime_driver.F90
20: cesm.exe 000000000041E7DE Unknown Unknown Unknown
20: libc-2.17.so 00002B1D4EB21B35 __libc_start_main Unknown Unknown
20: cesm.exe 000000000041E6E9 Unknown Unknown Unknown
20: application called MPI_Abort(MPI_COMM_WORLD, 1001) - process 20
21: Image PC Routine Line Source
21: cesm.exe 000000000259E08A Unknown Unknown Unknown
21: cesm.exe 000000000230BDAD shr_abort_mod_mp_ 114 shr_abort_mod.F90
21: cesm.exe 000000000230BE5A shr_abort_mod_mp_ 61 shr_abort_mod.F90
21: cesm.exe 000000000049ABBA seq_domain_mct_mp 421 seq_domain_mct.F90
21: cesm.exe 000000000043660A cime_comp_mod_mp_ 1732 cime_comp_mod.F90
21: cesm.exe 000000000043D5E7 MAIN__ 114 cime_driver.F90
21: cesm.exe 000000000041E7DE Unknown Unknown Unknown
21: libc-2.17.so 00002ADDB5F35B35 __libc_start_main Unknown Unknown
21: cesm.exe 000000000041E6E9 Unknown Unknown Unknown
21: application called MPI_Abort(MPI_COMM_WORLD, 1001) - process 21
18: Image PC Routine Line Source
18: cesm.exe 000000000259E08A Unknown Unknown Unknown
18: cesm.exe 000000000230BDAD shr_abort_mod_mp_ 114 shr_abort_mod.F90
18: cesm.exe 000000000230BE5A shr_abort_mod_mp_ 61 shr_abort_mod.F90
18: cesm.exe 000000000049ABBA seq_domain_mct_mp 421 seq_domain_mct.F90
18: cesm.exe 000000000043660A cime_comp_mod_mp_ 1732 cime_comp_mod.F90
18: cesm.exe 000000000043D5E7 MAIN__ 114 cime_driver.F90
18: cesm.exe 000000000041E7DE Unknown Unknown Unknown
18: libc-2.17.so 00002B5AF2B98B35 __libc_start_main Unknown Unknown
18: cesm.exe 000000000041E6E9 Unknown Unknown Unknown
18: application called MPI_Abort(MPI_COMM_WORLD, 1001) - process 18
19: application called MPI_Abort(MPI_COMM_WORLD, 1001) - process 19
22: Image PC Routine Line Source
22: cesm.exe 000000000259E08A Unknown Unknown Unknown
22: cesm.exe 000000000230BDAD shr_abort_mod_mp_ 114 shr_abort_mod.F90
22: cesm.exe 000000000230BE5A shr_abort_mod_mp_ 61 shr_abort_mod.F90
22: cesm.exe 000000000049ABBA seq_domain_mct_mp 421 seq_domain_mct.F90
22: cesm.exe 000000000043660A cime_comp_mod_mp_ 1732 cime_comp_mod.F90
22: cesm.exe 000000000043D5E7 MAIN__ 114 cime_driver.F90
22: cesm.exe 000000000041E7DE Unknown Unknown Unknown
22: libc-2.17.so 00002B66533BDB35 __libc_start_main Unknown Unknown
22: cesm.exe 000000000041E6E9 Unknown Unknown Unknown
22: application called MPI_Abort(MPI_COMM_WORLD, 1001) - process 22
23: Image PC Routine Line Source
23: cesm.exe 000000000259E08A Unknown Unknown Unknown
23: cesm.exe 000000000230BDAD shr_abort_mod_mp_ 114 shr_abort_mod.F90
23: cesm.exe 000000000230BE5A shr_abort_mod_mp_ 61 shr_abort_mod.F90
23: cesm.exe 000000000049ABBA seq_domain_mct_mp 421 seq_domain_mct.F90
23: cesm.exe 000000000043660A cime_comp_mod_mp_ 1732 cime_comp_mod.F90
23: cesm.exe 000000000043D5E7 MAIN__ 114 cime_driver.F90
23: cesm.exe 000000000041E7DE Unknown Unknown Unknown
23: libc-2.17.so 00002B0F1BB0AB35 __libc_start_main Unknown Unknown
23: cesm.exe 000000000041E6E9 Unknown Unknown Unknown
23: application called MPI_Abort(MPI_COMM_WORLD, 1001) - process 23
00: slurmd[cn3882]: *** STEP 4743063.0 KILLED AT 2021-11-02T15:22:15 WITH SIGNAL 9 ***
yhrun: Job step aborted: Waiting up to 2 seconds for job step to finish.
00: slurmd[cn3882]: *** STEP 4743063.0 KILLED AT 2021-11-02T15:22:15 WITH SIGNAL 9 ***
00: slurmd[cn3882]: *** STEP 4743063.0 KILLED AT 2021-11-02T15:22:15 WITH SIGNAL 9 ***
00: slurmd[cn3882]: *** STEP 4743063.0 KILLED AT 2021-11-02T15:22:15 WITH SIGNAL 9 ***
00: slurmd[cn3882]: *** STEP 4743063.0 KILLED AT 2021-11-02T15:22:15 WITH SIGNAL 9 ***
00: slurmd[cn3882]: *** STEP 4743063.0 KILLED AT 2021-11-02T15:22:15 WITH SIGNAL 9 ***
00: slurmd[cn3882]: *** STEP 4743063.0 KILLED AT 2021-11-02T15:22:15 WITH SIGNAL 9 ***
yhrun: error: cn3882: tasks 0-23: Killed

 

[cacraig]

I will answer the couple of questions you had.

Yes, F2000climo is the replacement for F2000 (we decided that the climo addition to the compset name indicated more completely what this compset is doing).

f09_g16 has been replaced with f09_g17 in CESM2.2. The ocn/ice components use the g17 grid now and the g16 grid is incompatible in this version. That said, the f09_g17 grid is not a supported grid for F2000climo while the f09_f09_mg17 is a tested and supported grid. The difference between these two grids is that the f09_f09_mg17 uses the f09 grid for the ocn/ice components.

The complete description of the g17 ocn/ice grid from the web page CESM2.2.0 Grid Resolution Definitions is:
gx1v7 is displaced Greenland pole 1-deg grid with Caspian as a land feature: with domain file(s):
$DIN_LOC_ROOT/share/domains/domain.ocn.gx1v7.151008.nc (only for grid match: atm|lnd)
$DIN_LOC_ROOT/share/domains/domain.ocn.gx1v7.151008.nc (only for grid match: ocnice)

 

[23lukia@tongji_edu_cn]

I will answer the couple of questions you had.

Yes, F2000climo is the replacement for F2000 (we decided that the climo addition to the compset name indicated more completely what this compset is doing).

f09_g16 has been replaced with f09_g17 in CESM2.2. The ocn/ice components use the g17 grid now and the g16 grid is incompatible in this version. That said, the f09_g17 grid is not a supported grid for F2000climo while the f09_f09_mg17 is a tested and supported grid. The difference between these two grids is that the f09_f09_mg17 uses the f09 grid for the ocn/ice components.

The complete description of the g17 ocn/ice grid from the web page CESM2.2.0 Grid Resolution Definitions is:
gx1v7 is displaced Greenland pole 1-deg grid with Caspian as a land feature: with domain file(s):
$DIN_LOC_ROOT/share/domains/domain.ocn.gx1v7.151008.nc (only for grid match: atm|lnd)
$DIN_LOC_ROOT/share/domains/domain.ocn.gx1v7.151008.nc (only for grid match: ocnice)

Thank you for your reply. I ran the case successfully. But I found that the run time of CESM2 F2000climo (more than 3 hours) is much longer than F2000 of CESM1.2.2 (about 6 min). Is there any way that can shorten the run time, i.e. turn off some parameterization scheme? (I want to simulate the North Atlantic Oscillation).
Looking forward to your reply, thanks a lot.

 

[andrew]

CAM6 should only be about 2x more expensive than CAM5 (in CESM1). So something else is obviously going on. I don't think CESM2 should be 30x more expensive than CESM1 (180min v. 6min) so something else is different in your configuration. Maybe the default machine layout and nodes used is different for the default for CESM2.

There are not large sections of code you can shut off. Aerosols are running in both CESM1 and CESM2, but not full chemistry.