Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

Unable to run CESM1.2.2 with POP2

ceppi

New Member
Dear CESM community,

I've been successfully running CESM1.2.2 on a particular supercomputer for years, either in AGCM or in atmosphere-slab ocean mode. However, when setting up CESM in AOGCM mode, the model always crashes at the start of the run, even though it has built properly.

I'm using compset B1850, and have tried two different grids (f19_g16, f45_g37), so it seems unlikely to be something specific to the choice of atmos/ocean grids.

The error message in cesm.log looks like this:
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
libintlc.so.5 00002AAAAB732BF1 tbk_trace_stack_i Unknown Unknown
libintlc.so.5 00002AAAAB730D2B tbk_string_stack_ Unknown Unknown
cesm.exe 0000000001D61E84 Unknown Unknown Unknown
cesm.exe 0000000001D61C96 Unknown Unknown Unknown
cesm.exe 0000000001CDEE47 Unknown Unknown Unknown
cesm.exe 0000000001CE9660 Unknown Unknown Unknown
libpthread-2.11.3 00002AAAACAC3850 Unknown Unknown Unknown
cesm.exe 0000000001311818 overflows_mp_ovf_ 5612 overflows.F90
cesm.exe 000000000151E8E1 initial_mp_pop_in 345 initial.F90
cesm.exe 00000000013C9887 pop_initmod_mp_po 102 POP_InitMod.F90
cesm.exe 00000000012F9A82 ocn_comp_mct_mp_o 261 ocn_comp_mct.F90
cesm.exe 0000000000435E89 ccsm_comp_mod_mp_ 1130 ccsm_comp_mod.F90
cesm.exe 00000000004385A8 MAIN__ 90 ccsm_driver.F90
cesm.exe 000000000041603E Unknown Unknown Unknown
libc-2.11.3.so 00002AAAACCF0C36 __libc_start_main Unknown Unknown
cesm.exe 0000000000415EC9 Unknown Unknown Unknown


I've checked the overflows.F90 source file, but am unable to identify the problem. Any help would be appreciated. Thank you!
 
In case this is useful to anyone else, here's a fix I found in this post Run T31_g37 B1850C5 compset error: malloc(): memory corruption (fast)

At line 5612, the line
HUM(:,:,:) = HU(:,:,:)
needs to be replaced by
!$OMP PARALLEL DO PRIVATE(iblock,i,j)
do iblock = 1,numBlocksClinic
do j=1,POP_nyBlock
do i=1,POP_nxBlock
HUM(i,j,iblock) = HU(i,j,iblock)
enddo
enddo
enddo
!$OMP END PARALLEL DO
Hi Ceppi,

Thanks for your post. After I replaced the original line with three do loops, my model ran pretty slow. Sometimes it just hangs there and did nothing. Did this happen to you?

Thank you in advance,
Shizhu
 

ceppi

New Member
Hi Ceppi,

Thanks for your post. After I replaced the original line with three do loops, my model ran pretty slow. Sometimes it just hangs there and did nothing. Did this happen to you?

Thank you in advance,
Shizhu
Hi Shizhu,

I don't remember having this issue, no... Sorry for not being able to offer a fix!

Paulo
 
Top