cesm_1_0_3 and mcmodel=medium

[bahls@arsc_edu]

We're attempting to compile build CESM 1_0_3 with the following create_newcase command:

RUN=COMPILE
CASENAME=TEST3
CASEDIR=$WORKDIR/$CASENAME
MACH=midnight

./create_newcase -case $CASEDIR
-mach $MACH
-res f05_t12 -compset G
-scratchroot $WORKDIR/CESM/temp
-din_loc_root_csmdata $WORKDIR/CESM/data
-max_tasks_per_node 16

For some reason we get an error message similar to the following during linking of ccsm.exe:

mpif90 -mcmodel=medium -o /wrkdir/bahls/TEST3/run/ccsm.exe ccsm_comp_mod.o ccsm_driver.o map_atmatm_mct.o map_atmice_mct.o map_atmlnd_mct.o map_atmocn_mct.o map_glcglc_mct.o map_iceice_mct.o map_iceocn_mct.o map_lndlnd_mct.o map_ocnocn_mct.o map_rofocn_mct.o map_rofrof_mct.o map_snoglc_mct.o map_snosno_mct.o mrg_x2a_mct.o mrg_x2g_mct.o mrg_x2i_mct.o mrg_x2l_mct.o mrg_x2o_mct.o mrg_x2s_mct.o seq_avdata_mod.o seq_diag_mct.o seq_domain_mct.o seq_flux_mct.o seq_frac_mct.o seq_hist_mod.o seq_rearr_mod.o seq_rest_mod.o -L/wrkdir/bahls/TEST3/lib -latm -llnd -lice -locn -lglc -L/wrkdir/bahls/TEST3/lib -lcsm_share -lmct -lmpeu -lpio -L/wrkdir10/bahls/netcdf/lib -lnetcdf -mcmodel=medium
/wrkdir10/bahls/netcdf/lib/libnetcdf.a(netcdf.o): In function `netcdf_nf90_put_att_one_onebyteint_':
/import/wrkdir10/bahls/netcdf/wrk/netcdf-3.6.3/f90/./netcdf_attributes.f90:97: relocation truncated to fit: R_X86_64_PC32 against `.bss'
/import/wrkdir10/bahls/netcdf/wrk/netcdf-3.6.3/f90/./netcdf_attributes.f90:97: relocation truncated to fit: R_X86_64_PC32 against `.bss'
/wrkdir10/bahls/netcdf/lib/libnetcdf.a(netcdf.o): In function `netcdf_nf90_get_att_one_onebyteint_':
/import/wrkdir10/bahls/netcdf/wrk/netcdf-3.6.3/f90/./netcdf_attributes.f90:117: relocation truncated to fit: R_X86_64_PC32 against `.bss'
/import/wrkdir10/bahls/netcdf/wrk/netcdf-3.6.3/f90/./netcdf_attributes.f90:117: relocation truncated to fit: R_X86_64_PC32 against `.bss'
/wrkdir10/bahls/netcdf/lib/libnetcdf.a(netcdf.o): In function `netcdf_nf90_put_att_one_twobyteint_':
/import/wrkdir10/bahls/netcdf/wrk/netcdf-3.6.3/f90/./netcdf_attributes.f90:137: relocation truncated to fit: R_X86_64_PC32 against `.bss'
/import/wrkdir10/bahls/netcdf/wrk/netcdf-3.6.3/f90/./netcdf_attributes.f90:137: relocation truncated to fit: R_X86_64_PC32 against `.bss'
/wrkdir10/bahls/netcdf/lib/libnetcdf.a(netcdf.o): In function `netcdf_nf90_get_att_one_twobyteint_':
/import/wrkdir10/bahls/netcdf/wrk/netcdf-3.6.3/f90/./netcdf_attributes.f90:157: relocation truncated to fit: R_X86_64_PC32 against `.bss'
/import/wrkdir10/bahls/netcdf/wrk/netcdf-3.6.3/f90/./netcdf_attributes.f90:157: relocation truncated to fit: R_X86_64_PC32 against `.bss'
/wrkdir10/bahls/netcdf/lib/libnetcdf.a(netcdf.o): In function `netcdf_nf90_put_att_one_fourbyteint_':
/import/wrkdir10/bahls/netcdf/wrk/netcdf-3.6.3/f90/./netcdf_attributes.f90:177: relocation truncated to fit: R_X86_64_PC32 against `.bss'
/import/wrkdir10/bahls/netcdf/wrk/netcdf-3.6.3/f90/./netcdf_attributes.f90:177: relocation truncated to fit: R_X86_64_PC32 against `.bss'
/wrkdir10/bahls/netcdf/lib/libnetcdf.a(netcdf.o): In function `netcdf_nf90_get_att_one_fourbyteint_':
/import/wrkdir10/bahls/netcdf/wrk/netcdf-3.6.3/f90/./netcdf_attributes.f90:201: additional relocation overflows omitted from the output

We have been able to compile some resolutions by adding "-mcmodel=medium", however this particular resolution isn't working. We even attempted to recompile netcdf with -fpic, but it didn't make a difference.

Should -mcmodel=medium be required for CESM? It doesn't look like any of the other Macro files uses the mcmodel=medium flag and there don't appear to be references to it on this message board.

Let us know if you have any idea why -mcmodel=medium is required.

Thanks,
Don

 

[jedwards]

As far as I know we do not support a machine called midnight. The -mcmodel was added by someone at your site. Have you tried building a simple program with your netcdf installation? For example:

program nftest
use netcdf
end program

 

[bahls@arsc_edu]

We added the "-mcmodel=medium" because we got error similar to the following:

mpif90 -o /wrkdir/uajchen1/test3/run/ccsm.exe ccsm_comp_mod.o ccsm_driver.o map_atmatm_mct.o map_atmice_mct.o map_atmlnd_mct.o map_atmocn_mct.o map_glcglc_mct.o map_iceice_mct.o map_iceocn_mct.o map_lndlnd_mct.o map_ocnocn_mct.o map_rofocn_mct.o map_rofrof_mct.o map_snoglc_mct.o map_snosno_mct.o mrg_x2a_mct.o mrg_x2g_mct.o mrg_x2i_mct.o mrg_x2l_mct.o mrg_x2o_mct.o mrg_x2s_mct.o seq_avdata_mod.o seq_diag_mct.o seq_domain_mct.o seq_flux_mct.o seq_frac_mct.o seq_hist_mod.o seq_rearr_mod.o seq_rest_mod.o -L/wrkdir/uajchen1/test3/lib -latm -llnd -lice -locn -lglc -L/wrkdir/uajchen1/test3/lib -lcsm_share -lmct -lmpeu -lpio -L/usr/local/pkg/netcdf/netcdf-3.6.3.pgi/lib -lnetcdf -Wl,--allow-multiple-definition
/import/usrlocal/pkg/pgi/pgi-11.2/linux86-64/11.2/lib/libpgf90.a(initpar.o): In function `__hpf_myprocnum':
initpar.c:(.text+0x2): relocation truncated to fit: R_X86_64_PC32 against symbol `__hpf_lcpu' defined in COMMON section in /import/usrlocal/pkg/pgi/pgi-11.2/linux86-64/11.2/lib/libpgf90.a(initpar.o)
/import/usrlocal/pkg/pgi/pgi-11.2/linux86-64/11.2/lib/libpgf90.a(initpar.o): In function `__hpf_ncpus':
initpar.c:(.text+0x12): relocation truncated to fit: R_X86_64_PC32 against symbol `__hpf_tcpus' defined in COMMON section in /import/usrlocal/pkg/pgi/pgi-11.2/linux86-64/11.2/lib/libpgf90.a(initpar.o)
/import/usrlocal/pkg/pgi/pgi-11.2/linux86-64/11.2/lib/libpgf90.a(initpar.o): In function `__hpf_getioproc':
initpar.c:(.text+0x22): relocation truncated to fit: R_X86_64_PC32 against symbol `__hpf_ioproc' defined in COMMON section in /import/usrlocal/pkg/pgi/pgi-11.2/linux86-64/11.2/lib/libpgf90.a(initpar.o)
...
initpar.c:(.text+0x673): additional relocation overflows omitted from the output
gmake: *** [/wrkdir/uajchen1/test3/run/ccsm.exe] Error 2

The netcdf build passes "make check" validation and links the simple fortran program.

It seems that something with the resolution is causing the static data to use more that 2GB in the .bss section. Does anyone know if
this sort of error occurs on other systems?

Don

 

[jedwards]

Hi Don,

I don't see anyplace else where we require that mcmodel flag - not sure why you would need it. Did you try building the netcdf test with that flag? It may be that the netcdf library needs to be rebuilt with the flag if it is to be linked with a cesm that uses that flag.

 

[bdobbins@gmail_com]

I feel slightly bad jumping in here without any firm knowledge or recent experience with the PGI compilers, but I do recall once having to build NetCDF with the '--with-pic' option to configure and the associated compiler flags (-fpic, I think, for pgf90) for CFLAGS/FFLAGS/etc.

Hope that helps!

- Brian

 

[bahls@arsc_edu]

Thanks for the pointers. As it turns out I was attempting to build netcdf static libraries only without shared libraries so the -fpic we were using was meaningless. It appears that the versions of netcdf-3 and netcdf-4 end up with some routines in the static netcdf library so the error occurs still even if -fpic is used with dynamic libraries.

We're switching to a different resolution that doesn't require -mcmodel=medium.

 

[bahls@arsc_edu]

It looks like this issue was resolved with version 1_0_4 of CESM. There were no linking errors with the newer version.
We used the generic_linux_pgi config files.