CESM 1.2.2.1 cesm_setup module issue on Cheyenne

[swang54@uh_edu]

Hi Jedwards,
Thank your for keep updating the debugging progress! Yes. I ran a B case (BRCP45C5CN). I will try to help diagnose the issue but probably not as efficient as you do. If I was able to find something, i'll update it here.

Many thanks!
Sally

 

[swang54@uh_edu]

Hi Jedwards,
Thank your for keep updating the debugging progress! Yes. I ran a B case (BRCP45C5CN). I will try to help diagnose the issue but probably not as efficient as you do. If I was able to find something, i'll update it here.

Many thanks!
Sally

 

[sungduk@uci_edu]

Thank you, jedwards and Sally!I checked out the newer pio, but still I can't run CESM. I tested both pio 4.4.1.1 and 4.6.1, also B and F compsets (B1850C5 and F1850C5). However, any compbination of these returned a same error. (jedwards: would you share your env_mach_specific.cheyenne? I may have set something wrong cus I could not run a F-compset.)- Sungduk

 

[sungduk@uci_edu]

Thank you, jedwards and Sally!I checked out the newer pio, but still I can't run CESM. I tested both pio 4.4.1.1 and 4.6.1, also B and F compsets (B1850C5 and F1850C5). However, any compbination of these returned a same error. (jedwards: would you share your env_mach_specific.cheyenne? I may have set something wrong cus I could not run a F-compset.)- Sungduk

 

[jedwards]

My F-case is in /glade/scratch/jedwards/cesm1_2_2_1.F

 

[jedwards]

My F-case is in /glade/scratch/jedwards/cesm1_2_2_1.F

 

[zke@uwyo_edu]

Dear Jedwards:By copy your env_mach_specific settings and following your pio advices, I still can not build the case like in your cesm1_2_2_1.Fthe error is
zke3@cheyenne4:$ ./cesm_setup
Creating Macros file for cheyenne
/gpfs/u/home/zke3/cesm1_2_2_1/scripts/ccsm_utils/Machines/config_compilers.xml intel cheyenne
Creating batch script cesm1_2_2_1.F.run
PHASE is set_batch
mppsize is 128 4
PHASE is set_exe
Locking file env_mach_pes.xml
Creating user_nl_xxx files for components and cpl
Running preview_namelist script
 infile is /glade/u/home/zke3/cases/cesm1_2_2_1.F/Buildconf/cplconf/cesm_namelist
** build-namelist - CCSM inputdata root is not a directory: "/inputdata" **
ERROR: cpl.buildnml.csh failed
ERROR: /gpfs/u/home/zke3/cases/cesm1_2_2_1.F/preview_namelists failed: 25344

---------------------------
Thanks!

Ziming Ke

 

[zke@uwyo_edu]

Dear Jedwards:By copy your env_mach_specific settings and following your pio advices, I still can not build the case like in your cesm1_2_2_1.Fthe error is
zke3@cheyenne4:$ ./cesm_setup
Creating Macros file for cheyenne
/gpfs/u/home/zke3/cesm1_2_2_1/scripts/ccsm_utils/Machines/config_compilers.xml intel cheyenne
Creating batch script cesm1_2_2_1.F.run
PHASE is set_batch
mppsize is 128 4
PHASE is set_exe
Locking file env_mach_pes.xml
Creating user_nl_xxx files for components and cpl
Running preview_namelist script
 infile is /glade/u/home/zke3/cases/cesm1_2_2_1.F/Buildconf/cplconf/cesm_namelist
** build-namelist - CCSM inputdata root is not a directory: "/inputdata" **
ERROR: cpl.buildnml.csh failed
ERROR: /gpfs/u/home/zke3/cases/cesm1_2_2_1.F/preview_namelists failed: 25344

---------------------------
Thanks!

Ziming Ke

 

[jedwards]

It looks like you messed something up in your login window.   Logout and back in and then try again with a new case. 

 

[jedwards]

It looks like you messed something up in your login window.   Logout and back in and then try again with a new case. 

 

[zke@uwyo_edu]

Hi, Jedwards:I redo the process. You are right I may messed somthing in login. Now it can to to the final build step 'CESM', but same error as reported yesterday.Here is the error----------------------------Mon Mar 11 11:24:22 MDT 2019 /glade/scratch/zke3/cesm1_2_2_1.F/bld/rof.bldlog.190311-111849
Mon Mar 11 11:24:31 MDT 2019 /glade/scratch/zke3/cesm1_2_2_1.F/bld/cesm.bldlog.190311-111849
ERROR: cesm.buildexe.csh failed, see /glade/scratch/zke3/cesm1_2_2_1.F/bld/cesm.bldlog.190311-111849
ERROR: cat /glade/scratch/zke3/cesm1_2_2_1.F/bld/cesm.bldlog.190311-111849-------------------------ccsm_comp_mod.F90:(.text+0x26): relocation truncated to fit: R_X86_64_32 against symbol `seq_avdata_mod_mp_infodata_' defined in COMMON section in seq_avdata_mod.o
ccsm_comp_mod.F90:(.text+0x467): relocation truncated to fit: R_X86_64_32S against symbol `ccsm_comp_mod_mp_begstep_' defined in COMMON section in ccsm_comp_mod.o
ccsm_comp_mod.F90:(.text+0x495): relocation truncated to fit: R_X86_64_32S against symbol `ccsm_comp_mod_mp_dtime_' defined in COMMON section in ccsm_comp_mod.o
ccsm_comp_mod.F90:(.text+0x4c8): relocation truncated to fit: R_X86_64_PC32 against symbol `ccsm_comp_mod_mp_dtime_' defined in COMMON section in ccsm_comp_mod.o
ccsm_comp_mod.F90:(.text+0x4e7): relocation truncated to fit: R_X86_64_PC32 against symbol `ccsm_comp_mod_mp_ncpl_' defined in COMMON section in ccsm_comp_mod.o
ccsm_comp_mod.F90:(.text+0x4ee): relocation truncated to fit: R_X86_64_PC32 against symbol `ccsm_comp_mod_mp_cktime_acc_' defined in COMMON section in ccsm_comp_mod.o
ccsm_comp_mod.F90:(.text+0x4f5): relocation truncated to fit: R_X86_64_PC32 against symbol `ccsm_comp_mod_mp_cktime_acc_' defined in COMMON section in ccsm_comp_mod.o
ccsm_comp_mod.F90:(.text+0x4fc): relocation truncated to fit: R_X86_64_PC32 against symbol `ccsm_comp_mod_mp_cktime_acc_' defined in COMMON section in ccsm_comp_mod.o
ccsm_comp_mod.F90:(.text+0x503): relocation truncated to fit: R_X86_64_PC32 against symbol `ccsm_comp_mod_mp_cktime_acc_' defined in COMMON section in ccsm_comp_mod.o
ccsm_comp_mod.F90:(.text+0x50a): relocation truncated to fit: R_X86_64_PC32 against symbol `ccsm_comp_mod_mp_cktime_acc_' defined in COMMON section in ccsm_comp_mod.o
ccsm_comp_mod.F90:(.text+0x512): additional relocation overflows omitted from the output
/glade/u/home/zke3/cases/cesm1_2_2_1.F/Tools/Makefile:629: recipe for target '/glade/scratch/zke3/cesm1_2_2_1.F/bld/cesm.exe' failed
gmake: *** [/glade/scratch/zke3/cesm1_2_2_1.F/bld/cesm.exe] Error 1-----------------Ziming Ke 

 

[zke@uwyo_edu]

Hi, Jedwards:I redo the process. You are right I may messed somthing in login. Now it can to to the final build step 'CESM', but same error as reported yesterday.Here is the error----------------------------Mon Mar 11 11:24:22 MDT 2019 /glade/scratch/zke3/cesm1_2_2_1.F/bld/rof.bldlog.190311-111849
Mon Mar 11 11:24:31 MDT 2019 /glade/scratch/zke3/cesm1_2_2_1.F/bld/cesm.bldlog.190311-111849
ERROR: cesm.buildexe.csh failed, see /glade/scratch/zke3/cesm1_2_2_1.F/bld/cesm.bldlog.190311-111849
ERROR: cat /glade/scratch/zke3/cesm1_2_2_1.F/bld/cesm.bldlog.190311-111849-------------------------ccsm_comp_mod.F90:(.text+0x26): relocation truncated to fit: R_X86_64_32 against symbol `seq_avdata_mod_mp_infodata_' defined in COMMON section in seq_avdata_mod.o
ccsm_comp_mod.F90:(.text+0x467): relocation truncated to fit: R_X86_64_32S against symbol `ccsm_comp_mod_mp_begstep_' defined in COMMON section in ccsm_comp_mod.o
ccsm_comp_mod.F90:(.text+0x495): relocation truncated to fit: R_X86_64_32S against symbol `ccsm_comp_mod_mp_dtime_' defined in COMMON section in ccsm_comp_mod.o
ccsm_comp_mod.F90:(.text+0x4c8): relocation truncated to fit: R_X86_64_PC32 against symbol `ccsm_comp_mod_mp_dtime_' defined in COMMON section in ccsm_comp_mod.o
ccsm_comp_mod.F90:(.text+0x4e7): relocation truncated to fit: R_X86_64_PC32 against symbol `ccsm_comp_mod_mp_ncpl_' defined in COMMON section in ccsm_comp_mod.o
ccsm_comp_mod.F90:(.text+0x4ee): relocation truncated to fit: R_X86_64_PC32 against symbol `ccsm_comp_mod_mp_cktime_acc_' defined in COMMON section in ccsm_comp_mod.o
ccsm_comp_mod.F90:(.text+0x4f5): relocation truncated to fit: R_X86_64_PC32 against symbol `ccsm_comp_mod_mp_cktime_acc_' defined in COMMON section in ccsm_comp_mod.o
ccsm_comp_mod.F90:(.text+0x4fc): relocation truncated to fit: R_X86_64_PC32 against symbol `ccsm_comp_mod_mp_cktime_acc_' defined in COMMON section in ccsm_comp_mod.o
ccsm_comp_mod.F90:(.text+0x503): relocation truncated to fit: R_X86_64_PC32 against symbol `ccsm_comp_mod_mp_cktime_acc_' defined in COMMON section in ccsm_comp_mod.o
ccsm_comp_mod.F90:(.text+0x50a): relocation truncated to fit: R_X86_64_PC32 against symbol `ccsm_comp_mod_mp_cktime_acc_' defined in COMMON section in ccsm_comp_mod.o
ccsm_comp_mod.F90:(.text+0x512): additional relocation overflows omitted from the output
/glade/u/home/zke3/cases/cesm1_2_2_1.F/Tools/Makefile:629: recipe for target '/glade/scratch/zke3/cesm1_2_2_1.F/bld/cesm.exe' failed
gmake: *** [/glade/scratch/zke3/cesm1_2_2_1.F/bld/cesm.exe] Error 1-----------------Ziming Ke 

 

[kangwanying1992@gmail_com]

Hi,I have met the same problem, and it seems that the model tends to allocate a too large array (greater than the 2GB limit). I searched online and found one solution which is to add "-mcmodel=medium" to the compile flag. You could add this flag somewhere to Machines/config_compilers.xml under intel machine. After this fix, the compile can go through ok, but the executable is not runable... Any comment and advice on that?Best,Wanying

 

[kangwanying1992@gmail_com]

Hi,I have met the same problem, and it seems that the model tends to allocate a too large array (greater than the 2GB limit). I searched online and found one solution which is to add "-mcmodel=medium" to the compile flag. You could add this flag somewhere to Machines/config_compilers.xml under intel machine. After this fix, the compile can go through ok, but the executable is not runable... Any comment and advice on that?Best,Wanying

 

[jedwards]

I posted this earlier and gave the wrong revision number - corrected instructions are here:So the next suggestion is a little more complicated.  In your source tree:cd models/utilsmv pio pio.oldsvn co https://github.com/NCAR/ParallelIO.git/tags/pio1_8_14/pio then clean and rebuild your case.  You should be able to use either netcdf version with this change, I recommend netcdf-mpi/4.6.1

 

[jedwards]

I posted this earlier and gave the wrong revision number - corrected instructions are here:So the next suggestion is a little more complicated.  In your source tree:cd models/utilsmv pio pio.oldsvn co https://github.com/NCAR/ParallelIO.git/tags/pio1_8_14/pio then clean and rebuild your case.  You should be able to use either netcdf version with this change, I recommend netcdf-mpi/4.6.1

 

[christopher_maloney@colorado_edu]

Hi Jedwards,I appreciate all of the work that you have put into helping fix this issue. I have updated my env_mach_specific.cheyenne to match yours, downloaded the new 'pio' library per your instructions, and performed a clean build; however, my case is still failing to build. The error message has changed from the ones prevoiously mentioned though. Any suggestions for fixes would be greatly appreciated. I have put the error message below:shr_pio_mod.F90:(.text+0x21b6): undefined reference to `piodarray_mp_pio_set_buffer_size_limit_'/gpfs/u/home/cmaloney/cesm1_2_2_1/scripts/testing_new_mpt_v2/Tools/Makefile:629: recipe for target '/glade/scratch/cmaloney/testing_new_mpt_v2/bld/cesm.exe' failed gmake: *** [/glade/scratch/cmaloney/testing_new_mpt_v2/bld/cesm.exe] Error 1 Thanks,Chris Maloney 

 

[christopher_maloney@colorado_edu]

Hi Jedwards,I appreciate all of the work that you have put into helping fix this issue. I have updated my env_mach_specific.cheyenne to match yours, downloaded the new 'pio' library per your instructions, and performed a clean build; however, my case is still failing to build. The error message has changed from the ones prevoiously mentioned though. Any suggestions for fixes would be greatly appreciated. I have put the error message below:shr_pio_mod.F90:(.text+0x21b6): undefined reference to `piodarray_mp_pio_set_buffer_size_limit_'/gpfs/u/home/cmaloney/cesm1_2_2_1/scripts/testing_new_mpt_v2/Tools/Makefile:629: recipe for target '/glade/scratch/cmaloney/testing_new_mpt_v2/bld/cesm.exe' failed gmake: *** [/glade/scratch/cmaloney/testing_new_mpt_v2/bld/cesm.exe] Error 1 Thanks,Chris Maloney 

 

[jedwards]

I don't see why you would get that error - everything looks correct to me.   Try removing the bld directoryrm -fr /glade/scratch/cmaloney/testing_new_mpt_v2/bldand building again.

 

[jedwards]

I don't see why you would get that error - everything looks correct to me.   Try removing the bld directoryrm -fr /glade/scratch/cmaloney/testing_new_mpt_v2/bldand building again.