Problems with running CCSM4.0 with prescribed varying SST forcing

[meir02ster@gmail_com]

Hi, all,

I got a problem to run CCSM4.02 with prescribed varying SST forcing. Here is the endding part of the log file:

"(seq_mct_drv) : Model initialization complete
(shr_sys_abort) ERROR: ERROR: shr_cal gregorian requires USE_ESMF_LIB
(shr_sys_abort) WARNING: calling shr_mpi_abort() and stopping
Segmentation fault"

I was using pgi 10.4, netcdf4, and mpich2-1.0.6 and running the model at 1.9*2.5 with fv core. Btw: there is no problem while running the model with prescribed climatological SST forcing.

I really appreciate if someone here can help me diagnose the problem.
Thank you very much!

Rui

 

[dbailey]

Hi,

It appears you are trying to use a Gregorian calendar with your time-varying forcing. This does require recompiling the code with the ESMF libraries as suggested by the message. Can you do this with a 365-day calendar?

Dave

 

[meir02ster@gmail_com]

Hi, Dave,

Thank you for your reply. So now the model accomodates greorian calendar?
I thought it was still using 365-day calendar as before. Could you give me more clues on the solutions? Is there something I need to modify?

Rui

 

[eaton]

The support for Gregorian calendar is limited, and I don't think it is the default for any compset. We need more information about how you've set up the run to be able to help. After running create_newcase the env_run.xml file in the case directory contains the default setting for the calendar. You should see:



Unless you change this, the model should be configured to run with a 365 day calendar.

 

[meir02ster@gmail_com]

Hi, eaton,

Thanks for the reply. Now I realize I even have a problem to run the model with climatological SST forcing. When I thought I was fine to run the model with climatological SST forcing, I only tried that for one day and it was successful. Now I tried to run it for 10 years (since I still did not figure out how to run it with varying SST forcing, I decided to run it with prescribed SST forcing for 10 years to take a first look at how CAM4 performs in simulating precipitation compared with CAM3). But the model was just hanging there without output, error message and the writing of log file.

Here is the endding part of log file and my jobscript file:
"
180000 19400104
Total Mass= 985.2695568093355 (mb), Dry Mass=
982.8799780309273 (mb)
Total Precipitable Water = 24.36814041748967 (kg/m**2)
PS max = 1047.911115274770 min = 549.6662992235437
U max = 92.26953350202251 min = -68.57900242803366
V max = 59.68747385227867 min = -57.92985126665950
T max = 310.2187808465124 min = 183.6989801679336
W (mb/day) max = 772.7241957332034 min = -902.8095749590150
Average Height (geopotential units) = 1231.199359421252
PRECC max = 35.71931887821241 min = 0.000000000000000
PRECL max = 86.89424617961171 min = 0.000000000000000
Total precp= 2.798526481205589 CON= 1.766949527594800 LS=
1.031576953610790

nstep, te 180 0.33242659855718966E+10 0.33242686315894747E+10 0.14663320151694743E-03 0.98526957883182680E+05
nstep, te 181 0.33242662625432196E+10 0.33242688392802362E+10 0.14279389336135437E-03 0.98526969940127776E+05
clm2: completed timestep 181
clm2: completed timestep 182
nstep, te 182 0.33242608243161049E+10 0.33242644730018263E+10 0.20219759122376080E-03 0.98526974057447587E+05
nstep, te 183 0.33242566303797498E+10 0.33242602201988907E+10 0.19893538545172934E-03 0.98526985277451909E+05
clm2: completed timestep 183
clm2: completed timestep 184
nstep, te 184 0.33242431319661398E+10 0.33242483982877774E+10 0.29184137234658422E-03 0.98526983432200112E+05
nstep, te 185 0.33242317054321771E+10 0.33242368880665698E+10 0.28720368183573727E-03 0.98526994448362268E+05
clm2: completed timestep 185
clm2: completed timestep 186
nstep, te 186 0.33242096660045004E+10 0.33242165715782337E+10 0.38268306648760604E-03 0.98526983518883324E+05"

"#! /bin/csh -f
#
#=======================================================================
#
# run-pc.csh
#
# Generic batch submission script for PC-linux using PBS.
#
#-----------------------------------------------------------------------
# Batch options for machine with PBS batch system. (anchorage)
# Usage for Lahey compiler (default):
# qsub run-pc.csh
# Usage for pgf90 compiler with pgcc:
# env OPT_BLD=pgf90-pgcc qsub run-pc.csh
#-----------------------------------------------------------------------
#
# Name of the queue (CHANGE THIS if needed)
#PBS -q long
# Maximum number of processes (CHANGE THIS if needed)
#PBS -l nodes=4
# output file base name
#PBS -N run-pc
# Put standard error and standard out in same file
#PBS -j oe
# Export all Environment variables
#PBS -V
# End of options
#=======================================================================


setenv INC_MPI /usr/local/mpi/include
setenv LIB_MPI /usr/local/mpi/lib
set mpirun = /usr/local/mpi/bin/mpirun
echo "${0}: Set mpirun to $mpirun";
setenv INC_NETCDF /usr/local/netcdf4/include
setenv LIB_NETCDF /usr/local/netcdf4/lib

## set this equal to #nodes X #ppn
set procs = 8
## Do our best to get sufficient stack memory
limit stacksize unlimited

## ROOT OF CAM DISTRIBUTION - probably needs to be customized.
## Contains the source code for the CAM distribution.
## (the root directory contains the subdirectory "models")
set camroot = /home2/meir02/ccsm4

## ROOT OF CAM DATA DISTRIBUTION - needs to be customized unless running at NCAR.
## Contains the initial and boundary data for the CAM distribution.
## (the root directory contains the subdirectories "atm" and "lnd")
setenv CSMDATA /home2/meir02/ccsm4/inputdata

## Default namelist settings:
## $case is the case identifier for this run. It will be placed in the namelist.
## $runtype is the run type: startup, continue, or branch.
## $stop_n is the number of days to integrate (units depends on stop_option)
set case = climatol

## $wrkdir is a working directory where the model will be built and run.
## $blddir is the directory where model will be compiled.
## $rundir is the directory where the model will be run.
## $cfgdir is the directory containing the CAM configuration scripts.
set wrkdir = /home2/meir02/ccsm4/ctl
set blddir = $wrkdir/$case/bld
set rundir = $wrkdir/$case
set cfgdir = $camroot/models/atm/cam/bld

## set data and time for the run
setenv LID "`date +%y%m%d-%H%M%S`"

## Ensure that run and build directories exist
mkdir -p $rundir || echo "cannot create $rundir" && exit 1
mkdir -p $blddir || echo "cannot create $blddir" && exit 1


## Create the namelist
cd $blddir
cat >! namelistfile

 

[eaton]

If I use your configure command, but use default namelist settings without overriding anything except the start date and run length variables, I can run successfully for 10 days (your run didn't make it through 5 days). When I compare the default namelist settings with the overrides you requested a few things stick out as possible problems. The following are defaults:

atm_cpl_dt = 1800
finidat = '/fs/cgd/csm/inputdata/lnd/clm2/initdata/clmi.BCN.2000-01-01_1.9x2.5_gx1v6_simyr2000_c100309.nc'
fpftcon = '/fs/cgd/csm/inputdata/lnd/clm2/pftdata/pft-physiology.c100226'

You have reset atm_cpl_dt to 3600. I think that is most likely the problem. Both cam standalone and the fully coupled ccsm set atm_cpl_dt equal to the cam timestep. Do anything else and you're in uncharted territory.

I think it's also possible that the differences in the CLM files are causing a problem, although usually when CLM datasets are the problem the model just crashes at startup. So I think it's much more likely that atm_cpl_dt is the problem.

 

[meir02ster@gmail_com]

Hi, eaton,

Thanks for even testing my jobscript and trying to solve the problem with so many details. I really appreciate your help. I followed your advice, and tried the default setting but the run still crapped at the fifth day approximately.

Then I tried to run the jobscript with the same default setting on our another machine (Redhat) with pgi 7.0 instead of pgi 10.4 on the previous machine (Ubuntu). Now it can go through 20 days and it's still running forward. I believe it will carry on successfully. So now things get confusing to me.

The computation capability is so different between those two machines: the ubuntu has 8 cpus, but the redhat only has 4cpus. So I really want to fix it on Ubuntu. Do you have any thoughts on this issue?

Cheers,
Rui

 

[meir02ster@gmail_com]

I tried PGI7.0 on Ubuntu and it still failed. So now with the same setting and same PGI, NETCDF, MPI, the model can run on redhat but failed on Ubuntu.

Is it possible the Ubuntu system needs to be updated ?

 

[eaton]

Hi Rui,

This does sound like a system problem. At NCAR we are always running on bleeding edge HPC systems and sometimes it feels like we spend more time tracking down system problems than doing code development. So I do feel your pain, but there's not alot I can do to help.

That said, if you want to try tracking down the problem on your Ubuntu system, the most common problems seem to be MPI related. So one approach would be to try a lower resolution configuration and do a serial run. Add the following arguments to your configure command: "-hgrid 10x15 -nospmd -nosmp". That will eliminate the MPI library as a possible problem. If that run is successful, then you'll need to resolve the MPI problems. I can't offer any specific guidance there, but sometimes trying different distribtutions can be successful. If the serial run is not successful, hopefully you'll get more information from the run to guide your next steps.

Another possibly more direct route is to stick with your current MPI run, but turn on the debug options, i.e., add the -debug argument to the configure command. With any luck that will trigger a core dump and a traceback.

Brian

 

[meir02ster@gmail_com]

Hi, Brian,

Thanks for the advice. I will keep on trying to solve this issue. I was also thinking to run it on our SGI machine. But we also need to upgrade the fortran compiler, netcd and mpi.

Could you tell me the successful version of fortran compiler (both PGI fortran and intel fortran), netcdf and mpi, you have been testing for CCSM4?

Thank you very much.

Rui

 

[eaton]

On our local linux cluster we're currently using pgi-7.2.5. We test with pgi-9.3 on cray-xt4/5 platforms and are about to upgrade to than on our local cluster.

The netcdf lib should be version 3.6 or later.

We use mpich-1.2.7.p1 on our local cluster.

The intel compiler is version 11.0.074.

I should mention that this is a small cluster and we only test CAM standalone at low resolutions on it, not a fully coupled CCSM configuration. It's mostly useful for our regression testing.

 

[evanskj]

Dear Rui-

I have seen that error. The error thrown is different in CCSM4 and less obvious than for earlier CCSM versions. The issue is with the ocean SST bc dataset. To whatever dataset you are using, perform the following command using nco:

ncatted -a calendar,time,c,c,365_day sst_file.nc

And it should now be accepted properly.

Kate

 

[meir02ster@gmail_com]

Hi, Brian,

I still have problem to run CAM4 with prescribed varying SST forcing. I think I did not change the value of calendar and it should be NO_LEAP. So how to diagnose this problem?

Thanks,
Rui

The support for Gregorian calendar is limited, and I don't think it is the default for any compset. We need more information about how you've set up the run to be able to help. After running create_newcase the env_run.xml file in the case directory contains the default setting for the calendar. You should see:



Unless you change this, the model should be configured to run with a 365 day calendar.

 

[meir02ster@gmail_com]

I missed Kate's post and I went back to use nco to process the sst forcing file. And that gregorian calendar error message is really gone. But it comes with a new error message as follows. I tracked this message and it comes from shr_stream_mod.F90. I guess it was complaining about date over_record but it should not since the sst forcing record covers 1949-2001 and in my jobscript I was prescribing to run it from 19500101 for one year. Maybe I did not understand that message.

Thank you for any help


(seq_frac_check) [atm init] ifrac min/max = 0.00000000000000000 0.00000000000000000
(seq_frac_check) [atm init] afrac min/max = 1.00000000000000000 1.00000000000000000 (seq_frac_check) [atm init] lfrin min/max = 0.00000000000000000 1.00000000000000000

(seq_frac_check) [atm init] sum min/max = 1.00000000000000000 1.00000000000000000 (seq_frac_check) [atm init] lfrac min/max = 0.00000000000000000 1.00000000000000000

(seq_frac_check) [atm init] sum ncnt/maxerr = 0 0.00000000000000000
(seq_frac_check) [atm init] ofrac min/max = 0.00000000000000000 1.00000000000000000
(seq_frac_check) [atm init] ifrac min/max = 0.00000000000000000 0.00000000000000000
(seq_frac_check) [atm init] lfrin min/max = 0.00000000000000000 1.00000000000000000
(seq_frac_check) [atm init] sum min/max = 1.00000000000000000 1.00000000000000000
(seq_frac_check) [atm init] sum ncnt/maxerr = 0 0.00000000000000000
(seq_frac_check) [lnd init] afrac min/max = 1.00000000000000000 1.00000000000000000
(seq_frac_check) [lnd init] lfrac min/max = 0.208518197640250590E-02 1.00000000000000000
(seq_frac_check) [lnd init] lfrin min/max = 0.208518197640250590E-02 1.00000000000000000
(seq_frac_check) [ice init] afrac min/max = 1.00000000000000000 1.00000000000000000
(seq_frac_check) [ice init] ofrac min/max = 0.00000000000000000 1.00000000000000000
(seq_frac_check) [ice init] ifrac min/max = 0.00000000000000000 0.00000000000000000
(seq_frac_check) [ocn init] afrac min/max = 1.00000000000000000 1.00000000000000000
(seq_frac_check) [ocn init] ofrac min/max = 0.00000000000000000 1.00000000000000000
(seq_frac_check) [ocn init] ifrac min/max = 0.00000000000000000 0.00000000000000000
(seq_frac_check) [atm init] afrac min/max = 1.00000000000000000 1.00000000000000000
(seq_frac_check) [atm init] lfrac min/max = 0.00000000000000000 1.00000000000000000
(seq_frac_check) [atm init] ofrac min/max = 0.00000000000000000 1.00000000000000000
(seq_frac_check) [atm init] ifrac min/max = 0.00000000000000000 0.00000000000000000
(seq_frac_check) [atm init] lfrin min/max = 0.00000000000000000 1.00000000000000000
(seq_frac_check) [atm init] sum min/max = 1.00000000000000000 1.00000000000000000
(seq_frac_check) [atm init] sum ncnt/maxerr = 0 0.00000000000000000
(seq_mct_drv) : Setting fractions
(seq_mct_drv) : Initializing atm/ocn flux component
(seq_flux_atmocn_mct) computing only ocn albedos
(seq_mct_drv) : Calling map_lnd2atm_mct
(seq_mct_drv) : Calling map_ocn2atm_mct for mapping o2x_ox to o2x_ax
(seq_mct_drv) : Calling map_ocn2atm_mct for mapping xao_ox to xao_ax
(seq_mct_drv) : Calling map_ice2atm_mct for mapping i2x_ix to i2x_ax
(seq_mct_drv) : Calling mrg_x2a_run_mct
(seq_mct_drv) : Calling atm_init_mct
FV subcycling - n2 nsplit = 1 4
Divergence damping: use 2nd order damping
nstep, te 0 0.33264096717818675E+10 0.33264096717818675E+10 0.00000000000000000E+00 0.98525963359206798E+05

(seq_mct_drv) : Model initialization complete

(shr_stream_findBounds) ERROR: LVD not found, all data is after yearLast
(shr_sys_abort) ERROR: (shr_stream_findBounds) ERROR: LVD not found, all data is after yearLast
(shr_sys_abort) WARNING: calling shr_mpi_abort() and stopping
rank 0 in job 11 water_32985 caused collective abort of all ranks
exit status of rank 0: killed by signal 9

 

[meir02ster@gmail_com]

It seems I have fixed the issue but I still want to get a confirmation here. As I mentioned that I run the model with prescribed varying (SST+ice) forcing, the forcing data covers 1949-2001, and I set the namelists as follow. So the simulation period is from 19500101 to 19500131. When I tracked down the error message, it brings up two variables yearFirst and yearLast and those two variables are both 0 out of the range 1949~2001. Then I found out those two variables are read from two namelists: stream_year_first and stream_year_last (they are both zero in default). So I set both of them to 1950 and solved the problem. I do not know why it does not use the namelist start_ymd and stop_n as references to retrieve yearFirst and yearLast.

So is there any other namelists that I need to modify when I use prescribed varying SST+ice forcing? I hope the model is running what I expect with the current setting. So I just want to get a confirmation from anyone who knows. Thank you very much.

Rui

&camexp
absems_data = '/home/meir02/ccsm4_0/inputdata/atm/cam/rad/abs_ems_factors_fastvx.c030508.nc'
bnd_topo = '/home/meir02/ccsm4_0/inputdata/atm/cam/topo/USGS-gtopo30_1.9x2.5_remap_c050602.nc'
ncdata = '/home/meir02/ccsm4_0/inputdata/atm/cam/inic/fv/cami_0000-01-01_1.9x2.5_L26_c070408.nc'
bndtvs = '/home/meir02/ccsm4_0/inputdata/atm/cam/sst/sst_HadOIBl_bc_1.9x2.5_1949_2001_c040810.nc'
focndomain = '/home/meir02/ccsm4_0/inputdata/atm/cam/ocnfrac/domain.camocn.1.9x2.5_gx1v6_090403.nc'
sstcyc = .false.
/
&ccsm_inparm
case_name = 'ctl'
start_type = 'startup'
/
&timemgr_inparm
restart_option = 'monthly'
stop_option = 'ndays'
stop_n = 31
start_ymd = 19500101
/
&clm_inparm
faerdep = '/home/meir02/ccsm4_0/inputdata/lnd/clm2/snicardata/aerosoldep_monthly_2000_mean_1.9x2.5_c090421.nc'
fatmgrid = '/home/meir02/ccsm4_0/inputdata/lnd/clm2/griddata/griddata_1.9x2.5_060404.nc'
fatmlndfrc = '/home/meir02/ccsm4_0/inputdata/lnd/clm2/griddata/fracdata_1.9x2.5_gx1v6_c090206.nc'
fsnowaging = '/home/meir02/ccsm4_0/inputdata/lnd/clm2/snicardata/snicar_drdt_bst_fit_60_c070416.nc'
fsnowoptics = '/home/meir02/ccsm4_0/inputdata/lnd/clm2/snicardata/snicar_optics_5bnd_c090915.nc'
fsurdat = '/home/meir02/ccsm4_0/inputdata/lnd/clm2/surfdata/surfdata_1.9x2.5_simyr2000_c091005.nc'
finidat = '/home/meir02/ccsm4_0/inputdata/lnd/clm2/initdata/clmi.BCN.2000-01-01_1.9x2.5_gx1v6_simyr2000_c100309.nc'
fpftcon = '/home/meir02/ccsm4_0/inputdata/lnd/clm2/pftdata/pft-physiology.c100226'
/

 

[meir02ster@gmail_com]

Hi, all,

I met a new problem, I created the netcdf file of SST-varying forcing data using Fortran.
And the model complains about netcdf when reading the forcing file. So I guess the problem is caused by the file. But it works fine with CCSM3. Is there any easy way to creat the netcdf file of SST forcing?

Thanks,

Rui

 

[eaton]

You have found the necessary namelist variables to set the year boundaries of the multi-year SST dataset, i.e., stream_year_first and stream_year_last. These values should be set to the first and last years respectively in the SST dataset. For the SST dataset you used the correct settings would be stream_year_first=1949 and stream_year_last=2001.

You have specified alot of values to build-namelist and I'd like to point out that this really isn't necessary. Suppose that $cfgdir is the directory in the source tree than contains the configure command, $blddir is the directory where the model will be built, $rundir is the directory where the model will run, and $CSMDATA is the root of the inputdata directory. Then the following commands are all that is necessary to produce the settings that you are explicitly setting in the namelist:

% cd $blddir
% $cfgdir/configure -dyn fv -hgrid 1.9x2.5
% cd $rundir
% $cfgdir/build-namelist -case ctl -config $blddir/config_cache.xml
-ignore_ic_year
-namelist "&exp start_type='startup' start_ymd=19500101
stop_option='ndays' stop_n=31 restart_option='monthly'
bndtvs='$CSMDATA/atm/cam/sst/sst_HadOIBl_bc_1.9x2.5_1949_2001_c040810.nc'
sstcyc=.false. stream_year_first=1949 stream_year_last=2001 /"

Note the following things about the preceding build-namelist command:
1) only non-default values need to be specified.
2) the namelist group (&exp) is not an actual group read by the code. The group name is only required because the parser of the -namelist argument requires valid namelist syntax. But the group name is ignored by build-namelist because it is using a namelist definition file to validate the user input. That definition file contains the correct namelist group for each variable. So the user doesn't need to know the correct group names. The build-namelist utility automatically takes care of putting variables in the correct group, and putting the namelist groups into the correct output files.

I'll also mention that there are more up to date AMIP SST datasets available from the svn data repository than the one you are using. For example, for 1.9x2.5 the latest dataset is atm/cam/sst/sst_HadOIBl_bc_1.9x2.5_1850_2008_c100127.nc

Finally, if you wish to produce your own SST datasets, I'd recommend that you start from one that is in the svn data repository that you know works. Use it as a template file so that you automatically get all the variable names and other metadata correct. Just use your fortran code to overwrite the values in the file to meet your requirements.

 

[sirajkhan78@gmail_com]

Hi Eaton,

I did some experiments using CAM4 for climatolgy and control runs. For control run I used prescribed SST and follow the method as you mention here in this post. I copied the part of my script that I am using. As I am doing some comparison between CAM4 and CAM3 I need to run this script for CAM3 as well. For CAM4 every thing is ok but when I change -phy cam3 in this script keeping every thing else same then i get get errors in running model. Model just crash after some time (copied bellows as well). DO i need to to change some thing else as well ?
---------My Script-------------
$cfgdir/configure -fc ifort -test
-dyn eul
-hgrid 64x128
-spmd
-nosmp
-linker mpif90
-ntasks $ntask
|| echo "configure failed" && exit 1

echo "building CAM in $blddir ..."
rm -f Depends
gmake >&! MAKE.out || echo "CAM build failed: see $blddir/MAKE.out" && exit 1
endif

cd $blddir || echo " cd $blddir failed " && exit
touch namelistout.txt
echo " building the namelist..."
$cfgdir/build-namelist -test -case $case -config $blddir/config_cache.xml
-ignore_ic_year
-namelist "&exp start_type='startup' start_ymd=19780101
stop_option='nyears' stop_n=22 restart_option='monthly' /"
>&! namelistout.txt

-------------Error-----------------
/hpc/home/sislam/ccsm4_working_copy/inputdata/atm/cam/ocnfrac/domain.camocn.64x128_USGS_070807.nc
(GETFIL): attempting to find local file
sst_HadOIBl_bc_64x128_1850_2008_c100128.nc
(GETFIL): using
/hpc/home/sislam/ccsm4_working_copy/inputdata/atm/cam/sst/sst_HadOIBl_bc_64x128_1850_2008_c100128.nc
SSTINI: Read sst data for dates 19771216 43200 and 19780116
43200
(seq_mct_drv) : Initialize ice component
(shr_sys_abort) ERROR: ice_comp.F90
(shr_sys_abort) WARNING: calling shr_mpi_abort() and stopping
p3_22211: p4_error: interrupt SIGSEGV: 11
rm_l_3_22290: (20.859375) net_send: could not write to fd=5, errno = 32
rm_l_2_13304: (21.253906) net_send: could not write to fd=5, errno = 32
p6_19895: p4_error: interrupt SIGSEGV: 11
rm_l_6_19974: (19.539062) net_send: could not write to fd=5, errno = 32
(shr_sys_abort) ERROR: ice_comp.F90
(shr_sys_abort) WARNING: calling shr_mpi_abort() and stopping
p7_11982: p4_error: interrupt SIGSEGV: 11
rm_l_7_12061: (18.652344) net_send: could not write to fd=5, errno = 32
1978 1 1
0
Stop date (yr mon day tod): 2000 1 1
0
Reference date (yr mon day tod): 1978 1 1
0
Current step number: 0
Current date (yr mon day tod): 1978 1 1
0
************************************************
(GETFIL): attempting to find local file domain.camocn.64x128_USGS_070807.nc
(GETFIL): using
/hpc/home/sislam/ccsm4_working_copy/inputdata/atm/cam/ocnfrac/domain.camocn.64x128_USGS_070807.nc
p5_16412: p4_error: interrupt SIGSEGV: 11
rm_l_5_16491: (20.074219) net_send: could not write to fd=5, errno = 32

-------------------------------------------------------

Siraj

 

[sirajkhan78@gmail_com]

The part of the script I mention above is for climatology run. Here is the part of the script I am using for control run.

cd $blddir
echo " building the namelist..."
$cfgdir/build-namelist -case $case -config $blddir/config_cache.xml
-ignore_ic_year
-namelist "&exp start_type='startup' start_ymd=19780101
stop_option='nyears' stop_n=22 restart_option='monthly'
bndtvs='$CSMDATA/atm/cam/sst/sst_HadOIBl_bc_64x128_1850_2008_c100128.nc'
sstcyc=.false. stream_year_first=1850 stream_year_last=2008 /"
echo "build-namelist done."

 

[eaton]

The problem is that "-phys cam3" is an unadvertised, and not fully implemented feature of configure. You need to realize that in CAM4 the "cam3" option only gets you part of the way back to cam3 physics. The cam3 aerosol and ozone datasets are used, and some of the physics changes between cam3 and cam4 are correctly reverted back to the cam3 version when this option is used. But the most important changes to the physics going from cam3 to cam4 were the Neale/Richter mods which modified the deep convection parameterization, and these changes are not yet backed out when the cam3 option is specified. This is something we are working on and will probably release in an update to the cesm code in the near future (this will not make it into the next cesm release which will happen in just a few weeks).

That said, the reason your run is failing is that the "-phys cam3" option is currently causing the old sea ice model which was used in cam3, i.e., CSIM4, to be used in place of the new sea ice model, CICE. I'm not sure why CSIM4 is not working with the AMIP SST dataset, but I was able to reproduce your problem. The workaround is to use CICE (I'll change configure so that this happens by default in the future). So in your configure command use the options "-phys cam3 -ice cice".