Hi all
I have been struggling to run a restart run when doing regional simulations with ctsm5.3.012. I have previously succesfully set up a spinup simulation over a region of Central Africa with various model configurations (I2000Clm50Sp,I2000ClmFATES). Now my goal is to start a new run with transient climate. I know how to set this up with the xml commands and finidat for the FATES initialisation from previous experience with single site runs. The regional land surface and domain datasets were created with the python commands provided for this.
However my regional runs crash early in the simulation, they end just before finishing the initialisation of history output (see lnd log) with a segmentation fault and not much more info (something about memory mismatch in the cesm log attached, not any specific info on the crash in the lnd log). Other (no restart) simulations work fine so it doesn't seem to be related to limited resources or anything like that. Is there currently a known bug related to restarting runs from regional simulations?
The only few things to note is that when submitting my case the model tries to download a mosart restart file (see below) despite my regional runs never having created any mosart output during the spinup runs. That's another side question I have, is it normal that mosart doesn't run for these compsets or is this incompatible with the regional setup?
Any input is much appreciated!
Cheers
Steven
Model mosart missing file finidat = '/scratch/brussel/vo/000/bvo00003/vsc46573/cesm/output/FATES_SPINUP_I2000Clm50Fates_congo_def.03-25/run/FATES_SPINUP_I2000Clm50Fates_congo_def.03-25.mosart.r.2150-01-01-00000.nc'
Checking server ftp://gridanon.cgd.ucar.edu:2811/cesm/inputdata/ with protocol gftp
Setting resource.RLIMIT_STACK to -1 from (-1, -1)
Client protocol gftp not enabled
Checking server ftp://ftp.cgd.ucar.edu/cesm/inputdata/ with protocol wget
Setting resource.RLIMIT_STACK to -1 from (-1, -1)
Using protocol wget with user anonymous and passwd user@example.edu
Could not connect to repo 'ftp://ftp.cgd.ucar.edu/cesm/inputdata/'
This is most likely either a proxy, or network issue .(location 2)
Checking server ftp.cgd.ucar.edu/cesm/inputdata with protocol ftp
Setting resource.RLIMIT_STACK to -1 from (-1, -1)
Using protocol ftp with user anonymous and passwd user@example.edu
server address ftp.cgd.ucar.edu root path cesm/inputdata
ftp login timeout! [Errno 111] Connection refused
Checking server - Revision 70792: /trunk/inputdata with protocol svn
Setting resource.RLIMIT_STACK to -1 from (-1, -1)
Using protocol svn with user and passwd
Loading input file list: 'Buildconf/ctsm.input_data_list'
Loading input file list: 'Buildconf/cpl.input_data_list'
Loading input file list: 'Buildconf/datm.input_data_list'
Loading input file list: 'Buildconf/mosart.input_data_list'
Model mosart missing file finidat = '/scratch/brussel/vo/000/bvo00003/vsc46573/cesm/output/FATES_SPINUP_I2000Clm50Fates_congo_def.03-25/run/FATES_SPINUP_I2000Clm50Fates_congo_def.03-25.mosart.r.2150-01-01-00000.nc'
Cannot download file since it lives outside of the input_data_root '/scratch/brussel/vo/000/bvo00003/vsc46573/cesm/inputdata'
GET_REFCASE is false, the user is expected to stage the refcase to the run directory.
Check case OK
I have been struggling to run a restart run when doing regional simulations with ctsm5.3.012. I have previously succesfully set up a spinup simulation over a region of Central Africa with various model configurations (I2000Clm50Sp,I2000ClmFATES). Now my goal is to start a new run with transient climate. I know how to set this up with the xml commands and finidat for the FATES initialisation from previous experience with single site runs. The regional land surface and domain datasets were created with the python commands provided for this.
However my regional runs crash early in the simulation, they end just before finishing the initialisation of history output (see lnd log) with a segmentation fault and not much more info (something about memory mismatch in the cesm log attached, not any specific info on the crash in the lnd log). Other (no restart) simulations work fine so it doesn't seem to be related to limited resources or anything like that. Is there currently a known bug related to restarting runs from regional simulations?
The only few things to note is that when submitting my case the model tries to download a mosart restart file (see below) despite my regional runs never having created any mosart output during the spinup runs. That's another side question I have, is it normal that mosart doesn't run for these compsets or is this incompatible with the regional setup?
Any input is much appreciated!
Cheers
Steven
Model mosart missing file finidat = '/scratch/brussel/vo/000/bvo00003/vsc46573/cesm/output/FATES_SPINUP_I2000Clm50Fates_congo_def.03-25/run/FATES_SPINUP_I2000Clm50Fates_congo_def.03-25.mosart.r.2150-01-01-00000.nc'
Checking server ftp://gridanon.cgd.ucar.edu:2811/cesm/inputdata/ with protocol gftp
Setting resource.RLIMIT_STACK to -1 from (-1, -1)
Client protocol gftp not enabled
Checking server ftp://ftp.cgd.ucar.edu/cesm/inputdata/ with protocol wget
Setting resource.RLIMIT_STACK to -1 from (-1, -1)
Using protocol wget with user anonymous and passwd user@example.edu
Could not connect to repo 'ftp://ftp.cgd.ucar.edu/cesm/inputdata/'
This is most likely either a proxy, or network issue .(location 2)
Checking server ftp.cgd.ucar.edu/cesm/inputdata with protocol ftp
Setting resource.RLIMIT_STACK to -1 from (-1, -1)
Using protocol ftp with user anonymous and passwd user@example.edu
server address ftp.cgd.ucar.edu root path cesm/inputdata
ftp login timeout! [Errno 111] Connection refused
Checking server - Revision 70792: /trunk/inputdata with protocol svn
Setting resource.RLIMIT_STACK to -1 from (-1, -1)
Using protocol svn with user and passwd
Loading input file list: 'Buildconf/ctsm.input_data_list'
Loading input file list: 'Buildconf/cpl.input_data_list'
Loading input file list: 'Buildconf/datm.input_data_list'
Loading input file list: 'Buildconf/mosart.input_data_list'
Model mosart missing file finidat = '/scratch/brussel/vo/000/bvo00003/vsc46573/cesm/output/FATES_SPINUP_I2000Clm50Fates_congo_def.03-25/run/FATES_SPINUP_I2000Clm50Fates_congo_def.03-25.mosart.r.2150-01-01-00000.nc'
Cannot download file since it lives outside of the input_data_root '/scratch/brussel/vo/000/bvo00003/vsc46573/cesm/inputdata'
GET_REFCASE is false, the user is expected to stage the refcase to the run directory.
Check case OK
