This site is migrating to a new forum software on Tuesday, September 24th 2019, you may experience a short downtime during this transition

Main menu

Navigation

CESM run fails on hopper with ERROR: ionf_mod.F90

11 posts / 0 new
Last post
mvangordon@...
CESM run fails on hopper with ERROR: ionf_mod.F90

I'm working running CESM/CLM on hopper. I can build successfully, but the run fails with the following error in my $RUNDIR/cesm.log... file:

...
Opened existing file
/project/projectdirs/ccsm1/inputdata/share/domains/domain.lnd.fv0.9x1.25_gx1v6.090309.nc
65536
Rank 1 [Thu Mar 13 23:28:58 2014] [c9-2c2s7n3] application called MPI_Abort(MPI_COMM_WORLD, 1) - proce
ss 1
pio_support::pio_die:: myrank= -1 : ERROR: ionf_mod.F90:
228 : Permission denied
_pmiu_daemon(SIGCHLD): [NID 04785] [c9-2c2s7n3] [Thu Mar 13 23:28:58 2014] PE RANK 1 exit signal Abort
ed
[NID 04785] 2014-03-13 16:28:58 Apid 26957592: initiated application termination
...

I would appreciate any insight on what the issue is or how I should go about addressing it.

Thanks.

jedwards

Is this a reproducable error?   Sometimes intermitant file system issues will cause it.   If it is repeatable please let us know the details -

what is the model version, the case setup, any changes from the default configuration.

CESM Software Engineer

mvangordon@...

It is a reproducable error. Model version is CESM 1.2.1. Set up is as follows:

cesm1_2_1/scripts> ./create_newcase -case ~/cesm1_2_1/cases/quickstart_test -mach hopper -res f09_g16 -compset I1850CRUCLM45BGC

cases/quickstart_test> ./xmlchange STOP_OPTION=nmonth,STOP_N=1

cases/quickstart_test> ./cesm_setup

cases/quickstart_test> ./quickstart_test.build

cases/quickstart_test> ./quickstart_test.submit

jedwards

I just tried it twice and didn't have any problems.   Can you access that file with the netcdf ncdump command? 

CESM Software Engineer

mvangordon@...

I can, yes.

adrianne

I have gotten that error message when I was missing the clm2.rh restarts.

mvangordon@...

Hi Adrianne,

Could you offer some more detail? How/why would I be missing these and where should they be? This is a first quickstart test run without changing any defaults.

Thanks.

adrianne

If it's a first quickstart test, and you haven't changed any defaults, then the model should have given you initial datasets, so restarts (including the clm.rh files) aren't likely to be the problem.

erik

Hi


It looks like you aren't in the ccsm1 group. I think that might be an issue. You should be able to read the file, but I think sometimes it stilkl wants you to have write permission even on a read...

Here's the permissions on that file.

-rw-rw-r-- 1 mickelso ccsm1 5532048 Jul 29  2012 /project/projectdirs/ccsm1/inputdata/share/domains/domain.lnd.fv0.9x1.25_gx1v6.090309.nc

Try asking system folks at NERSC to add you to the ccsm1 group and see if it works after that.

It's also possible it's another file that it's having trouble with. You might need to load it in a debugger to see exactly where it dies and in what file.

 

Good luck.

 

Erik Kluzek ...............

CESM Land Model (CLM) Software Liason

CESM Software Engineering Group, NCAR

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

mvangordon@...

Thanks, Erik! Being added to the ccsm1 group fixed the issue.

wagmanbe@...

I had the same error on stampede. The solution was to use

 

./xmlchange DIN_LOC_ROOT= <some new directory>

Create that directory somewhere with plenty of space. When you setup/build, the input files will be sent over. I did chmod 777 on the problematic input file, but not sure if that was necessary. No further issues with permissions. 

 

Log in or register to post comments

Who's new

  • jwolff
  • tinna.gunnarsdo...
  • sarthak2235@...
  • eolivares@...
  • shubham.gandhi@...