ERROR: RUN FAIL: in CLM5

[MrIgnition]

I successfully submitted a case but simulation was not done. While this may be caused by many reasons, is it possible to have an overview of things that could make run fail even when check_case shows your case is OK?
Please attached is cesm.log file to see if a solution can be provided.

 

[oleson]

This looks like a datm-related error (data atmosphere model). Are you running CLM5 out of the box (with no changes)? If not, maybe you have an error in one of your datm files (e.g., datm_in). You might get more information about the error by looking at your atm.log

 

[MrIgnition]

Thank you Oleson for this. It made impact. I have checked my atm.log file and found this...


(datm_comp_init) decomp = 1d
(datm_comp_init) iradsw = 1
(datm_comp_init) factorFn = null
(datm_comp_init) restfilm = undefined
(datm_comp_init) restfils = undefined
(datm_comp_init) presaero = T
(datm_comp_init) force_prognostic_true = F
(datm_comp_init) wiso_datm = F
(datm_comp_init) inst_index = 1
(datm_comp_init) inst_name = ATM
(datm_comp_init) inst_suffix =
(shr_strdata_readnml) reading input namelist file: datm_in
(shr_stream_init) Reading file user_datm.streams.cordexAfricacase.solar.stream.txt
(shr_stream_init) ERROR: unexpected end-of-file while reading <dataSource>
(shr_stream_init) error code = ^]^@^@^@
ERROR: (shr_stream_init) ERROR: unexpected end-of-file

It suggests that either the file user_datm.streams.cordexAfricacase.solar.stream.txt could not be read or the netcdf files listed in it could not be read. I located the netcdf files and they "looked" okay. However, I am yet to locate user_datm.streams.cordexAfricacase.solar.stream.txt. Any help from here please?

 

[MrIgnition]

Thanks once again Oleson. The error has now been corrected. It was due to a type in the stream file specification.
I however saw another error as shown in my cesm.log. file.

the last few lines containing the error are hereby attached.


NODE# NAME
( 0) jrc0673
NetCDF: Invalid dimension ID or name
NetCDF: Invalid dimension ID or name
NetCDF: Invalid dimension ID or name
NetCDF: Invalid dimension ID or name
NetCDF: Invalid dimension ID or name
NetCDF: Variable not found
NetCDF: Invalid dimension ID or name
NetCDF: Invalid dimension ID or name
NetCDF: Invalid dimension ID or name
NetCDF: Invalid dimension ID or name
NetCDF: Invalid dimension ID or name
ERROR:
ERROR: lonc is negative and currently can NOT be (see ESCOM
P/ctsm/issues/507)ERROR in surfrdMod.F90 at line 243
Image PC Routine Line Source
cesm.exe 00000000012515FA Unknown Unknown Unknown
cesm.exe 0000000000D9BEBE shr_abort_mod_mp_ 114 shr_abort_mod.F90
cesm.exe 000000000050AC0F abortutils_mp_end 50 abortutils.F90
cesm.exe 000000000065A0B4 surfrdmod_mp_surf 242 surfrdMod.F90
cesm.exe 000000000051B628 clm_initializemod 143 clm_initializeMod.F90
cesm.exe 0000000000502BF5 lnd_comp_mct_mp_l 199 lnd_comp_mct.F90
cesm.exe 0000000000435CD9 component_mod_mp_ 267 component_mod.F90
cesm.exe 0000000000424CF4 cime_comp_mod_mp_ 1237 cime_comp_mod.F90
cesm.exe 0000000000432E59 MAIN__ 114 cime_driver.F90
cesm.exe 0000000000416592 Unknown Unknown Unknown
libc-2.17.so 00002B5EE014A505 __libc_start_main Unknown Unknown
cesm.exe 00000000004164A9 Unknown Unknown Unknown
application called MPI_Abort(MPI_COMM_WORLD, 1001) - process 0srun: error: jrc0673: task 0: Exited with exit code 233

 

[oleson]

The error indicates that there are negative longitudes in your surface dataset, which is not allowed for various reasons at this point in time. If your domain goes over the zero longitude line, the only workaround at this time is to split your domain in two.

 

[MrIgnition]

The error indicates that there are negative longitudes in your surface dataset, which is not allowed for various reasons at this point in time. If your domain goes over the zero longitude line, the only workaround at this time is to split your domain in two.

Thanks. I appreciate.
Just wondering. From my experience in atmospheric modelling, boundary value problems do exist. Can I avoid that if I split my domain into two in CLM? This will go a long way to determine the course of my research.
Thanks once again Oleson, I'm grateful.

 

[oleson]

If the spatial resolution of your atmospheric forcing data is the same as your surface dataset, then there shouldn't be any differences between the answers you get by splitting the domain and the answers you would get if you could run the entire domain. If the resolutions are different it's possible there could be answer differences due to interpolation of the atm data to the domain(s).

 

[MrIgnition]

If the spatial resolution of your atmospheric forcing data is the same as your surface dataset, then there shouldn't be any differences between the answers you get by splitting the domain and the answers you would get if you could run the entire domain. If the resolutions are different it's possible there could be answer differences due to interpolation of the atm data to the domain(s).

Thanks Oleson,
I completely agree. Especially because there is no horizontal transport in CLM5. So I'll get back to work and make some more progress.
Thanks once again.

 

[MrIgnition]

The error indicates that there are negative longitudes in your surface dataset, which is not allowed for various reasons at this point in time. If your domain goes over the zero longitude line, the only workaround at this time is to split your domain in two.

I was thinking I understood this. I splitted my domain into two. the negative longitude and the positive longitude, then made a case for each. Starting with the negative longitude domain, i still get the same error. Must the whole domain be in the same quadrant (negative lon, negative lat)? Please help.

 

[oleson]

You need to express your negative longitude domain in terms of positive longitudes (degrees E).

 

[MrIgnition]

You need to express your negative longitude domain in terms of positive longitudes (degrees E).

Thank you for this, Kindly help see if this configuration for creating SCRIPGRID file is correct. I have tried performing the simulation even on the positive domain but the same error keeps coming up.

export PTNAME=cordexAfricaWest
export S_LAT=-45.76
export N_LAT=42.24
export E_LON=359.99
export W_LON=335.36
export NX=57
export NY=201


export PTNAME=cordexAfricaEast
export S_LAT=-45.76
export N_LAT=42.24
export E_LON=60.28
export W_LON=00.00
export NX=138
export NY=201

 

[MrIgnition]

I have also tried to perform simulation over a smaller domain within 2 - 15 Degrees Longitude and 2 - 15 Degrees latitude but the same error saying lonc is negative still comes up. This is surprising. I have also looked at the domain file itself and there is even no lonc or longitude within its dimensions except for lsmlon and lsmlat which didnt even have any negative values. What do you advise please.

 

[oleson]

Looking at the code (surfrdMod.F90) it looks like it should be reading off 'xc' and 'yc' from the domain file to get lonc and latc, respectively. Do you have any negative 'xc' on your domain file?

 

[MrIgnition]

Thank you so much for responding. I do have a negative xc in the domain file which seems to be global in coverage.

 

[MrIgnition]

Thank you for that tip.
i have now used ncap2 -O -s 'where(xc<0) xc=360+xc' ifile ofile to correct the domain file. And for the first time, simulation had gone beyond 7 minutes. Seems this error has been fixed.

Now Christmas can happen.
Thanks once again Oleson.

 

[komorebi]

Thanks once again Oleson. The error has now been corrected. It was due to a type in the stream file specification.
I however saw another error as shown in my cesm.log. file.

the last few lines containing the error are hereby attached.


NODE# NAME
( 0) jrc0673
NetCDF: Invalid dimension ID or name
NetCDF: Invalid dimension ID or name
NetCDF: Invalid dimension ID or name
NetCDF: Invalid dimension ID or name
NetCDF: Invalid dimension ID or name
NetCDF: Variable not found
NetCDF: Invalid dimension ID or name
NetCDF: Invalid dimension ID or name
NetCDF: Invalid dimension ID or name
NetCDF: Invalid dimension ID or name
NetCDF: Invalid dimension ID or name
ERROR:
ERROR: lonc is negative and currently can NOT be (see ESCOM
P/ctsm/issues/507)ERROR in surfrdMod.F90 at line 243
Image PC Routine Line Source
cesm.exe 00000000012515FA Unknown Unknown Unknown
cesm.exe 0000000000D9BEBE shr_abort_mod_mp_ 114 shr_abort_mod.F90
cesm.exe 000000000050AC0F abortutils_mp_end 50 abortutils.F90
cesm.exe 000000000065A0B4 surfrdmod_mp_surf 242 surfrdMod.F90
cesm.exe 000000000051B628 clm_initializemod 143 clm_initializeMod.F90
cesm.exe 0000000000502BF5 lnd_comp_mct_mp_l 199 lnd_comp_mct.F90
cesm.exe 0000000000435CD9 component_mod_mp_ 267 component_mod.F90
cesm.exe 0000000000424CF4 cime_comp_mod_mp_ 1237 cime_comp_mod.F90
cesm.exe 0000000000432E59 MAIN__ 114 cime_driver.F90
cesm.exe 0000000000416592 Unknown Unknown Unknown
libc-2.17.so 00002B5EE014A505 __libc_start_main Unknown Unknown
cesm.exe 00000000004164A9 Unknown Unknown Unknown
application called MPI_Abort(MPI_COMM_WORLD, 1001) - process 0srun: error: jrc0673: task 0: Exited with exit code 233

Hello, could you please give me a demonstration of stream file description? I have also encountered similar problems. I would appreciate it very much if you can.

 

[oleson]

Which streams file are you referring to? There should be streams files generated when you set up a case that you can use as examples.

 

[MrIgnition]

The error indicates that there are negative longitudes in your surface dataset, which is not allowed for various reasons at this point in time. If your domain goes over the zero longitude line, the only workaround at this time is to split your domain in two.

UPDATE:
Today we were able to run CLM5 over Africa (which has both negative and positive longitudes) as a single domain.

• We designated W_LON as negative longitudes during the creation of both the domain and surface files.
• We then used NCO’s ncap2 to change the negative longitudes to positive values specifying “where xc<0, xc=xc+360” and “where xv<0, xv=xv+360” for the domain file. A similar adjustment was made for the LONGXY variable in the surface file.

This has worked for the GSWP forcing and I think it will work for CRUNCEP as well.