Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

CLM Model Segmentation Fault When Using Custom Aerosol Dataset

G

gaoxiongwei

yfT5XpMP8fVkXg3
New Member
Hi,

I am using CTSM version 5.3.063 to build a 0.5-degree resolution case with compset I2000Clm60SpCrujra:

./create_newcase --case I2000Clm60SpCrujra --compset I2000Clm60SpCrujra --res hcru_hcru_mt13 --mach gaoxiongwei --run-unsupported

However, I am using a custom aerosol dataset. I configured the model as follows:

./xmlchange RUN_STARTDATE=1987-06-05
./xmlchange DATM_YR_START=1987
./xmlchange DATM_YR_END=1987
./xmlchange DATM_YR_ALIGN=1987
./xmlchange DEBUG=TRUE

I also set the model to perform a cold start:

./xmlchange CLM_FORCE_COLDSTART=on

In user_nl_clm, I set:

hist_nhtfrq = -24
hist_mfilt = 1
use_snicar_frc = .true.

Additionally, I modified the stream_definition_datm.xml file to include the following entry:
<stream_entry name="presaero.clim_2000">
<stream_meshfile>
<meshfile>$DIN_LOC_ROOT/atm/datm7/atm_forcing.datm7.CRUJRA.0.5d.c20241231/three_stream/mesh_0.5x0.5_CRUJRAv2.5mask_cdf5_c20240916.nc</meshfile>
</stream_meshfile>
<stream_datafiles>
<file>/home/gaoxiongwei/esmf/cesm_inputdata/atm/cam/chem/trop_mozart_aero/aero/aero_198706_modified_interpolated.nc</file>
</stream_datafiles>
<stream_datavars>
<var>BCDEPWET Faxa_bcphiwet</var>
<var>BCPHODRY Faxa_bcphodry</var>
<var>BCPHIDRY Faxa_bcphidry</var>
<var>OCDEPWET Faxa_ocphiwet</var>
<var>OCPHIDRY Faxa_ocphidry</var>
<var>OCPHODRY Faxa_ocphodry</var>
<var>DSTX01WD Faxa_dstwet1</var>
<var>DSTX01DD Faxa_dstdry1</var>
<var>DSTX02WD Faxa_dstwet2</var>
<var>DSTX02DD Faxa_dstdry2</var>
<var>DSTX03WD Faxa_dstwet3</var>
<var>DSTX03DD Faxa_dstdry3</var>
<var>DSTX04WD Faxa_dstwet4</var>
<var>DSTX04DD Faxa_dstdry4</var>
</stream_datavars>
<stream_lev_dimname>null</stream_lev_dimname>
<stream_mapalgo>
<mapalgo>bilinear</mapalgo>
</stream_mapalgo>
<stream_vectors>null</stream_vectors>
<stream_year_align>1</stream_year_align>
<stream_year_first>1987</stream_year_first>
<stream_year_last>1987</stream_year_last>
<stream_offset>0</stream_offset>
<stream_tintalgo>
<tintalgo>linear</tintalgo>
</stream_tintalgo>
<stream_taxmode>
<taxmode>cycle</taxmode>
</stream_taxmode>
<stream_dtlimit>
<dtlimit>1.5</dtlimit>
</stream_dtlimit>
<stream_readmode>single</stream_readmode>
</stream_entry>

Despite all these settings, the CLM model fails with a segmentation fault. The error message in the cesm.log file is:

[0] (t_initf) Read in prof_inparm namelist from: drv_in
...
===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= PID 51221 RUNNING AT gaoxiongwei
= EXIT CODE: 139
= CLEANING UP REMAINING PROCESSES
...
YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11)


My aerosol dataset contains hourly data from June 1 to June 30, 1987. While it is not a full year, it does match the simulation period I configured. My aerosol dataset has a 0.5-degree spatial resolution with hourly data covering the entire month of June 1987. Since the file is too large to upload, I’m providing the list of variables along with their detailed attribute information for your reference. I hope this will be helpful.
The NetCDF file (6GB, hourly aerosol data at 0.5° resolution for June 1987) is available on Google Drive:
netcdf F:\cam\aero_198706_modified_interpolated.nc {
dimensions:
time = 720;
lat = 360;
lon = 720;
variables:
double time(time=720);
:_FillValue = NaN; // double
:standard_name = "time";
:long_name = "time";
:axis = "T";
:cell_methods = "time: mean";
:units = "days since 1987-06-01 00:30:00";
:calendar = "noleap";

int datesec(time=720);
:long_name = "current seconds of current date";
:cell_methods = "time: mean";

float lat(lat=360);
:_FillValue = -900.0f; // float
:long_name = "latitude";
:units = "degrees_north";

float lon(lon=720);
:_FillValue = -900.0f; // float
:long_name = "longitude";
:units = "degrees_east";

float BCDEPWET(time=720, lat=360, lon=720);
:units = "kg m-2 s-1";
:cell_methods = "time: mean";
:_ChunkSizes = 180U, 90U, 180U; // uint
:long_name = "Hydrophylic black carbon wet deposition flux";
:_FillValue = 1.0E30f; // float

float BCPHIDRY(time=720, lat=360, lon=720);
:_ChunkSizes = 180U, 90U, 180U; // uint
:long_name = "Hydrophylic black carbon dry deposition flux";
:_FillValue = 1.0E30f; // float
:units = "kg m-2 s-1";
:cell_methods = "time: mean";

float BCPHODRY(time=720, lat=360, lon=720);
:long_name = "Hydrophobic black carbon dry deposition flux";
:_ChunkSizes = 180U, 90U, 180U; // uint
:_FillValue = 1.0E30f; // float
:units = "kg m-2 s-1";
:cell_methods = "time: mean";

float DSTX01DD(time=720, lat=360, lon=720);
:long_name = "Dust dry deposition flux (size 1)";
:cell_methods = "time: mean";
:_ChunkSizes = 180U, 90U, 180U; // uint
:_FillValue = 1.0E30f; // float
:units = "kg m-2 s-1";

float DSTX01WD(time=720, lat=360, lon=720);
:long_name = "Dust wet deposition flux (size 1)";
:cell_methods = "time: mean";
:_ChunkSizes = 180U, 90U, 180U; // uint
:units = "kg m-2 s-1";
:_FillValue = 1.0E30f; // float

float DSTX02DD(time=720, lat=360, lon=720);
:_ChunkSizes = 180U, 90U, 180U; // uint
:long_name = "Dust dry deposition flux (size 2)";
:cell_methods = "time: mean";
:_FillValue = 1.0E30f; // float
:units = "kg m-2 s-1";

float DSTX02WD(time=720, lat=360, lon=720);
:long_name = "Dust wet deposition flux (size 2)";
:units = "kg m-2 s-1";
:cell_methods = "time: mean";
:_ChunkSizes = 180U, 90U, 180U; // uint
:_FillValue = 1.0E30f; // float

float DSTX03DD(time=720, lat=360, lon=720);
:_FillValue = 1.0E30f; // float
:units = "kg m-2 s-1";
:cell_methods = "time: mean";
:_ChunkSizes = 180U, 90U, 180U; // uint
:long_name = "Dust dry deposition flux (size 3)";

float DSTX03WD(time=720, lat=360, lon=720);
:long_name = "Dust wet deposition flux (size 3)";
:cell_methods = "time: mean";
:_ChunkSizes = 180U, 90U, 180U; // uint
:units = "kg m-2 s-1";
:_FillValue = 1.0E30f; // float

float DSTX04DD(time=720, lat=360, lon=720);
:_ChunkSizes = 180U, 90U, 180U; // uint
:long_name = "Dust dry deposition flux (size 4)";
:units = "kg m-2 s-1";
:cell_methods = "time: mean";
:_FillValue = 1.0E30f; // float

float DSTX04WD(time=720, lat=360, lon=720);
:_ChunkSizes = 180U, 90U, 180U; // uint
:long_name = "Dust wet deposition flux (size 4)";
:_FillValue = 1.0E30f; // float
:units = "kg m-2 s-1";
:cell_methods = "time: mean";

float OCDEPWET(time=720, lat=360, lon=720);
:units = "kg m-2 s-1";
:cell_methods = "time: mean";
:_ChunkSizes = 180U, 90U, 180U; // uint
:long_name = "Hydrophylic organic carbon wet deposition flux";
:_FillValue = 1.0E30f; // float

float OCPHIDRY(time=720, lat=360, lon=720);
:long_name = "Hydrophylic organic carbon dry deposition flux";
:_FillValue = 1.0E30f; // float
:units = "kg m-2 s-1";
:cell_methods = "time: mean";
:_ChunkSizes = 180U, 90U, 180U; // uint

float OCPHODRY(time=720, lat=360, lon=720);
:_ChunkSizes = 180U, 90U, 180U; // uint
:long_name = "Hydrophobic organic carbon dry deposition flux";
:_FillValue = 1.0E30f; // float
:units = "kg m-2 s-1";
:cell_methods = "time: mean";

int date(time=720);
:cell_methods = "time: mean";
:_ChunkSizes = 720; // int
:long_name = "current date (YYYYMMDD)";

// global attributes:
}

From the atm.log output, it seems that CESM does recognize and attempts to open my aerosol file:

(shr_strdata_readstrm) opening : /home/gaoxiongwei/esmf/cesm_inputdata/atm/cam/chem/trop_mozart_aero/aero/aero_198706_modified_interpolated.nc
(shr_strdata_readstrm) setting pio descriptor : /home/gaoxiongwei/esmf/cesm_inputdata/atm/cam/chem/trop_mozart_aero/aero/aero_198706_modified_interpolated.nc
However, after this point, there’s no indication that any variables were actually read or processed. In contrast, for other forcing files like Solr, Prec, and TPQWL, I can see the variables being associated (e.g., FSDS, PRECTmms, TBOT) and the corresponding dimensions being recognized (dimlens(1), dimlens(2)).
I’m wondering:
  • Could it be that CESM is not recognizing the variables in my aerosol file? (however, my variable settings seem to be consistent with the default file)
  • Is there an issue with the time dimension in the file?
  • Are there any additional namelist or XML settings required to ensure proper coupling?
Any help or guidance on getting CESM to fully recognize and use this aerosol file would be greatly appreciated.

I have also attached the following files for your reference:
  • cesm.log
  • lnd.log
  • atm.log
  • med.log
  • user_nl_clm
I am eager to understand the cause of this segmentation fault and how to resolve it. Any help or suggestions would be greatly appreciated. Thank you very much!
 

Attachments

  • atm.log.250807-223714.txt
    16.2 KB · Views: 0
  • cesm.log.250807-223714.txt
    1.2 KB · Views: 1
  • lnd.log.250807-223714.txt
    27.8 KB · Views: 1
  • med.log.250807-223714.txt
    40.6 KB · Views: 0
  • user_nl_clm.txt
    1.4 KB · Views: 0
  • stream_definition_datm.xml.txt
    183.9 KB · Views: 0
Sort by date Sort by votes
slevis

slevis

Moderator
Staff member
I usually first suggest this:
Confirm that you can run a simulation without the custom dataset. If you can (maybe you have already), then make the custom dataset as similar as possible to the default one. Make your changes gradually from the default to the custom so that you can eventually identify the exact step where you made a mistake and caused the custom dataset to fail.
 
Vote Upvote 0 Downvote
You must log in or register to reply here.
Top