Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

yellowstone model output with compressed NetCDF4 format (netcdf4c)

X

xiaoyihf@gmail_com

Member
Dear all, I'm trying to use compressed NetCDF4 format in release cesm1_0_5 from yellowstone to save space on HPSS. I followed the following instruction from user's guide by setting netcdf4c in env_run.xml  http://www.cesm.ucar.edu/models/cesm1.0/cesm/cesm_doc_1_0_4/c1810.html#running_ccsm_env_output But the model complains that PIO was not built with NETCDF 4 run/ccsm.log.130910-135427: 139: WARNING: PIO was not built with NETCDF 4 support changing iotype to netcdfI checked the model was built with netcdf/4.2 Any suggestions? Thanks!
 
J

jedwards

CSEG and Liaisons
Staff member
You need to update the modules in env_mach_specific to use a parallel build of netcdf4.   Although parallel is not required or even supported for netcdf4c files, it is the way pio determines if netcdf4c is supported or not.  (If netcdf4p is available so is netcdf4c)
 
J

jedwards

CSEG and Liaisons
Staff member
You need to update the modules in env_mach_specific to use a parallel build of netcdf4.   Although parallel is not required or even supported for netcdf4c files, it is the way pio determines if netcdf4c is supported or not.  (If netcdf4p is available so is netcdf4c)
 
X

Xax

Member
Thank you very much. Is there also a simple way changin this pio check?
I'm currently on a system without netcdf parallel installed.

So my statement above that I have the precise same problem is not 100% correct, as I'm not working on Yellowstone.
 
X

Xax

Member
Thank you very much. Is there also a simple way changin this pio check?
I'm currently on a system without netcdf parallel installed.

So my statement above that I have the precise same problem is not 100% correct, as I'm not working on Yellowstone.
 
J

jedwards

CSEG and Liaisons
Staff member
No there isn't, at least in the version of the model you are using.   It would be easier to build and install parallel netcdf4.  
 
J

jedwards

CSEG and Liaisons
Staff member
No there isn't, at least in the version of the model you are using.   It would be easier to build and install parallel netcdf4.  
 
X

Xax

Member
thank you j.edwards.

I have compiled and installed pnetcdf and added the path to PNETCDF_PATH in Macros.

Within the cesm buildlog I now receive many unresolved reference errors. I'm not quite sure what they are pointing at, my MPI is properly installed and never caused any issues.

This is a subset of all these similar errors: /disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/ncmpidtype.c:356: undefined reference to `MPI_Type_get_contents'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/ncmpidtype.c:416: undefined reference to `MPI_Type_free'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/ncmpidtype.c:347: undefined reference to `MPI_Type_size'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/ncmpidtype.c:430: undefined reference to `MPI_Type_free'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/ncmpidtype.c:393: undefined reference to `MPI_Type_free'/disk02/usr/people/user/pnetcdf//lib/libpnetcdf.a(ncmpidtype.o): In function `ncmpii_data_repack':/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/ncmpidtype.c:547: undefined reference to `MPI_Type_size'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/ncmpidtype.c:548: undefined reference to `MPI_Type_size'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/ncmpidtype.c:566: undefined reference to `MPI_Pack_size'/disk02/usr/people/user/pnetcdf//lib/libpnetcdf.a(filetype.o): In function `ncmpii_type_create_subarray64':/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:486: undefined reference to `MPI_Type_get_extent'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:537: undefined reference to `MPI_Type_create_hvector'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:545: undefined reference to `MPI_Type_commit'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:562: undefined reference to `MPI_Type_commit'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:563: undefined reference to `MPI_Type_free'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:554: undefined reference to `MPI_Type_create_hvector'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:502: undefined reference to `MPI_Type_create_hindexed'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:505: undefined reference to `MPI_Type_commit'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:508: undefined reference to `MPI_Type_create_resized'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:456: undefined reference to `MPI_Type_create_subarray'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:511: undefined reference to `MPI_Type_free'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:584: undefined reference to `MPI_Type_create_hindexed'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:587: undefined reference to `MPI_Type_commit'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:588: undefined reference to `MPI_Type_create_resized'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:591: undefined reference to `MPI_Type_free'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:601: undefined reference to `MPI_Type_free'/disk02/usr/people/user/pnetcdf//lib/libpnetcdf.a(filetype.o): In function `ncmpii_vara_create_filetype':/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:743: undefined reference to `MPI_Type_commit'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:710: undefined reference to `MPI_Type_create_hvector'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:720: undefined reference to `MPI_Type_free'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:702: undefined reference to `MPI_Type_commit'/disk02/usr/people/user/pnetcdf//lib/libpnetcdf.a(filetype.o): In function `ncmpii_vars_create_filetype':/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:871: undefined reference to `MPI_Type_commit'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:873: undefined reference to `MPI_Type_free'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:855: undefined reference to `MPI_Type_create_hvector'/disk02/usr/people/user/pnetcdf//lib/libpnetcdf.a(filetype.o): In function `ncmpii_file_set_view':/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:962: undefined reference to `MPI_Type_create_struct'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:966: undefined reference to `MPI_Type_commit'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:968: undefined reference to `MPI_File_set_view'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:970: undefined reference to `MPI_Type_free'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:976: undefined reference to `MPI_File_set_view'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:929: undefined reference to `MPI_File_set_view'
 
X

Xax

Member
thank you j.edwards.

I have compiled and installed pnetcdf and added the path to PNETCDF_PATH in Macros.

Within the cesm buildlog I now receive many unresolved reference errors. I'm not quite sure what they are pointing at, my MPI is properly installed and never caused any issues.

This is a subset of all these similar errors: /disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/ncmpidtype.c:356: undefined reference to `MPI_Type_get_contents'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/ncmpidtype.c:416: undefined reference to `MPI_Type_free'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/ncmpidtype.c:347: undefined reference to `MPI_Type_size'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/ncmpidtype.c:430: undefined reference to `MPI_Type_free'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/ncmpidtype.c:393: undefined reference to `MPI_Type_free'/disk02/usr/people/user/pnetcdf//lib/libpnetcdf.a(ncmpidtype.o): In function `ncmpii_data_repack':/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/ncmpidtype.c:547: undefined reference to `MPI_Type_size'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/ncmpidtype.c:548: undefined reference to `MPI_Type_size'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/ncmpidtype.c:566: undefined reference to `MPI_Pack_size'/disk02/usr/people/user/pnetcdf//lib/libpnetcdf.a(filetype.o): In function `ncmpii_type_create_subarray64':/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:486: undefined reference to `MPI_Type_get_extent'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:537: undefined reference to `MPI_Type_create_hvector'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:545: undefined reference to `MPI_Type_commit'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:562: undefined reference to `MPI_Type_commit'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:563: undefined reference to `MPI_Type_free'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:554: undefined reference to `MPI_Type_create_hvector'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:502: undefined reference to `MPI_Type_create_hindexed'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:505: undefined reference to `MPI_Type_commit'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:508: undefined reference to `MPI_Type_create_resized'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:456: undefined reference to `MPI_Type_create_subarray'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:511: undefined reference to `MPI_Type_free'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:584: undefined reference to `MPI_Type_create_hindexed'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:587: undefined reference to `MPI_Type_commit'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:588: undefined reference to `MPI_Type_create_resized'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:591: undefined reference to `MPI_Type_free'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:601: undefined reference to `MPI_Type_free'/disk02/usr/people/user/pnetcdf//lib/libpnetcdf.a(filetype.o): In function `ncmpii_vara_create_filetype':/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:743: undefined reference to `MPI_Type_commit'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:710: undefined reference to `MPI_Type_create_hvector'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:720: undefined reference to `MPI_Type_free'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:702: undefined reference to `MPI_Type_commit'/disk02/usr/people/user/pnetcdf//lib/libpnetcdf.a(filetype.o): In function `ncmpii_vars_create_filetype':/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:871: undefined reference to `MPI_Type_commit'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:873: undefined reference to `MPI_Type_free'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:855: undefined reference to `MPI_Type_create_hvector'/disk02/usr/people/user/pnetcdf//lib/libpnetcdf.a(filetype.o): In function `ncmpii_file_set_view':/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:962: undefined reference to `MPI_Type_create_struct'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:966: undefined reference to `MPI_Type_commit'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:968: undefined reference to `MPI_File_set_view'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:970: undefined reference to `MPI_Type_free'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:976: undefined reference to `MPI_File_set_view'/disk02/usr/people/user/pnetcdf/parallel-netcdf-1.7.0/src/lib/filetype.c:929: undefined reference to `MPI_File_set_view'
 
X

Xax

Member
I can successfully build cesm with pnetcdf when switching the mpi lib to "mpi". Previously it was set to "mpi-serial".

Is there a way to get it to work also with mpi-serial?
 
X

Xax

Member
I can successfully build cesm with pnetcdf when switching the mpi lib to "mpi". Previously it was set to "mpi-serial".

Is there a way to get it to work also with mpi-serial?
 
You must log in or register to reply here.
Top