Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

Problems when running the CAM5.0 alone

S

s9862323@m98_nthu_edu_tw

New Member
Hi guys,

I'm a complete novice at running CAM5.0 and I encountered the problems below which I cannot fix.

I used the commands below to build and run the CAM5.0 alone.
=======================================================================
export INC_NETCDF=$HOME/NetCDF-4.1.1-intel/include
export LIB_NETCDF=$HOME/NetCDF-4.1.1-intel/lib
export MOD_NETCDF=$HOME/NetCDF-4.1.1-intel/include
export INC_MPI=$HOME/MVAPICH2-1.6-intel/include
export LIB_MPI=$HOME/MVAPCH2-1.6-intel/lib
cd cesm1_0_4-intel/models/atm/cam/bld
./configure -dyn fv -hgrid 10x15 -ntasks 6 -nosmp -cc icc -fc ifort -fc_type intel
gmake -j2
export CSMDATA=$HOME/inputdata
./build-namelist -test -config ./config_cache.xml
./cam


Errors
================================================
13
/home/meir02/ccsm4_0/inputdata/atm/cam/chem/trop_mozart_aero/oxid/oxid_1.9x2.5_L26_1850-2005_c091123.nc
trcdata_init: file%has_ps = T
There are no offline tracer sources

airpl_src: NO and CO do not have external source --> no aircraft sources will be applied
chemini: after airpl_src on node 0
srf_emis_inti: n_emis_species = 9
pio_support:io_die:: myrank= 1 : ERROR: ionf_mod.F90:
212 : No such file or directory
rank 1 in job 15 water_56994 caused collective abort of all ranks
exit status of rank 1: killed by signal 9


After using the -test flag to configure, it seems I do not miss any file.
Could somebody help me to solve this problem?
Also, could somebody gives me some advice or materials I should study because I still don't know how to run this model very well after reading the user's guides and a lot of website I found.

Thanks for your help.

Albert
 
E

eaton

CSEG and Liaisons
It seems like the problem is with the MPI setup. I'd try a serial run first, i.e., replace the "-ntasks 6" argument by -nospmd. This will test that the code can be built and run with your ifort compiler and netcdf lib. If that works then go back to the MPI build and try "-ntasks 1". If that works, but using 6 tasks doesn't work, then I'd do some testing with simple mpi code to try and figure out why running with multiple tasks doesn't seem to be working.
 
S

s9862323@m98_nthu_edu_tw

New Member
Hi,

I found I cannot just run the executable of cam, or it will be wrong number of processes.
I changed to use the command " mpirun -n 6 ./cam " and it seems running successfully.
It produced a number of output files:
camrun.cam2.r.0000-01-02-00000.nc
camrun.cam2.rh0.0000-01-02-00000.nc
camrun.cam2.rs.0000-01-02-00000.nc
camrun.cice.r.0000-01-02-00000.nc
camrun.clm2.r.0000-01-02-00000.nc
camrun.clm2.rh0.0000-01-02-00000.nc
camrun.cpl.r.0000-01-02-00000.nc
camrun.docn.rs1.0000-01-02-00000.bin
ccsm_timing

Does this running correctly?

Also, I don't know how to open the NetCDF file. Should use some app to visualize the output?
Thank you very much.
 
hannay

hannay

Cecile Hannay
AMWG Liaison
Staff member
There are many tools to visualize netcdf files.
See: http://www.unidata.ucar.edu/software/netcdf/software.html

- You could start with "ncview" which is a quick and easy way to have a first look at your netCDF files.
See: http://meteora.ucsd.edu/~pierce/ncview_home_page.html

- For further analysis and visualization, you could use: NCL (http://www.ncl.ucar.edu/). It comes with many useful built-in functions and procedures for processing and manipulating data in netcdf (and many other formats).
 
S

s9862323@m98_nthu_edu_tw

New Member
Hi,

Thank you so much for the big help.

Now I use the command "./build-namelist -ntasks 32 -config ./config_cache.xml -runtype continue -namelist "&camexp stop_option='nmonths', stop_n=2 /" to generate some monthly data.
But I can only read the variable's name and value of camrun.cam2.h0.0000-02.nc

I have no idea what data I should use to generate graphs.
Could you please tell me how to analyze the output of CAM, what kind of info I can get from it.
Is there any document about the meaning and physical explanation of the output?
Because I have searched it for a long time, still don't know the meaning of it.

Also, could you tell me what is a complete inputdata?
Because the user's guide give me a svn website to download the inputdata, however, it doesn't separate it to many cases.
I don't know how to get ONLY the files I need for running just a case.

Thanks
Albert
 
S

s9862323@m98_nthu_edu_tw

New Member
Thanks for help.

I'd like to run AMWG diagnostics package , but I have no idea how to run it.
And the official website doesn't have the sample of running either.

Could you please give me some samples of how to use this package?

Thank you so much.
Albert
 
S

s9862323@m98_nthu_edu_tw

New Member
Thank you very much.
However, when I add the -test option to the build-namelist, it always gives me the same list of inputdata file.
Is there any ways that can solve this problem?

Thanks a lot.
Albert

---------------------------------
Writing CICE namelist to ./ice_in
Writing DOCN namelist to ./docn_ocn_in
Writing DOCN stream file to ./docn.stream.txt
Writing CLM namelist to ./lnd_in
Writing driver namelist to ./drv_in
Writing dry deposition namelist to ./drv_flds_in
Writing ocean component namelist to ./docn_in
Writing CAM namelist to ./atm_in
Checking whether input datasets exist locally...
OK -- found depvel_file = /home/albert7997/inputdata/atm/cam/chem/trop_mozart/dvel/depvel_monthly.nc
OK -- found tracer_cnst_filelist = /home/albert7997/inputdata/atm/cam/chem/trop_mozart_aero/oxid/oxid_1.9x2.5_L26_clim_list.c090805.txt
OK -- found tracer_cnst_datapath = /home/albert7997/inputdata/atm/cam/chem/trop_mozart_aero/oxid
OK -- found depvel_lnd_file = /home/albert7997/inputdata/atm/cam/chem/trop_mozart/dvel/regrid_vegetation.nc
OK -- found xs_long_file = /home/albert7997/inputdata/atm/waccm/phot/temp_prs_GT200nm_jpl06_c080930.nc
OK -- found rsf_file = /home/albert7997/inputdata/atm/waccm/phot/RSF_GT200nm_v3.0_c080416.nc
OK -- found clim_soilw_file = /home/albert7997/inputdata/atm/cam/chem/trop_mozart/dvel/clim_soilw.nc
OK -- found exo_coldens_file = /home/albert7997/inputdata/atm/cam/chem/trop_mozart/phot/exo_coldens.nc
OK -- found tracer_cnst_file = /home/albert7997/inputdata/atm/cam/chem/trop_mozart_aero/oxid/oxid_1.9x2.5_L26_1850-2005_c091123.nc
OK -- found season_wes_file = /home/albert7997/inputdata/atm/cam/chem/trop_mozart/dvel/season_wes.nc
OK -- found solar_data_file = /home/albert7997/inputdata/atm/cam/solar/solar_ave_sc19-sc23.c090810.nc
OK -- found soil_erod = /home/albert7997/inputdata/atm/cam/dst/dst_10x15_c090203.nc
OK -- found ncdata = /home/albert7997/inputdata/atm/cam/inic/fv/cami_0000-01-01_10x15_L30_c081013.nc
OK -- found bnd_topo = /home/albert7997/inputdata/atm/cam/topo/USGS-gtopo30_10x15_remap_c050520.nc
OK -- found bndtvs = /home/albert7997/inputdata/atm/cam/sst/sst_HadOIBl_bc_10x15_clim_c050526.nc
OK -- found focndomain = /home/albert7997/inputdata/atm/cam/ocnfrac/domain.camocn.10x15_USGS_070807.nc
OK -- found tropopause_climo_file = /home/albert7997/inputdata/atm/cam/chem/trop_mozart/ub/clim_p_trop.nc
OK -- found fpftcon = /home/albert7997/inputdata/lnd/clm2/pftdata/pft-physiology.c110425.nc
OK -- found fsnowaging = /home/albert7997/inputdata/lnd/clm2/snicardata/snicar_drdt_bst_fit_60_c070416.nc
OK -- found fatmlndfrc = /home/albert7997/inputdata/lnd/clm2/griddata/fracdata_10x15_USGS_070110.nc
OK -- found fsnowoptics = /home/albert7997/inputdata/lnd/clm2/snicardata/snicar_optics_5bnd_c090915.nc
OK -- found fsurdat = /home/albert7997/inputdata/lnd/clm2/surfdata/surfdata_10x15_simyr2000_c090928.nc
OK -- found fatmgrid = /home/albert7997/inputdata/lnd/clm2/griddata/griddata_10x15_070212.nc
OK -- found prescribed_ozone_datapath = /home/albert7997/inputdata/atm/cam/ozone
OK -- found prescribed_ozone_file = /home/albert7997/inputdata/atm/cam/ozone/ozone_1.9x2.5_L26_2000clim_c091112.nc
OK -- found liqopticsfile = /home/albert7997/inputdata/atm/cam/physprops/F_nwvl200_mu20_lam50_res64_t298_c080428.nc
OK -- found iceopticsfile = /home/albert7997/inputdata/atm/cam/physprops/iceoptics_c080917.nc
OK -- found water_refindex_file = /home/albert7997/inputdata/atm/cam/physprops/water_refindex_rrtmg_c080910.nc
OK -- found modal_optics_file = /home/albert7997/inputdata/atm/cam/physprops/modal_optics_3mode_c100507.nc
OK -- found ext_frc_specifier for SO2 -> /home/albert7997/inputdata/atm/cam/chem/trop_mozart_aero/emis/ar5_mam3_so2_elev_2000_c090726.nc
OK -- found ext_frc_specifier for bc_a1 -> /home/albert7997/inputdata/atm/cam/chem/trop_mozart_aero/emis/ar5_mam3_bc_elev_2000_c090726.nc
OK -- found ext_frc_specifier for num_a1 -> /home/albert7997/inputdata/atm/cam/chem/trop_mozart_aero/emis/ar5_mam3_num_a1_elev_2000_c090726.nc
OK -- found ext_frc_specifier for num_a2 -> /home/albert7997/inputdata/atm/cam/chem/trop_mozart_aero/emis/ar5_mam3_num_a2_elev_2000_c090726.nc
OK -- found ext_frc_specifier for pom_a1 -> /home/albert7997/inputdata/atm/cam/chem/trop_mozart_aero/emis/ar5_mam3_oc_elev_2000_c090726.nc
OK -- found ext_frc_specifier for so4_a1 -> /home/albert7997/inputdata/atm/cam/chem/trop_mozart_aero/emis/ar5_mam3_so4_a1_elev_2000_c090726.nc
OK -- found ext_frc_specifier for so4_a2 -> /home/albert7997/inputdata/atm/cam/chem/trop_mozart_aero/emis/ar5_mam3_so4_a2_elev_2000_c090726.nc
OK -- found srf_emis_specifier for DMS -> /home/albert7997/inputdata/atm/cam/chem/trop_mozart_aero/emis/aerocom_mam3_dms_surf_2000_c090129.nc
OK -- found srf_emis_specifier for SO2 -> /home/albert7997/inputdata/atm/cam/chem/trop_mozart_aero/emis/ar5_mam3_so2_surf_2000_c090726.nc
OK -- found srf_emis_specifier for SOAG -> /home/albert7997/inputdata/atm/cam/chem/trop_mozart_aero/emis/ar5_mam3_soag_1.5_surf_2000_c100217.nc
OK -- found srf_emis_specifier for bc_a1 -> /home/albert7997/inputdata/atm/cam/chem/trop_mozart_aero/emis/ar5_mam3_bc_surf_2000_c090726.nc
OK -- found srf_emis_specifier for num_a1 -> /home/albert7997/inputdata/atm/cam/chem/trop_mozart_aero/emis/ar5_mam3_num_a1_surf_2000_c090726.nc
OK -- found srf_emis_specifier for num_a2 -> /home/albert7997/inputdata/atm/cam/chem/trop_mozart_aero/emis/ar5_mam3_num_a2_surf_2000_c090726.nc
OK -- found srf_emis_specifier for pom_a1 -> /home/albert7997/inputdata/atm/cam/chem/trop_mozart_aero/emis/ar5_mam3_oc_surf_2000_c090726.nc
OK -- found srf_emis_specifier for so4_a1 -> /home/albert7997/inputdata/atm/cam/chem/trop_mozart_aero/emis/ar5_mam3_so4_a1_surf_2000_c090726.nc
OK -- found srf_emis_specifier for so4_a2 -> /home/albert7997/inputdata/atm/cam/chem/trop_mozart_aero/emis/ar5_mam3_so4_a2_surf_2000_c090726.nc
OK -- found rad_climate for P_so4_a1:/home/albert7997/inputdata/atm/cam/physprops/sulfate_rrtmg_c080918.nc
OK -- found rad_climate for P_pom_a1:/home/albert7997/inputdata/atm/cam/physprops/ocpho_rrtmg_c101112.nc
OK -- found rad_climate for P_soa_a1:/home/albert7997/inputdata/atm/cam/physprops/ocphi_rrtmg_c100508.nc
OK -- found rad_climate for P_bc_a1:/home/albert7997/inputdata/atm/cam/physprops/bcpho_rrtmg_c100508.nc
OK -- found rad_climate for P_dst_a1:/home/albert7997/inputdata/atm/cam/physprops/dust4_rrtmg_c090521.nc
OK -- found rad_climate for P_ncl_a1:/home/albert7997/inputdata/atm/cam/physprops/ssam_rrtmg_c100508.nc
OK -- found rad_climate for P_so4_a2:/home/albert7997/inputdata/atm/cam/physprops/sulfate_rrtmg_c080918.nc
OK -- found rad_climate for P_soa_a2:/home/albert7997/inputdata/atm/cam/physprops/ocphi_rrtmg_c100508.nc
OK -- found rad_climate for P_ncl_a2:/home/albert7997/inputdata/atm/cam/physprops/ssam_rrtmg_c100508.nc
OK -- found rad_climate for P_dst_a3:/home/albert7997/inputdata/atm/cam/physprops/dust4_rrtmg_c090521.nc
OK -- found rad_climate for P_ncl_a3:/home/albert7997/inputdata/atm/cam/physprops/ssam_rrtmg_c100508.nc
OK -- found rad_climate for P_so4_a3:/home/albert7997/inputdata/atm/cam/physprops/sulfate_rrtmg_c080918.nc
---------------------------------

eaton said:
You can look at the CAM home page to see the standard diagnostics http://www.cesm.ucar.edu/models/cesm1.0/cam/

If you add the -test option to the build-namelist command it will generate the list of inputdata files that are needed.
Click to expand...
 
S

s9862323@m98_nthu_edu_tw

New Member
I have another question that I cannot use more than 48 cores to run the input datasets.
I get the required input data files according to the -test flag.(the above reply)

I use the command
./build-namelist -test -ntasks 64 -config ./config_cache.xml -namelist "&camexp stop_option='nyears', stop_n=1 /"
mpirun --machinefile ./hostfile -np 64 ./cam
and I got the following error messages:
---------------------------
......
ENDRUN:SPMDINIT_DYN: less than 3 xy-longitudes per subdomain
nlat_p( 1 ) = 5
nlat_p( 2 ) = 5
nlat_p( 3 ) = 5
ENDRUN:SPMDINIT_DYN: less than 3 xy-longitudes per subdomain
ENDRUN:SPMDINIT_DYN: less than 3 xy-longitudes per subdomain
ENDRUN:SPMDINIT_DYN: less than 3 xy-longitudes per subdomain
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 1
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 2
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 3
......
---------------------------
 
S

s9862323@m98_nthu_edu_tw

New Member
I use the following commands to build cam:

export INC_NETCDF=$HOME/NetCDF-4.1.1-PGI/include
export LIB_NETCDF=$HOME/NetCDF-4.1.1-PGI/lib
export MOD_NETCDF=$HOME/NetCDF-4.1.1-PGI/include
export INC_MPI=$HOME/MVAPICH2-1.6-PGI/include
export LIB_MPI=$HOME/MVAPICH2-1.6-PGI/lib
./configure -dyn fv -hgrid 10x15 -ntasks 6 -nosmp -cc pgcc -fc pgf95 -fc_type pgi
 
E

eaton

CSEG and Liaisons
You can't change the value of -ntasks between the configure and build-namelist commands. If you try to use "-ntasks 64" in the configure command you'll get an error from the cice decomposition script because the grid is too coarse for 64 tasks.
 
E

eaton

CSEG and Liaisons
When you supply the -ntasks argument to build-namelist the variable npr_yz is automatically set (look in atm_in). Using "-ntasks 64" with the 10x15 grid produces npr_yz=4,16,16,4. This is a bug in the algorithm that is setting npr_yz. It should just fail because the maximum number of tasks that can be used with the 10x15 grid is 48 (npr_yz=6,8,8,6). The reason, as explained in the namelist documentation for npr_yz, is that each x (longitude) and each y (latitude) subdomain must contain 3 or more grid points. With 19 latitudes and 24 longitudes that means there can be a maximum of 6 y-subdomains and 8 x-subdomains.
 
S

s9862323@m98_nthu_edu_tw

New Member
Thank you for the big help.

Now I change to use -hgrid 4x5, and encounter other problems.
I want to run all the cases predefined in cesm1_0_4-pgi-4x5/models/atm/cam/bld/namelist_files/use_cases, but near all of them I cannot run it successfully.

I got many this kind of errors when running the build_namelist for some cases: 1850-2005_cam4_bgc, 1850-2005_cam4_super_fast_llnl, 1850_cam4_super_fast_llnl, 2000_cam4_trop_chem, 2005-2100_cam4_rcp26...
----------------------
CAM build-namelist - ERROR: No default value found for ncdata
user defined attributes:
key=ic_ymd val=18500101
----------------------
I found a maybe similar question on http://forum.cgd.ucar.edu/showthread.php?1791-Problem-running-CAM5-on-128x256-eu,
and I think maybe I should change some value in namelist_files/namelist_defaults_cam.xml
However, I don't know how to make it correct.


I also got a lot of errors when running mpirun for many cases: 1850_cam4_2xco2, 1850_cam4_trop_bam, 1850_cam4...
-------------------------------------
(GETFIL): attempting to find local file season_wes.nc
(GETFIL): using
/home/albert7997/inputdata/atm/cam/chem/trop_mozart/dvel/season_wes.nc
Opened existing file
/home/albert7997/inputdata/atm/cam/chem/trop_mozart/dvel/season_wes.nc
3014656
chemini: after drydep_inti on node 0
chemini: after fstrat_inti on node 0
photo_inti: solar_data file needs to contain irradiance spectrum
photo_inti: solar_data file needs to contain irradiance spectrum
photo_inti: solar_data file needs to contain irradiance spectrum
photo_inti: solar_data file needs to contain irradiance spectrum
ENDRUN:photo_inti: ERROR -- solar irradiance spectrum is missing
---------------------------

Could anyone please help me to solve these problems???
Thanks a lot.

Albert
 
E

eaton

CSEG and Liaisons
If there is no default initial file for the exact start date/time of your simulation, it is often sufficient to use the "-ignore_ic_year" option to build-namelist which will allow it to find an initial file with the correct month/day starting time, but will ignore the year.

The use case option is generally meant to be used with the CESM scripts. Using them with CAM standalone scripts requires expert knowledge of how to appropriately configure CAM for the particular use case. You haven't specified how you configured CAM, so there is no way to know what might be causing the error.
 
S

s9862323@m98_nthu_edu_tw

New Member
eaton said:
If there is no default initial file for the exact start date/time of your simulation, it is often sufficient to use the "-ignore_ic_year" option to build-namelist which will allow it to find an initial file with the correct month/day starting time, but will ignore the year.

The use case option is generally meant to be used with the CESM scripts. Using them with CAM standalone scripts requires expert knowledge of how to appropriately configure CAM for the particular use case. You haven't specified how you configured CAM, so there is no way to know what might be causing the error.
Click to expand...


Hi Eaton,

Thank you very much.
I only use the simple command to configure CAM (according to CAM5 user's guide).
-----------
export INC_NETCDF=$HOME/NetCDF-4.1.1-pgi/include
export LIB_NETCDF=$HOME/NetCDF-4.1.1-pgi/lib
export MOD_NETCDF=$HOME/NetCDF-4.1.1-pgi/include
export INC_MPI=$HOME/MVAPICH2-1.6-pgi/include
export LIB_MPI=$HOME/MVAPICH2-1.6-pgi/lib
cd cesm1_0_4-pgi-1.9x2.5/models/atm/cam/bld
./configure -dyn fv -hgrid 4x5 -ntasks 12 -nosmp -cc pgcc -fc pgf95 -fc_type pgi
-----------
Because I major in computer science not atmosphere science, it's difficult for me to understand all the flag which can be used to configure.
I try very hard to ask people about CAM, but still do not understand it very well.
Sorry to have so many questions.
 
E

eaton

CSEG and Liaisons
The configure command you provided will result in a CAM executable which uses the cam5 physics package (default) and that implies the trop_mam3 chemistry package. This build is inconsistent with all the use cases mentioned in your previous post (they all included a cam4 specifier). A correct namelist will be generated by build-namelist if you don't add an inconsistent use case argument. The default namelist will set up a present day climatological run.
 
S

s9862323@m98_nthu_edu_tw

New Member
eaton said:
The configure command you provided will result in a CAM executable which uses the cam5 physics package (default) and that implies the trop_mam3 chemistry package. This build is inconsistent with all the use cases mentioned in your previous post (they all included a cam4 specifier). A correct namelist will be generated by build-namelist if you don't add an inconsistent use case argument. The default namelist will set up a present day climatological run.
Click to expand...

Thank you for your reply.
I tried to use different flags when configuring each use_case, and I can run lots of cases successfully now.
However, I still don't know how to run some cases successfully.

below are the cases I can't run successfully:
==================================
1850-2005_cam4_bgc
2005-2100_cam4_rcp26
aquaplanet_cam4 (ERROR: CAM not configured for aquaplanet but user has set aqua_planet to TRUE in the namelist.)
cam4_bam_radpsv_geos5
cam4_chem_radpsv_geos5
cam4_trop_strat_chem
sd_waccm_geos5
sd_waccm_tslt_geos5
waccm_1974-2005_cam4
waccm_refb1
waccmx_2000_cam4

Can somebody help me with these problems?
Thank you so much.


BTW, I use the configure flags below to run these cases successfully.
------------
-phys cam4 -chem super_fast_llnl
1850-2005_cam4_super_fast_llnl, 1850_cam4_super_fast_llnl, 2000_cam4_super_fast_llnl

-phys cam4
1850-2005_cam4, 1850_cam4_2xco2, 1850_cam4, 2005_cam4, cam3

-phys cam4 -chem trop_bam
1850_cam4_trop_bam

-phys cam4 -chem trop_mozart
2000_cam4_trop_chem

-phys cam4 -chem waccm_mozart
waccm_1850-2005_cam4, waccm_1850_cam4, waccm_1955-2005_cam4, waccm_2000_cam4, waccm_2005-2100_cam4_rcp26

-phys cam4 -chem waccm_ghg
waccm_sc_2000_cam4
 
You must log in or register to reply here.
Top