Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

error running CAM4/CAM5 in E compset

Hi,

I was hoping somebody could help with the following problem. We have installed CESM1.0.4 on a new machine and certain compsets do not seem to run. F and some of the B compsets run fine, however none of the E ones do (with CAM4 or CAM5). Resolution is 1.9x2.5_1.9x2.5. Error messages look very much the same in different failed compset cases (E and B). Perhaps somebody else has encountered this before? I would be very grateful for any advice you may think of.
Our machine is a Linux cluster, 16 pes/node, PBS batch system and we use openmpi-1.6.2_intel13.

E1850_CAM5CN.o917:

CCSM BUILDNML SCRIPT STARTING
- Create modelio namelist input files
CCSM BUILDNML SCRIPT HAS FINISHED SUCCESSFULLY
-------------------------------------------------------------------------
CCSM PRESTAGE SCRIPT STARTING
- CCSM input data directory, DIN_LOC_ROOT_CSMDATA, is /data/ra/cesm-inputdata
- Case input data directory, DIN_LOC_ROOT, is /data/ra/cesm-inputdata
- Checking the existence of input datasets in DIN_LOC_ROOT
CCSM PRESTAGE SCRIPT HAS FINISHED SUCCESSFULLY
Fri Nov 9 15:39:19 PST 2012 -- CSM EXECUTION BEGINS HERE
Fri Nov 9 15:41:49 PST 2012 -- CSM EXECUTION HAS FINISHED
Model did not complete - see /data/ra/icv/E1850_CAM5CN/run/cpl.log.121109-153853

-----------------------------------------------------
cpl. log* says Model initialization complete
-----------------------------------------------------


ccsm.log.* :

...

Reading setup_nml
Reading grid_nml
Reading ice_nml
Reading tracer_nml
CalcWorkPerBlock: Total blocks: 128 Ice blocks: 128 IceFree blocks: 0 Land blocks: 0
MCT::m_Router::initp_: GSMap indices not increasing...Will correct
MCT::m_Router::initp_: RGSMap indices not increasing...Will correct
MCT::m_Router::initp_: RGSMap indices not increasing...Will correct
MCT::m_Router::initp_: GSMap indices not increasing...Will correct
(seq_frac_check) [lnd init] afrac min/max = 1.000000000000000000 1.000000000000000000
(seq_frac_check) [lnd init] lfrac min/max = 0.573489812125174137E-01 1.000000000000000000
(seq_frac_check) [lnd init] lfrin min/max = 0.573489812125174137E-01 1.000000000000000000
(seq_frac_check) [ice init] afrac min/max = 1.000000000000000000 1.000000000000000000
(seq_frac_check) [ice init] ofrac min/max = 0.00000000000000000 1.000000000000000000
(seq_frac_check) [ice init] ifrac min/max = 0.00000000000000000 0.0000000000000000

...

calcsize j,iq,jac, lsfrm,lstoo 2 3 2 13 19
calcsize j,iq,jac, lsfrm,lstoo 2 4 1 16 20
calcsize j,iq,jac, lsfrm,lstoo 2 4 2 16 20
forrtl: error (73): floating divide by zero
Image PC Routine Line Source
ccsm.exe 0000000002D57302 ice_dyn_evp_mp_st 1205 ice_dyn_evp.F90
ccsm.exe 0000000002D46247 ice_dyn_evp_mp_ev 380 ice_dyn_evp.F90
ccsm.exe 0000000002ECF0B3 ice_step_mod_mp_s 664 ice_step_mod.F90
ccsm.exe 0000000002D06EB4 ice_comp_mct_mp_i 554 ice_comp_mct.F90
ccsm.exe 00000000004E18CA ccsm_comp_mod_mp_ 1759 ccsm_comp_mod.F90
ccsm.exe 00000000004EB6AE MAIN__ 91 ccsm_driver.F90
ccsm.exe 00000000004CC20C Unknown Unknown Unknown
libc.so.6 000000347A61ECDD Unknown Unknown Unknown
ccsm.exe 00000000004CC109 Unknown Unknown Unknown
forrtl: error (65): floating invalid
Image PC Routine Line Source
ccsm.exe 0000000002D57302 ice_dyn_evp_mp_st 1205 ice_dyn_evp.F90
ccsm.exe 0000000002D46247 ice_dyn_evp_mp_ev 380 ice_dyn_evp.F90
ccsm.exe 0000000002ECF0B3 ice_step_mod_mp_s 664 ice_step_mod.F90
ccsm.exe 0000000002D06EB4 ice_comp_mct_mp_i 554 ice_comp_mct.F90
ccsm.exe 00000000004E18CA ccsm_comp_mod_mp_ 1759 ccsm_comp_mod.F90
ccsm.exe 00000000004EB6AE MAIN__ 91 ccsm_driver.F90
ccsm.exe 00000000004CC20C Unknown Unknown Unknown
libc.so.6 000000347A61ECDD Unknown Unknown Unknown
ccsm.exe 00000000004CC109 Unknown Unknown Unknown
...

forrtl: error (78): process killed (SIGTERM)
Image PC Routine Line Source
libpthread.so.0 00000032EE40C173 Unknown Unknown Unknown
libmlx4-rdmav2.so 00007F846EC7BA7B Unknown Unknown Unknown
mca_btl_openib.so 00007F84713904EF Unknown Unknown Unknown
libmpi.so.1 00007F84764AFC69 Unknown Unknown Unknown
libmpi.so.1 00007F84763E24F2 Unknown Unknown Unknown
libmpi.so.1 00007F8476409F04 Unknown Unknown Unknown
libmpi_f77.so.1 00007F8475F6EC45 Unknown Unknown Unknown
ccsm.exe 0000000003039D44 ice_boundary_mp_i 2747 ice_boundary.F90
ccsm.exe 0000000002D4719A ice_dyn_evp_mp_ev 434 ice_dyn_evp.F90
ccsm.exe 0000000002ECF0B3 ice_step_mod_mp_s 664 ice_step_mod.F90
ccsm.exe 0000000002D06EB4 ice_comp_mct_mp_i 554 ice_comp_mct.F90
ccsm.exe 00000000004E18CA ccsm_comp_mod_mp_ 1759 ccsm_comp_mod.F90
ccsm.exe 00000000004EB6AE MAIN__ 91 ccsm_driver.F90
ccsm.exe 00000000004CC20C Unknown Unknown Unknown
libc.so.6 00000032EDC1ECDD Unknown Unknown Unknown
ccsm.exe 00000000004CC109 Unknown Unknown Unknown
------------------------------------------
 
Could somebody please address my post?

user012 said:
Hi,

I was hoping somebody could help with the following problem. We have installed CESM1.0.4 on a new machine and certain compsets do not seem to run. F and some of the B compsets run fine, however none of the E ones do (with CAM4 or CAM5). Resolution is 1.9x2.5_1.9x2.5. Error messages look very much the same in different failed compset cases (E and B). Perhaps somebody else has encountered this before? I would be very grateful for any advice you may think of.
Our machine is a Linux cluster, 16 pes/node, PBS batch system and we use openmpi-1.6.2_intel13.

E1850_CAM5CN.o917:

CCSM BUILDNML SCRIPT STARTING
- Create modelio namelist input files
CCSM BUILDNML SCRIPT HAS FINISHED SUCCESSFULLY
-------------------------------------------------------------------------
CCSM PRESTAGE SCRIPT STARTING
- CCSM input data directory, DIN_LOC_ROOT_CSMDATA, is /data/ra/cesm-inputdata
- Case input data directory, DIN_LOC_ROOT, is /data/ra/cesm-inputdata
- Checking the existence of input datasets in DIN_LOC_ROOT
CCSM PRESTAGE SCRIPT HAS FINISHED SUCCESSFULLY
Fri Nov 9 15:39:19 PST 2012 -- CSM EXECUTION BEGINS HERE
Fri Nov 9 15:41:49 PST 2012 -- CSM EXECUTION HAS FINISHED
Model did not complete - see /data/ra/icv/E1850_CAM5CN/run/cpl.log.121109-153853

-----------------------------------------------------
cpl. log* says Model initialization complete
-----------------------------------------------------


ccsm.log.* :

...

Reading setup_nml
Reading grid_nml
Reading ice_nml
Reading tracer_nml
CalcWorkPerBlock: Total blocks: 128 Ice blocks: 128 IceFree blocks: 0 Land blocks: 0
MCT::m_Router::initp_: GSMap indices not increasing...Will correct
MCT::m_Router::initp_: RGSMap indices not increasing...Will correct
MCT::m_Router::initp_: RGSMap indices not increasing...Will correct
MCT::m_Router::initp_: GSMap indices not increasing...Will correct
(seq_frac_check) [lnd init] afrac min/max = 1.000000000000000000 1.000000000000000000
(seq_frac_check) [lnd init] lfrac min/max = 0.573489812125174137E-01 1.000000000000000000
(seq_frac_check) [lnd init] lfrin min/max = 0.573489812125174137E-01 1.000000000000000000
(seq_frac_check) [ice init] afrac min/max = 1.000000000000000000 1.000000000000000000
(seq_frac_check) [ice init] ofrac min/max = 0.00000000000000000 1.000000000000000000
(seq_frac_check) [ice init] ifrac min/max = 0.00000000000000000 0.0000000000000000

...

calcsize j,iq,jac, lsfrm,lstoo 2 3 2 13 19
calcsize j,iq,jac, lsfrm,lstoo 2 4 1 16 20
calcsize j,iq,jac, lsfrm,lstoo 2 4 2 16 20
forrtl: error (73): floating divide by zero
Image PC Routine Line Source
ccsm.exe 0000000002D57302 ice_dyn_evp_mp_st 1205 ice_dyn_evp.F90
ccsm.exe 0000000002D46247 ice_dyn_evp_mp_ev 380 ice_dyn_evp.F90
ccsm.exe 0000000002ECF0B3 ice_step_mod_mp_s 664 ice_step_mod.F90
ccsm.exe 0000000002D06EB4 ice_comp_mct_mp_i 554 ice_comp_mct.F90
ccsm.exe 00000000004E18CA ccsm_comp_mod_mp_ 1759 ccsm_comp_mod.F90
ccsm.exe 00000000004EB6AE MAIN__ 91 ccsm_driver.F90
ccsm.exe 00000000004CC20C Unknown Unknown Unknown
libc.so.6 000000347A61ECDD Unknown Unknown Unknown
ccsm.exe 00000000004CC109 Unknown Unknown Unknown
forrtl: error (65): floating invalid
Image PC Routine Line Source
ccsm.exe 0000000002D57302 ice_dyn_evp_mp_st 1205 ice_dyn_evp.F90
ccsm.exe 0000000002D46247 ice_dyn_evp_mp_ev 380 ice_dyn_evp.F90
ccsm.exe 0000000002ECF0B3 ice_step_mod_mp_s 664 ice_step_mod.F90
ccsm.exe 0000000002D06EB4 ice_comp_mct_mp_i 554 ice_comp_mct.F90
ccsm.exe 00000000004E18CA ccsm_comp_mod_mp_ 1759 ccsm_comp_mod.F90
ccsm.exe 00000000004EB6AE MAIN__ 91 ccsm_driver.F90
ccsm.exe 00000000004CC20C Unknown Unknown Unknown
libc.so.6 000000347A61ECDD Unknown Unknown Unknown
ccsm.exe 00000000004CC109 Unknown Unknown Unknown
...

forrtl: error (78): process killed (SIGTERM)
Image PC Routine Line Source
libpthread.so.0 00000032EE40C173 Unknown Unknown Unknown
libmlx4-rdmav2.so 00007F846EC7BA7B Unknown Unknown Unknown
mca_btl_openib.so 00007F84713904EF Unknown Unknown Unknown
libmpi.so.1 00007F84764AFC69 Unknown Unknown Unknown
libmpi.so.1 00007F84763E24F2 Unknown Unknown Unknown
libmpi.so.1 00007F8476409F04 Unknown Unknown Unknown
libmpi_f77.so.1 00007F8475F6EC45 Unknown Unknown Unknown
ccsm.exe 0000000003039D44 ice_boundary_mp_i 2747 ice_boundary.F90
ccsm.exe 0000000002D4719A ice_dyn_evp_mp_ev 434 ice_dyn_evp.F90
ccsm.exe 0000000002ECF0B3 ice_step_mod_mp_s 664 ice_step_mod.F90
ccsm.exe 0000000002D06EB4 ice_comp_mct_mp_i 554 ice_comp_mct.F90
ccsm.exe 00000000004E18CA ccsm_comp_mod_mp_ 1759 ccsm_comp_mod.F90
ccsm.exe 00000000004EB6AE MAIN__ 91 ccsm_driver.F90
ccsm.exe 00000000004CC20C Unknown Unknown Unknown
libc.so.6 00000032EDC1ECDD Unknown Unknown Unknown
ccsm.exe 00000000004CC109 Unknown Unknown Unknown
------------------------------------------
 

hannay

Cecile Hannay
AMWG Liaison
Staff member
The error is:
error (73): floating divide by zero

It sounds like you are trying to run on the components on the atmospheric grid.

Try to run with ocean and ice on their grid: f09_g16 for instance.
 
Thank you very much. Yes, the resolution seems to be a part of the problem. Before seeing your response I tried 1.9x2.5_gx1v6 and this allowed to run the model for a month before encountering the problems again. I will try now the resolution you have suggested but I am worried that there may be more to it.
This is what I got with 1.9x2.5_gx1v6:

ccsmlog:

(seq_frac_check) [atm init] sum ncnt/maxerr = 0 0.00000000000000000
Opened file iv_E1850CNor.cam2.h1.0001-01-01-00000.nc to write 29
BalanceCheck: soil balance error nstep = 105 point = 2333 imbalance = -0.000002 W/m2
BalanceCheck: soil balance error nstep = 106 point = 2333 imbalance = -0.000002 W/m2
QNEG4 WARNING from TPHYSAC Max possible LH flx exceeded at 1 points. , Worst excess = -1.4062E-06, lchnk = 97, i = 5, same as indices lat = 4, lon = 61
.....


BalanceCheck: soil balance error nstep = 4998 point = 6537 imbalance = -0.000006 W/m2
mpiexec: killing job...

mpiexec: abort is already in progress...hit ctrl-c again to forcibly terminate

forrtl: error (78): process killed (SIGTERM)
Image PC Routine Line Source
libmlx4-rdmav2.so 00002B78AC98EAD5 Unknown Unknown Unknown
mca_btl_openib.so 00002B78AAC514EF Unknown Unknown Unknown
libmpi.so.1 00002B78A5B9DC69 Unknown Unknown Unknown
libmpi.so.1 00002B78A5AD04F2 Unknown Unknown Unknown
libmpi.so.1 00002B78A5AF7F04 Unknown Unknown Unknown
libmpi_f77.so.1 00002B78A6099C45 Unknown Unknown Unknown
ccsm.exe 00000000010F8B81 ice_boundary_mp_i 2747 ice_boundary.F90
ccsm.exe 0000000000FE2FB6 ice_dyn_evp_mp_ev 434 ice_dyn_evp.F90
ccsm.exe 000000000107AF5B ice_step_mod_mp_s 664 ice_step_mod.F90
ccsm.exe 0000000000FBF4E6 ice_comp_mct_mp_i 554 ice_comp_mct.F90
ccsm.exe 00000000004B9493 ccsm_comp_mod_mp_ 1759 ccsm_comp_mod.F90
ccsm.exe 00000000004CBD91 MAIN__ 91 ccsm_driver.F90
ccsm.exe 00000000004B7C0C Unknown Unknown Unknown
libc.so.6 0000003C8D41ECDD Unknown Unknown Unknown
ccsm.exe 00000000004B7B09 Unknown Unknown Unknown
 
Hello user012

I too went through similar problems. First i configure and build E1850CN with f19_f19, it was not running. Then I tried with f19_g16 and now its running. But again similar errors are showing up in ccsmlog(part of it is shown below). Also, the run is very slow.

I checked the ccsm_timing.$CASE.$datestamp which shows:

Overall Metrics:
Model Cost: 9440.31 pe-hrs/simulated_year (scale= 1.00)
Model Throughput: 0.15 simulated_years/day

I hope you have already solved the problem. Please help me out of this issue.


ccsmlog:

QNEG4 WARNING from TPHYSAC Max possible LH flx exceeded at 1 points. , Worst excess = -9.5149E-06, lchnk = 774, i = 13, same as indices lat = 84, lon = 30
QNEG4 WARNING from TPHYSAC Max possible LH flx exceeded at 1 points. , Worst excess = -1.2209E-06, lchnk = 775, i = 13, same as indices lat = 84, lon = 31
QNEG4 WARNING from TPHYSAC Max possible LH flx exceeded at 1 points. , Worst excess = -1.7625E-06, lchnk = 776, i = 13, same as indices lat = 84, lon = 32
QNEG4 WARNING from TPHYSAC Max possible LH flx exceeded at 1 points. , Worst excess = -3.5511E-07, lchnk = 810, i = 1, same as indices lat = 85, lon = 30
QNEG4 WARNING from TPHYSAC Max possible LH flx exceeded at 2 points. , Worst excess = -1.7197E-06, lchnk = 777, i = 9, same as indices lat = 83, lon = 33
QNEG4 WARNING from TPHYSAC Max possible LH flx exceeded at 1 points. , Worst excess = -5.7334E-06, lchnk = 811, i = 1, same as indices lat = 85, lon = 31
QNEG4 WARNING from TPHYSAC Max possible LH flx exceeded at 2 points. , Worst excess = -1.1129E-06, lchnk = 778, i = 13, same as indices lat = 84, lon = 34
QNEG4 WARNING from TPHYSAC Max possible LH flx exceeded at 1 points. , Worst excess = -2.0332E-06, lchnk = 812, i = 1, same as indices lat = 85, lon = 32
QNEG4 WARNING from TPHYSAC Max possible LH flx exceeded at 2 points. , Worst excess = -9.4973E-07, lchnk = 779, i = 9, same as indices lat = 83, lon = 35
QNEG4 WARNING from TPHYSAC Max possible LH flx exceeded at 2 points. , Worst excess = -2.5846E-06, lchnk = 813, i = 1, same as indices lat = 85, lon = 33
QNEG4 WARNING from TPHYSAC Max possible LH flx exceeded at 2 points. , Worst excess = -2.1007E-06, lchnk = 814, i = 1, same as indices lat = 85, lon = 34

...

QNEG3 from TPHYSBCb:m= 5 lat/lchnk= 529 Min. mixing ratio violated at 1 points. Reset to 0.0E+00 Worst =-4.7E-10 at i,k= 13 24
filew failed, worst i, j, qtmp, q = 1 61
-2.740070676559263E-015 0.000000000000000E+000
BalanceCheck: soil balance error nstep = 1231 point = 7919 imbalance = -0.000008 W/m2
BalanceCheck: soil balance error nstep = 1232 point = 7919 imbalance = -0.000008 W/m2
BalanceCheck: soil balance error nstep = 1251 point = 7429 imbalance = -0.000001 W/m2
BalanceCheck: soil balance error nstep = 1252 point = 7429 imbalance = -0.000001 W/m2
QNEG3 from vertical diffusion/SO2:m= 8 lat/lchnk= 805 Min. mixing ratio violated at 2 points. Reset to 1.0E-36 Worst =-1.0E-12 at i,k= 1 30
QNEG3 from convtran2/SOAG:m= 10 lat/lchnk= 399 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-8.2E-11 at i,k= 16 30
BalanceCheck: soil balance error nstep = 1395 point = 7428 imbalance = -0.000002 W/m2
BalanceCheck: soil balance error nstep = 1396 point = 7428 imbalance = -0.000002 W/m2
BalanceCheck: soil balance error nstep = 1403 point = 11680 imbalance = -0.000003 W/m2
BalanceCheck: soil balance error nstep = 1404 point = 11680 imbalance = -0.000003 W/m2
BalanceCheck: soil balance error nstep = 1405 point = 11680 imbalance = -0.000002 W/m2
BalanceCheck: soil balance error nstep = 1405 point = 8152 imbalance = 0.000000 W/m2
BalanceCheck: soil balance error nstep = 1406 point = 11680 imbalance = -0.000002 W/m2
BalanceCheck: soil balance error nstep = 1406 point = 8152 imbalance = 0.000000 W/m2
Opened file ./somcam5.clm2.h0.0001-01.nc to write 63
Opened file somcam5.cam2.h0.0001-01.nc to write 63
BalanceCheck: soil balance error nstep = 1499 point = 11677 imbalance = 0.000000 W/m2
BalanceCheck: soil balance error nstep = 1500 point = 11677 imbalance = 0.000000 W/m2
filew failed, worst i, j, qtmp, q = 1 35
-1.403648097160563E-015 0.000000000000000E+000
dpcoup cant adjust 3 313 9 -9.243279892192106E-019
0.000000000000000E+000 7.796021544792771E-019
filew failed, worst i, j, qtmp, q = 1 60
-1.726614084843839E-011 0.000000000000000E+000
filew failed, worst i, j, qtmp, q = 1 60
-1.207380182004439E-002 7.570681531833429E-009
QNEG3 from TPHYSBCb:m= 5 lat/lchnk= 529 Min. mixing ratio violated at 2 points. Reset to 0.0E+00 Worst =-1.6E-06 at i,k= 13 21
BalanceCheck: soil balance error nstep = 1691 point = 11684 imbalance = -0.000001 W/m2
BalanceCheck: soil balance error nstep = 1692 point = 11684 imbalance = -0.000001 W/m2
BalanceCheck: soil balance error nstep = 1737 point = 11684 imbalance = -0.000001 W/m2
BalanceCheck: soil balance error nstep = 1738 point = 11684 imbalance = -0.000001 W/m2
BalanceCheck: soil balance error nstep = 1739 point = 4869 imbalance = 0.000000 W/m2
BalanceCheck: soil balance error nstep = 1740 point = 4869 imbalance = 0.000000 W/m2
BalanceCheck: soil balance error nstep = 1783 point = 5736 imbalance = -0.000001 W/m2
BalanceCheck: soil balance error nstep = 1784 point = 5736 imbalance = -0.000001 W/m2
BalanceCheck: soil balance error nstep = 1791 point = 5736 imbalance = -0.000006 W/m2
BalanceCheck: soil balance error nstep = 1792 point = 5736 imbalance = -0.000006 W/m2
QNEG3 from vertical diffusion/SO2:m= 8 lat/lchnk= 142 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-2.0E-12 at i,k= 8 30
BalanceCheck: soil balance error nstep = 1859 point = 7922 imbalance = 0.000000 W/m2
BalanceCheck: soil balance error nstep = 1860 point = 7922 imbalance = 0.000000 W/m2
BalanceCheck: soil balance error nstep = 1929 point = 11676 imbalance = 0.000000 W/m2
BalanceCheck: soil balance error nstep = 1930 point = 11676 imbalance = 0.000000 W/m2
BalanceCheck: soil balance error nstep = 1933 point = 1008 imbalance = -0.000003 W/m2
BalanceCheck: soil balance error nstep = 1934 point = 1008 imbalance = -0.000003 W/m2
BalanceCheck: soil balance error nstep = 1935 point = 1008 imbalance = -0.000002 W/m2
BalanceCheck: soil balance error nstep = 1936 point = 1008 imbalance = -0.000002 W/m2
BalanceCheck: soil balance error nstep = 1981 point = 11684 imbalance = 0.000000 W/m2
BalanceCheck: soil balance error nstep = 1982 point = 11684 imbalance = 0.000000 W/m2
BalanceCheck: soil balance error nstep = 2025 point = 8152 imbalance = 0.000000 W/m2
BalanceCheck: soil balance error nstep = 2026 point = 8152 imbalance = 0.000000 W/m2
QNEG3 from vertical diffusion/SO2:m= 8 lat/lchnk= 798 Min. mixing ratio violated at 2 points. Reset to 1.0E-36 Worst =-1.3E-12 at i,k= 9 30
QNEG3 from vertical diffusion/SO2:m= 8 lat/lchnk= 800 Min. mixing ratio violated at 1 points. Reset to 1.0E-36 Worst =-1.0E-12 at i,k= 1 30
filew failed, worst i, j, qtmp, q = 1 40
-1.604692839382339E-016 0.000000000000000E+000
BalanceCheck: soil balance error nstep = 2217 point = 11675 imbalance = -0.000004 W/m2
BalanceCheck: soil balance error nstep = 2218 point = 11675 imbalance = -0.000004 W/m2

Thanks
Angshuman
 
Top