Provided by: pcp_3.10.8build1_amd64 bug

NAME

       PCPIntro - introduction to the Performance Co-Pilot (PCP)

INTRODUCTION

       The  Performance  Co-Pilot (PCP) is a toolkit designed for monitoring and managing system-
       level performance.  These services are distributed and scalable to  accommodate  the  most
       complex system configurations and performance problems.

       PCP  supports  many  different  platforms,  including  (but not limited to) Linux, MacOSX,
       Solaris and Windows.  From a high-level PCP can be considered to contain  two  classes  of
       software utility:

       PCP Collectors
               These  are the parts of PCP that collect and extract performance data from various
               sources, e.g. the operating system kernel.

       PCP Monitors
               These are the parts of PCP that display data collected from  hosts  (or  archives)
               that  have  the PCP Collector installed.  Many monitor tools are available as part
               of the core PCP release, while other (typically graphical)  monitoring  tools  are
               available separately in the PCP GUI package.

       This  manual  entry  describes  the  high-level  features  and  options common to most PCP
       utilities available on all platforms.

OVERVIEW

       The PCP architecture is distributed in the sense  that  any  PCP  tool  may  be  executing
       remotely.   On  the  host  (or hosts) being monitored, each domain of performance metrics,
       whether the kernel, a service layer, a  database  management  system,  a  web  server,  an
       application,    etc.   requires  a  Performance  Metrics  Domain  Agent  (PMDA)  which  is
       responsible for collecting performance measurements  from  that  domain.   All  PMDAs  are
       controlled by the Performance Metrics Collector Daemon (pmcd(1)) on the same host.

       Client  applications (the monitoring tools) connect to pmcd(1), which acts as a router for
       requests, by forwarding requests to the appropriate PMDA and returning  the  responses  to
       the  clients.   Clients may also access performance data from a PCP archive (created using
       pmlogger(1)) for retrospective analysis.

   Security philosophy
       PCP redistributes a wealth of  performance  information  within  a  host  and  across  its
       networks.   The  following  security  philosophy underlies the setting of several defaults
       that control how much information is sent and received.

       By default, the information exposed by PMCD about a host  is  approximately  of  the  same
       level of confidentiality as available to a completely unprivileged user on that host.  So,
       performance data that is available to be read completely freely on a machine may  be  made
       available by PMCD to the network.

       However,  the  host  running  PMCD  and  its  network  is not assumed to run only friendly
       applications.  Therefore, write type operations, including from the local  host,  are  not
       permitted by default.

       These  defaults  may  be  overridden  (expanded  or reduced) in several ways, including by
       specifying network ACLs in pmcd.conf, activating  non-default  PMDAs,  or  by  using  PMCD
       connections  that  pass  user  credentials.  For example, some PMDAs automatically provide
       greater information for particular credentialed users or groups.

   Applications
       The following performance monitoring applications are primarily console  based,  typically
       run  directly from the command line, and are just a small subset of the tools available as
       part of the base PCP package.

       Each tool or command is documented completely in its own reference page.

       pmstat Outputs an ASCII high-level summary of system performance.

       pmie   An inference engine that can evaluate predicate-action rules to perform alarms  and
              automate system management tasks.

       pminfo Interrogate specific performance metrics and the metadata that describes them.

       pmlogger
              Generates  PCP  archives  of  performance  metrics  suitable for replay by most PCP
              tools.

       pmval  Simple periodic reporting for some or all instances of a performance  metric,  with
              optional VCR time control.

       If the PCP GUI package is installed then the following additional tools are available.

       pmchart
              Displays  trends  over time of arbitrarily selected performance metrics from one or
              more hosts.

       pmtime Time control utility for coordinating the time between  multiple  tools  (including
              pmchart and pmval).

       pmdumptext
              Produce ASCII reports for arbitrary combinations of performance metrics.

COMMON COMMAND LINE ARGUMENTS

       There  is  a  set  of common command line arguments that are used consistently by most PCP
       tools.

       -a archive
              Performance metric information is retrospectively retrieved  from  the  Performance
              Co-Pilot    (PCP)   archive,   previously   generated   by   pmlogger(1).See   pcp-
              archive(5)forformatdocumentation.  The -a and -h options are mutually exclusive.

              archive is either the base name common to all of the physical files created  by  an
              instance  of  pmlogger(1),  or any one of the physical files, e.g.  myarchive (base
              name) or myarchive.meta (the metadata file) or myarchive.index (the temporal index)
              or   myarchive.0   (the  first  data  volume  of  archive)  or  myarchive.0.bz2  or
              myarchive.0.bz (the first data volume compressed with bzip2(1))  or  myarchive.0.gz
              or  myarchive.0.Z or myarchive.0.z (the first data volume compressed with gzip(1)),
              myarchive.1 or myarchive.3.bz2 or myarchive.42.gz etc.

       -a archive[,archive,...]
              An alternate form of -a for applications that are able to handle multiple archives.

       -h hostname
              Unless directed to another host by the -h option,  or  to  an  archive  by  the  -a
              option, the source of performance metrics will be the Performance Metrics Collector
              Daemon (PMCD) on the local host.  Refer to  the  PMCD  HOST  SPECIFICATION  section
              later  for  further details on the many options available when forming the hostname
              specification, as well  as  a  detailed  description  of  the  default  local  host
              connection.  The -a and -h options are mutually exclusive.

       -s samples
              The  argument  samples  defines the number of samples to be retrieved and reported.
              If samples is 0 or -s is not specified, the  application  will  sample  and  report
              continuously  (in  real  time mode) or until the end of the PCP archive (in archive
              mode).

       -z     Change the reporting timezone to the local timezone at the host that is the  source
              of the performance metrics, as identified via either the -h or -a options.

       -Z timezone
              By  default, applications report the time of day according to the local timezone on
              the system where the application is executed.  The -Z option changes  the  timezone
              to  timezone  in  the  format  of  the  environment  variable  TZ  as  described in
              environ(7).

INTERVAL SPECIFICATION AND ALIGNMENT

       Most PCP tools operate with periodic sampling or reporting, and the -t and -A options  may
       be  used  to  control  the duration of the sample interval and the alignment of the sample
       times.

       -t interval
              Set the update or reporting interval.

              The interval argument is specified as a sequence of one or  more  elements  of  the
              form
                        number[units]
              where  number is an integer or floating point constant (parsed using strtod(3)) and
              the optional units is one of: seconds, second, secs, sec, s, minutes, minute, mins,
              min, m, hours, hour, h, days, day and d.  If the unit is empty, second is assumed.

              In  addition,  the  upper  case (or mixed case) version of any of the above is also
              acceptable.

              Spaces anywhere in the interval  are  ignored,  so  4  days  6  hours  30  minutes,
              4day6hour30min, 4d6h30m and 4d6.5h are all equivalent.

              Multiple  specifications  are additive, e.g. ``1hour 15mins 30secs'' is interpreted
              as 3600+900+30 seconds.

       -A align
              By default samples are not necessarily aligned on any natural unit of time.  The -A
              option  may  be used to force the initial sample to be aligned on the boundary of a
              natural time unit.  For example -A 1sec, -A 30min and -A 1hour specify alignment on
              whole seconds, half and whole hours respectively.

              The  align argument follows the syntax for an interval argument described above for
              the -t option.

              Note that alignment occurs by advancing the time as required, and that -A acts as a
              modifier  to  advance  both the start of the time window (see the next section) and
              the origin time (if the -O option is specified).

TIME WINDOW SPECIFICATION

       Many PCP tools are designed to operate in some time window of interest, e.g. to  define  a
       termination  time  for real-time monitoring or to define a start and end time within a PCP
       archive log.

       In the absence of the -O and -A options to specify an initial sample time origin and  time
       alignment  (see above), the PCP application will retrieve the first sample at the start of
       the time window.

       The following options may be used to specify a time window of interest.

       -S starttime
              By default the time window commences immediately in real-time  mode,  or  coincides
              with  time  at the start of the PCP archive log in archive mode.  The -S option may
              be used to specify a later time for the start of the time window.

              The starttime parameter may be given in one of three forms (interval is the same as
              for the -t option as described above, datetime is described below):

              interval
                     To  specify  an  offset  from  the  current  time (in real-time mode) or the
                     beginning of a PCP archive (in archive mode) simply specify the interval  of
                     time  as  the argument.  For example -S 30min will set the start of the time
                     window to be exactly 30 minutes from now in real-time mode,  or  exactly  30
                     minutes from the start of a PCP archive.

              -interval
                     To  specify an offset from the end of a PCP archive log, prefix the interval
                     argument with a minus sign.  In this case, the  start  of  the  time  window
                     precedes  the time at the end of archive by the given interval.  For example
                     -S -1hour will set the start of the time  window  to  be  exactly  one  hour
                     before the time of the last sample in a PCP archive log.

              @datetime
                     To specify the calendar date and time (local time in the reporting timezone)
                     for the start of the time window, use the datetime syntax preceded by an  at
                     sign.  Refer to the datetime description below for detailed information.

       -T endtime
              By  default  the end of the time window is unbounded (in real-time mode) or aligned
              with the time at the end of a PCP archive log (in archive mode).  The -T option may
              be used to specify an earlier time for the end of the time window.

              The  endtime  parameter may be given in one of three forms (interval is the same as
              for the -t option as described above, datetime is described below):

              interval
                     To specify an offset from the start  of  the  time  window  simply  use  the
                     interval  of time as the argument.  For example -T 2h30m will set the end of
                     the time window to be 2 hours and 30 minutes after the  start  of  the  time
                     window.

              -interval
                     To  specify  an  offset  back from the time at the end of a PCP archive log,
                     prefix the interval argument with a minus sign.  For example  -T  -90m  will
                     set  the end of the time window to be 90 minutes before the time of the last
                     sample in a PCP archive log.

              @datetime
                     To specify the calendar date and time (local time in the reporting timezone)
                     for  the  end  of the time window, use the datetime syntax preceded by an at
                     sign.  Refer to the datetime description below for detailed information.

       -O origin
              By default samples are fetched from the start of the time window  (see  description
              of -S option) to the end of the time window (see description of -T option).  The -O
              option allows the specification of an origin within the time window to be  used  as
              the initial sample time.  This is useful for interactive use of a PCP tool with the
              pmtime(1) VCR replay facility.

              The origin argument accepted by -O conforms to the same syntax and semantics as the
              starttime argument for the -T option.

              For  example  -O -0 specifies that the initial position should be at the end of the
              time window; this is most useful when wishing to replay  ``backwards''  within  the
              time window.

       The datetime argument for the -O, -S and -T options consists of:
                 date time zone day relative
       A  date can be one of: YY-MM-DD, MM/DD/YY, DD Month YYYY, or Month DD YYYY.  A time can be
       one of: HH:MM:SS, HH:MM.  HH:MM can use either the 12 hour (via an am or pm suffix) or  24
       hour  convention.   A  day  of  the  week can be a spelled out day of the week, optionally
       preceded by an ordinal number such as second tuesday.  A zone is  a  time  zone  value  as
       specified  by  the  tzselect(1)  command.   A  relative  time  can be a time unit that is:
       preceded by a cardinal number such as 1 year or 2 months, preceded  by  one  of  the  time
       words this or last, or succeeded by the time word ago.  A relative time can also be one of
       the time words: yesterday, today,  tomorrow,  now.   Examples  of  datetime  strings  are:
       1996-03-04 13:07:47 EST Mon, 1996-03-05 14:07:47 EST -1hour, Mon Mar  4 13:07:47 1996, Mar
       4 1996, Mar 4, Mar, 13:07:50 or 13:08.

       For any missing low order fields, the default value of 0 is assumed for hours, minutes and
       seconds,  1 for day of the month and Jan for months.  Hence, the following are equivalent:
       -S '@ Mar 1996' and -S '@ Mar 1 00:00:00 1996'.

       If any high order fields are missing, they are filled in by starting with the year,  month
       and  day  from  the  current time (real-time mode) or the time at the beginning of the PCP
       archive log (archive mode) and advancing the time until it matches  the  fields  that  are
       specified.   So,  for example if the time window starts by default at ``Mon Mar 4 13:07:47
       1996'', then -S @13:10 corresponds to 13:10:00  on  Mon  Mar  4,  1996,  while  -S  @10:00
       corresponds to 10:00:00 on Tue Mar 5, 1996 (note this is the following day).

       For  greater  precision  than  afforded  by  datetime(3),  the  seconds component may be a
       floating point number.

PERFORMANCE METRICS - NAMES AND IDENTIFIERS

       The number of performance metric names supported by PCP on most platforms ranges from many
       hundreds  to  several  thousand.   The  PCP  libraries  and  applications  use an internal
       identification scheme that unambiguously associates  a  single  integer  with  each  known
       performance  metric.  This integer is known as the Performance Metric Identifier, or PMID.
       Although not a requirement, PMIDs tend to have global consistency across all systems, so a
       particular performance metric usually has the same PMID.

       For  all users and most applications, direct use of the PMIDs would be inappropriate (e.g.
       this would limit the range of accessible metrics, make the code hard  to  maintain,  force
       the  user  interface  to be particularly baroque, etc.).  Hence a Performance Metrics Name
       Space (PMNS) is used to  provide  external  names  and  a  hierarchic  classification  for
       performance  metrics.   A  PMNS is represented as a tree, with each node having a label, a
       pointer to either a PMID (for leaf nodes) or a set of descendent nodes in  the  PMNS  (for
       non-leaf nodes).

       A  node label must begin with an alphabetic character, followed by zero or more characters
       drawn from the alphabetics, the digits and character  `_ยด  (underscore).   For  alphabetic
       characters in a node label, upper and lower case are distinguished.

       By  convention,  the  name  of a performance metric is constructed by concatenation of the
       node labels on a path through the PMNS from the root node to a leaf node, with a ``.''  as
       a  separator.   The  root node in the PMNS is unlabeled, so all names begin with the label
       associated with one of the descendent  nodes  below  the  root  node  of  the  PMNS,  e.g.
       kernel.percpu.syscall.   Typically  (although this is not a requirement) there would be at
       most one name for each PMID in a PMNS.  For example kernel.all.cpu.idle and  disk.dev.read
       are the unique names for two distinct performance metrics, each with a unique PMID.

       Groups  of  related  PMIDs  may  be named by naming a non-leaf node in the PMNS tree, e.g.
       disk.

       The default local PMNS used by pmcd  is  located  at  $PCP_VAR_DIR/pmns/root  however  the
       environment  variable  PMNS_DEFAULT  may  be  set to the full pathname of a different PMNS
       which will then be used as the default local PMNS.

       Most applications do not use the local PMNS directly, but rather import parts of the  PMNS
       as  required  from  the same place that performance metrics are fetched, i.e. from pmcd(1)
       for live monitoring or from a PCP archive for retrospective monitoring.

       To explore the PMNS use pminfo(1), or if the PCP GUI package is installed  the  New  Chart
       and Metric Search windows within pmchart(1).

PERFORMANCE METRIC SPECIFICATIONS

       In   configuration   files   and  (to  a  lesser  extent)  command  line  options,  metric
       specifications adhere to the following syntax rules.

       If the source of performance metrics is real-time from pmcd(1) then the accepted syntax is
                 host:metric[instance1,instance2,...]

       If the source of performance metrics is a PCP archive log then the accepted syntax is
                 archive/metric[instance1,instance2,...]

       The host:, archive/ and [instance1,instance2,...]  components are all optional.

       The , delimiter in the list of instance names may be replaced by white space.

       Special characters in instance names may be escaped by  surrounding  the  name  in  double
       quotes or preceding the character with a backslash.

       White space is ignored everywhere except within a quoted instance name.

       An  empty  instance  is  silently  ignored,  and  in  particular  ``[]'' is the same as no
       instance, while ``[one,,,two]'' is parsed as specifying just the two instances ``one'' and
       ``two''.

       As  a  special  case,  if  the  host  is  the single character ``@'' then this refers to a
       PM_CONTEXT_LOCAL source, see pmNewContext(3).

SECURE PMCD CONNECTIONS

       Since PCP version 3.6.11, a monitor can  explicitly  request  a  secure  connection  to  a
       collector host running pmcd(1) or pmproxy(1) using the PM_CTXFLAG_SECURE context flag.  If
       the PCP Collector host supports this feature - refer to the pmcd.feature.secure metric for
       confirmation  of  this  -  a  TLS/SSL  (Transport  Layer Security or Secure Sockets Layer)
       connection can be established which uses public key cryptography and  related  techniques.
       These  features  aim  to  prevent  eavesdropping and data tampering from a malicious third
       party, as well as providing server-side  authentication  (confident  identification  of  a
       server by a client) which can be used to guard against man-in-the-middle attacks.

       A  secure  pmcd connection requires use of certificate-based authentication.  The security
       features offered by pmcd and pmproxy are implemented using the Network  Security  Services
       (NSS)  APIs  and  utilities.   The  NSS  certutil  tool can be used to create certificates
       suitable for establishing trust between PCP monitor and collector hosts.

       A complete description is beyond the scope of this document, refer to the PCP ENVIRONMENT,
       FILES and SEE ALSO sections for detailed information.  This includes links to tutorials on
       the steps involved in setting up the available security features.

PMCD HOST SPECIFICATION

       In the absence of an explicit host name specification, most  tools  will  default  to  the
       local  host  in  live update mode.  In PCP releases since 3.8.4 onward, this results in an
       efficient local protocol being selected - typically a Unix domain socket.  If this  option
       is used (which can also be explicitly requested via the unix: host specification described
       below), it is important to note that all connections will be automatically  authenticated.
       In  other  words, the credentials of the user invoking a client tool will automatically be
       made available to pmcd(1) and all of its PMDAs, on the users behalf, such that results can
       be customized to the privilege levels of individual users.

       Names of remote hosts running the pmcd(1) daemon can of course also be provided to request
       a remote host be used.  The most basic form of pmcd host specification is  a  simple  host
       name, possibly including the domain name if necessary.  However, this can be extended in a
       number of ways to further refine attributes of the connection made to pmcd.

       The pmcd port number and also optional pmproxy(1) hostname and its  port  number,  can  be
       given  as  part  of  the  host specification, since PCP version 3.0.  These supersede (and
       override) the old-style PMCD_PORT, PMPROXY_HOST and PMPROXY_PORT environment variables.

       The following are valid hostname specifications that specify connections to pmcd  on  host
       nas1.servers.com  with/without  a  list  of  ports,  with/without  a pmproxy(1) connection
       through a firewall, and with IPv6 and IPv4 addresses as shown.

            $ pcp -h nas1.servers.com:44321,4321@firewall.servers.com:44322
            $ pcp -h nas1.servers.com:44321@firewall.servers.com:44322
            $ pcp -h nas1.servers.com:44321@firewall.servers.com
            $ pcp -h nas1.servers.com@firewall.servers.com
            $ pcp -h nas1.servers.com:44321
            $ pcp -h [fe80::2ad2:44ff:fe88:e4f1%p2p1]
            $ pcp -h 192.168.0.103

       In addition, ``connection attributes'' can also be  specified.   These  include  username,
       password  (can  be  given  interactively  and  may  depend on the authentication mechanism
       employed), whether  to  target  a  specific  running  container,  whether  to  use  secure
       (encrypted)  or  native (naked) protocol, and so on.  The previous examples all default to
       native protocol, and use no authentication.  This can be  altered,  as  in  the  following
       examples.

            $ pcp -h pcps://app2.servers.com?container=cae8e6edc0d5
            $ pcp -h pcps://nas1.servers.com:44321?username=tanya&method=gssapi
            $ pcp -h pcps://nas2.servers.com@firewalls.r.us?method=plain
            $ pcp -h pcp://nas3.servers.com
            $ pcp -h 192.168.0.103?container=cae8e6edc0d5,method=digest-md5
            $ pcp -h unix:
            $ pcp -h local:

       The  choice  of  authentication  method,  and  other  resulting  parameters like username,
       optionally password, etc, depends on the SASL2 configuration used by each  (remote)  pmcd.
       Tutorials  are  available  specifying  various  aspects  of configuring the authentication
       module(s) used, these fine details are outside the scope of this document.

       In all situations, host names can be used interchangeably with  IPv4  or  IPv6  addressing
       (directly),  as  shown  above.   In  the case of an IPv6 address, the full address must be
       enclosed by square brackets and the scope (interface) must also be specified.

       The final local: example above is now the default for most tools.  This connection  is  an
       automatically  authenticated  local  host  connection  on  all platforms that support Unix
       domain sockets.  No password is required and authentication is automatic.   This  is  also
       the most efficient (lowest overhead) communication channel available.

       The  difference between unix: and local: is that the former is a strict Unix domain socket
       specification (connection fails if it cannot connect that way), whereas the latter  has  a
       more  forgiving fallback to using localhost (i.e. a regular Inet socket connection is used
       when Unix domain socket connections are unavailable).

ENVIRONMENT

       In addition to the PCP run-time environment and configuration variables described  in  the
       PCP   ENVIRONMENT  section  below,  the  following  environment  variables  apply  to  all
       installations.

       PCP_CONSOLE
              When set, this changes the default console from /dev/tty  (on  Unix)  or  CON:  (on
              Windows)  to  be  the  specified console.  The special value of none can be used to
              indicate no console is available for use.  This is used in  places  where  console-
              based  tools  need  to  interact  with  the  user,  and  in particular is used when
              authentication is being performed.

       PCP_DEBUG
              When set, this variable provides an alternate way to initialize the value  used  to
              control  diagnostic  and  debug  output  (the  default  value is 0 to suppress this
              output).  The value for $PCP_DEBUG should be a decimal integer, see pmdbg(1) for  a
              description of the supported values.

       PCP_DERIVED_CONFIG
              When  set, this variable defines a colon separated list of files and/or directories
              (the syntax is the same as for the $PATH variable for sh(1)).  The  components  are
              expanded  into a list of files as follows: if a component of $PCP_DERIVED_CONFIG is
              a file, then that file is added to the list, else if a  component  is  a  directory
              then  recursive  descent  is  used  to enumerate all files below that directory and
              these are added to the list.

              Each file in the resulting list  is  assumed  to  contain  definitions  of  derived
              metrics as per the syntax described in pmLoadDerivedConfig(3), and these are loaded
              in order.

              Derived metrics may be used to extend the  available  metrics  with  new  (derived)
              metrics using simple arithmetic expressions.

              If  PCP_DERIVED_CONFIG  is  set,  the  derived  metric  definitions  are  processed
              automatically as each new source of performance metrics is established  (i.e.  each
              time a pmNewContext(3) is called) or when requests are made against the PMNS.

              Any component in the $PCP_DERIVED_CONFIG list or the expanded list of files that is
              not a file, or is not a directory or is not accessible (due to permissions or a bad
              symbolic link) will be silently ignored.

       PCP_SECURE_SOCKETS
              When set, this variable forces any monitor tool connections to be established using
              the certificate-based  secure  sockets  feature.   If  the  connections  cannot  be
              established securely, they will fail.

       PCP_SECURE_DB_METHOD
              With  secure  socket  connections, the certificate and key database is stored using
              the sql: method by default.  Use PCP_SECURE_DB_METHOD to override the default, most
              usually setting the value to the empty string (for the older database methods).

       PCP_STDERR
              Many  PCP  tools  support the environment variable PCP_STDERR, which can be used to
              control where error messages are sent.  When unset, the default  behavior  is  that
              ``usage''  messages and option parsing errors are reported on standard error, other
              messages after initial startup are sent to the default destination  for  the  tool,
              i.e. standard error for ASCII tools, or a dialog for GUI tools.

              If  PCP_STDERR  is  set  to  the  literal  value  DISPLAY then all messages will be
              displayed in a dialog.  This is used for any tools  launched  from  the  a  Desktop
              environment.

              If PCP_STDERR is set to any other value, the value is assumed to be a filename, and
              all messages will be written there.

       PMCD_CONNECT_TIMEOUT
              When attempting to connect to a remote pmcd(1) on a machine that  is  booting,  the
              connection attempt could potentially block for a long time until the remote machine
              finishes its initialization.  Most PCP applications and some  of  the  PCP  library
              routines  will abort and return an error if the connection has not been established
              after some specified interval has elapsed.  The  default  interval  is  5  seconds.
              This  may  be modified by setting PMCD_CONNECT_TIMEOUT in the environment to a real
              number of seconds for the desired timeout.  This is most useful in cases where  the
              remote  host  is  at  the  end  of  a  slow  network, requiring longer latencies to
              establish the connection correctly.

       PMCD_RECONNECT_TIMEOUT
              When a monitor  or  client  application  loses  a  connection  to  a  pmcd(1),  the
              connection  may  be re-established by calling a service routine in the PCP library.
              However, attempts to reconnect are controlled  by  a  back-off  strategy  to  avoid
              flooding  the  network with reconnection requests.  By default, the back-off delays
              are 5, 10, 20, 40 and 80 seconds  for  consecutive  reconnection  requests  from  a
              client  (the last delay will be repeated for any further attempts after the fifth).
              Setting the environment variable PMCD_RECONNECT_TIMEOUT to a comma  separated  list
              of   positive   integers   will   re-define   the  back-off  delays,  e.g.  setting
              PMCD_RECONNECT_TIMEOUT to ``1,2'' will back-off for 1 second, then attempt  another
              connection request every 2 seconds thereafter.

       PMCD_REQUEST_TIMEOUT
              For  monitor or client applications connected to pmcd(1), there is a possibility of
              the application "hanging" on a request for performance metrics or metadata or  help
              text.   These  delays  may become severe if the system running pmcd crashes, or the
              network   connection   is   lost.    By   setting    the    environment    variable
              PMCD_REQUEST_TIMEOUT  to  a  number of seconds, requests to pmcd will timeout after
              this number of seconds.  The default behavior is to be willing to wait  10  seconds
              for a response from every pmcd for all applications.

       PMCD_WAIT_TIMEOUT
              When  pmcd(1)  is  started  from  $PCP_RC_DIR/pcp  then  the  primary  instance  of
              pmlogger(1) will be started if the  configuration  flag  pmlogger  is  chkconfig(8)
              enabled and pmcd is running and accepting connections.

              The  check  on pmcd's readiness will wait up to PMCD_WAIT_TIMEOUT seconds.  If pmcd
              has a long startup time (such as on a very large  system),  then  PMCD_WAIT_TIMEOUT
              can be set to provide a maximum wait longer than the default 60 seconds.

       PMNS_DEFAULT
              If  set, then interpreted as the full pathname to be used as the default local PMNS
              for  pmLoadNameSpace(3).   Otherwise,  the  default  local  PMNS  is   located   at
              $PCP_VAR_DIR/pcp/pmns/root for base PCP installations.

       PCP_COUNTER_WRAP
              Many  of  the  performance  metrics  exported from PCP agents have the semantics of
              counter meaning they are expected  to  be  monotonically  increasing.   Under  some
              circumstances,  one  value of these metrics may smaller than the previously fetched
              value.  This can happen when a counter of finite precision overflows, or  when  the
              PCP  agent  has  been reset or restarted, or when the PCP agent is exporting values
              from  some  underlying  instrumentation  that  is  subject  to  some   asynchronous
              discontinuity.

              The  environment  variable  PCP_COUNTER_WRAP  may  be set to indicate that all such
              cases of a decreasing ``counter'' should be treated  as  a  counter  overflow,  and
              hence  the  values  are  assumed  to  have  wrapped  once  in  the interval between
              consecutive samples.  This ``wrapping'' behavior was the  default  in  earlier  PCP
              versions, but by default has been disabled in PCP release from version 1.3 on.

       PMDA_PATH
              The  PMDA_PATH  environment  variable may be used to modify the search path used by
              pmcd(1) and pmNewContext(3) (for PM_CONTEXT_LOCAL contexts) when  searching  for  a
              daemon  or  DSO  PMDA.   The  syntax  follows  that for PATH in sh(1), i.e. a colon
              separated   list   of   directories,   and   the    default    search    path    is
              ``/var/pcp/lib:/usr/pcp/lib'',  (or ``/var/lib/pcp/lib'' on Linux, depending on the
              value of the $PCP_VAR_DIR environment variable).

       PMCD_PORT
              The TPC/IP port(s) used by pmcd(1) to create the socket  for  incoming  connections
              and  requests,  was  historically  4321 and more recently the officially registered
              port 44321; in the current release, both port numbers are  used  by  default  as  a
              transitional  arrangement.   This  may  be  over-ridden  by  setting PMCD_PORT to a
              different port number, or a comma-separated list of port numbers.  If a non-default
              port  is used when pmcd is started, then every monitoring application connecting to
              that pmcd must also have PMCD_PORT set in their  environment  before  attempting  a
              connection.

       The  following  environment  variables are relevant to installations in which pmlogger(1),
       the PCP archive logger, is used.

       PMLOGGER_PORT
              The environment variable PMLOGGER_PORT may be used to change the base  TCP/IP  port
              number used by pmlogger(1) to create the socket to which pmlc(1) instances will try
              and connect.  The default base port  number  is  4330.   When  used,  PMLOGGER_PORT
              should be set in the environment before pmlogger is executed.

       PMLOGGER_REQUEST_TIMEOUT
              When  pmlc(1)  connects  to  pmlogger(1),  there  is  a  remote possibility of pmlc
              "hanging" on a request for information as a consequence of a failure of the network
              or  pmlogger.   By  setting  the environment variable PMLOGGER_REQUEST_TIMEOUT to a
              number of seconds, requests to pmlogger will timeout after this number of  seconds.
              The  default  behavior  is  to  be willing to wait forever for a response from each
              request to a pmlogger.  When used, PMLOGGER_REQUEST_TIMEOUT should be  set  in  the
              environment before pmlc is executed.

       If  you  have  the  PCP  product  installed,  then the following environment variables are
       relevant to the Performance Metrics Domain Agents (PMDAs).

       PMDA_LOCAL_PROC
              Use this variable has been deprecated and it is now ignored.  If the ``proc''  PMDA
              is  configured  as  a  DSO  for  use with pmcd(1) on the local host then all of the
              ``proc'' metrics  will  be  available  to  applications  using  a  PM_CONTEXT_LOCAL
              context.

              The  previous  behaviour  was  that  if  this  variable  was  set,  then  a context
              established with the type of PM_CONTEXT_LOCAL will have access to the ``proc'' PMDA
              to retrieve performance metrics about individual processes.

       PMDA_LOCAL_SAMPLE
              Use  this  variable  has  been deprecated and it is now ignored.  If the ``sample''
              PMDA is configured as a DSO for use with pmcd(1) on the local host then all of  the
              ``sample''  metrics  will  be  available  to  applications using a PM_CONTEXT_LOCAL
              context.

              The previous  behaviour  was  that  if  this  variable  was  set,  then  a  context
              established  with  the  type of PM_CONTEXT_LOCAL will have access to the ``sample''
              PMDA if this optional PMDA has been installed locally.

       PMIECONF_PATH
              If set, pmieconf(1) will form its pmieconf(5) specification (set  of  parameterized
              pmie(1) rules) using all valid pmieconf files found below each subdirectory in this
              colon-separated  list  of   subdirectories.    If   not   set,   the   default   is
              $PCP_VAR_DIR/config/pmieconf.

FILES

       /etc/pcp.conf
                 Configuration file for the PCP runtime environment, see pcp.conf(5).
       /etc/pki/nssdb
                 Optionally  contains a Network Security Services database with a "PCP Collector"
                 certificate providing trusted identification for the collector host.
       $HOME/.pcp
                 User-specific directories containing configuration files  for  customisation  of
                 the various monitor tools, such as pmchart(1).
       $HOME/.pki/nssdb
                 A  shared Network Security Services (NSS) database directory containing per-user
                 certificates identifying known valid  remote  pmcd  collector  hosts.   The  NSS
                 certutil tool is one of several that can be used to maintain this database.
       $PCP_RC_DIR/pcp
                 Script for starting and stopping pmcd(1).
       $PCP_PMCDCONF_PATH
                 Control file for pmcd(1).
       $PCP_PMCDOPTIONS_PATH
                 Command  line options passed to pmcd(1) when it is started from $PCP_RC_DIR/pcp.
                 All the command line option lines should  start  with  a  hyphen  as  the  first
                 character.  This file can also contain environment variable settings of the form
                 "VARIABLE=value".
       $PCP_BINADM_DIR
                 Location of PCP utilities for collecting and maintaining PCP archives, PMDA help
                 text, PMNS files etc.
       $PCP_PMDAS_DIR
                 Parent  directory  of the installation directory for Dynamic Shared Object (DSO)
                 PMDAs.
       $PCP_RUN_DIR/pmcd.pid
                 If pmcd is running, this file contains an ascii decimal  representation  of  its
                 process ID.
       $PCP_LOG_DIR/pmcd
                 Default  location of log files for pmcd(1), current directory for running PMDAs.
                 Archives generated by pmlogger(1) are generally below $PCP_LOG_DIR/pmlogger.
       $PCP_LOG_DIR/pmcd/pmcd.log
                 Diagnostic and status log for the current running pmcd(1)  process.   The  first
                 place to look when there are problems associated with pmcd.
       $PCP_LOG_DIR/pmcd/pmcd.log.prev
                 Diagnostic and status log for the previous pmcd(1) instance.
       $PCP_LOG_DIR/NOTICES
                 Log of pmcd(1) and PMDA starts, stops, additions and removals.
       $PCP_VAR_DIR/config
                 Contains directories of configuration files for several PCP tools.
       $PCP_SYSCONF_DIR/pmcd/rc.local
                 Local script for controlling PCP boot, shutdown and restart actions.
       $PCP_VAR_DIR/pmns
                 Directory containing the set of PMNS files for all installed PMDAs.
       $PCP_VAR_DIR/pmns/root
                 The ASCII pmns(5) exported by pmcd(1) by default.  This PMNS is be the super set
                 of all other PMNS files installed in $PCP_VAR_DIR/pmns.
       In addition, if the PCP product is installed  the  following  files  and  directories  are
       relevant.
       $PCP_LOG_DIR/NOTICES
              In addition to the pmcd(1) and PMDA activity, may be used to log alarms and notices
              from pmie(1) via pmpost(1).
       $PCP_PMLOGGERCONTROL_PATH
              Control file for pmlogger(1) instances launched from $PCP_RC_DIR/pcp and/or managed
              by  pmlogger_check(1)  and  pmlogger_daily(1)  as  part of a production PCP archive
              collection setup.

PCP ENVIRONMENT

       Environment variables with the prefix PCP_ are used to parameterize the file and directory
       names used by PCP.  On each installation, the file /etc/pcp.conf contains the local values
       for these variables.  The $PCP_CONF  variable  may  be  used  to  specify  an  alternative
       configuration file, as described in pcp.conf(5).

SEE ALSO

       pmcd(1),  pmie(1),  pmie_daily(1),  pminfo(1),  pmlc(1),  pmlogger(1),  pmlogger_daily(1),
       pmstat(1), pmval(1), pcp(1), pcp.conf(5), pcp.env(5), pmns(5) and chkconfig(8).

       If the PCP GUI package is installed, then the following entries are also relevant:
       pmchart(1), pmtime(1), and pmdumptext(1).

       If the secure sockets extensions have been enabled, then the following references are also
       relevant:
       http://www.pcp.io/documentation.html
       http://www.mozilla.org/projects/security/pki/nss/#documentation
       http://www.mozilla.org/projects/security/pki/nss/tools/certutil.html

       Also  refer  to  the  books  Performance  Co-Pilot  User's  and  Administrator's Guide and
       Performance Co-Pilot Programmer's Guide which can be found at http://www.pcp.io/