Provided by: pcp_4.3.1-1_amd64 bug

NAME

       PCPIntro - introduction to the Performance Co-Pilot (PCP)

INTRODUCTION

       The  Performance  Co-Pilot (PCP) is a toolkit designed for monitoring and managing system-
       level performance.  These services are distributed and scalable to  accommodate  the  most
       complex system configurations and performance problems.

       PCP  supports  many  different  platforms,  including  (but not limited to) Linux, MacOSX,
       Solaris and Windows.  From a high-level PCP can be considered to contain  two  classes  of
       software utility:

       PCP Collectors
               These  are the parts of PCP that collect and extract performance data from various
               sources, for example the operating system kernel.

       PCP Monitors
               These are the parts of PCP that display data collected from  hosts  (or  archives)
               that  have  the PCP Collector installed.  Many monitor tools are available as part
               of the core PCP release, while other (typically graphical)  monitoring  tools  are
               available separately in the PCP GUI package.

       This  manual  entry  describes  the  high-level  features  and  options common to most PCP
       utilities available on all platforms.

OVERVIEW

       The PCP architecture is distributed in the sense  that  any  PCP  tool  may  be  executing
       remotely.   On  the  host  (or hosts) being monitored, each domain of performance metrics,
       whether the kernel, a service layer, a  database  management  system,  a  web  server,  an
       application, etc.  requires a Performance Metrics Domain Agent (PMDA) which is responsible
       for collecting performance measurements from that domain.  All PMDAs are controlled by the
       Performance Metrics Collector Daemon (pmcd(1)) on the same host.

       Client  applications (the monitoring tools) connect to pmcd(1), which acts as a router for
       requests, by forwarding requests to the appropriate PMDA and returning  the  responses  to
       the  clients.  Clients may also access performance data from sets of PCP archives (created
       using pmlogger(1)) for retrospective analysis.

   Security philosophy
       PCP redistributes a wealth of  performance  information  within  a  host  and  across  its
       networks.   The  following  security  philosophy underlies the setting of several defaults
       that control how much information is sent and received.

       By default, the information exposed by PMCD about a host  is  approximately  of  the  same
       level of confidentiality as available to a completely unprivileged user on that host.  So,
       performance data that is available to be read completely freely on a machine may  be  made
       available by PMCD to the network.

       However,  the  host  running  PMCD  and  its  network  is not assumed to run only friendly
       applications.  Therefore, write type operations, including from the local  host,  are  not
       permitted by default.

       These  defaults  may  be  overridden  (expanded  or reduced) in several ways, including by
       specifying network ACLs in pmcd.conf, activating  non-default  PMDAs,  or  by  using  PMCD
       connections  that  pass  user  credentials.  For example, some PMDAs automatically provide
       greater information for particular credentialed users or groups.

   Applications
       The following performance monitoring applications are primarily console  based,  typically
       run  directly from the command line, and are just a small subset of the tools available as
       part of the base PCP package.

       Each tool or command is documented completely in its own reference page.

       pmstat Outputs an ASCII high-level summary of system performance.

       pmie   An inference engine that can evaluate predicate-action rules to perform alarms  and
              automate system management tasks.

       pminfo Interrogate specific performance metrics and the metadata that describes them.

       pmlogger
              Generates  PCP  archives  of  performance  metrics  suitable for replay by most PCP
              tools.

       pmrep  Highly customizable performance metrics reporter with support for various different
              output modes.

       pmval  Simple  periodic  reporting for some or all instances of a performance metric, with
              optional VCR time control.

       If the PCP GUI package is installed then the following additional tools are available.

       pmchart
              Displays trends over time of arbitrarily selected performance metrics from  one  or
              more hosts.

       pmtime Time  control  utility  for coordinating the time between multiple tools (including
              pmchart and pmval).

       pmdumptext
              Produce ASCII reports for arbitrary combinations of performance metrics.

COMMON COMMAND LINE ARGUMENTS

       There is a set of common command line arguments that are used  consistently  by  most  PCP
       tools.

       -a archive, --archive=archive
              Performance  metric  information  is  retrospectively  retrieved  from  the  set of
              Performance Co-Pilot (PCP) archives identified by archive previously  generated  by
              pmlogger(1).   See  LOGIMPORT(3)  and LOGARCHIVE(5) for archive creation interfaces
              and format documentation.

              archive is a comma-separated list of names, each of which may  be  the  name  of  a
              directory  containing  one  or  more  archives,  the base name common to all of the
              physical files created by an instance of pmlogger(1), or any one  of  the  physical
              files,   e.g.    /path/to/myarchives   (directory)  or  myarchive  (base  name)  or
              myarchive.meta (the metadata file)  or  myarchive.index  (the  temporal  index)  or
              myarchive.0 (the first data volume of archive) or myarchive.0.bz2 or myarchive.0.bz
              (the first data volume compressed with bzip2(1)) or myarchive.0.gz or myarchive.0.Z
              or  myarchive.0.z  (the  first data volume compressed with gzip(1)), myarchive.1 or
              myarchive.3.bz2 or myarchive.42.gz etc.

       -h host, --host=host
              Unless directed to another host by the -h (or  --host)  option,  or  to  a  set  of
              archives by the -a (or --archive) option, the source of performance metrics will be
              the Performance Metrics Collector Daemon (PMCD) on the local host.   Refer  to  the
              PMCD  HOST  SPECIFICATION  section  later  for  further details on the many options
              available when forming the host specification, as well as a detailed description of
              the  default  local  host  connection.   The  -a (or --archive), and -h (or --host)
              options are mutually exclusive.

       -s samples, --samples=samples
              The argument samples defines the number of samples to be  retrieved  and  reported.
              If  samples is 0 or -s (or --samples) is not specified, the application will sample
              and report continuously (in real time mode) or until the end  of  the  set  of  PCP
              archives (in archive mode).

       -z, --hostzone
              Change  the reporting timezone to the local timezone at the host that is the source
              of the performance metrics, as identified via either the -h (or --host) or  -a  (or
              --archive) options.

       -Z timezone, --timezone=timezone
              By  default, applications report the time of day according to the local timezone on
              the system where the application  is  executed.   The  -Z  (or  --timezone)  option
              changes  the  timezone  to timezone in the format of the environment variable TZ as
              described in environ(7).

       -D debugspec, --debug=debugspec
              Sets the PCP debugging options to debugspec to enable diagnostics and tracing  that
              is most helpful for developers or when trying to diagnose the misbehaviour of a PCP
              application.  debugspec should  be  a  comma-separated  list  of  debugging  option
              name(s)  and/or  decimal  integers, see pmdbg(1) for a description of the supported
              option names and values.

       In the absense of a live or archive source of metrics, a heuristic search for archive logs
       for the local host can be invoked via the -O (or --origin) option.  When using this option
       without an explicit source of metrics, monitor tools attempt to use archives from a system
       archive  location  such  as  $PCP_LOG_DIR/pmlogger/`hostname`.   Refer  to the TIME WINDOW
       SPECIFICATION section below for details on the acceptable syntax for  the  origin  option,
       but a typical invocation in this mode would be -O today or --origin yesterday.

INTERVAL SPECIFICATION AND ALIGNMENT

       Most PCP tools operate with periodic sampling or reporting, and the -t (or --interval) and
       -A (or --align) options may be used to control the duration of the sample interval and the
       alignment of the sample times.

       -t interval, --interval=interval
              Set the update or reporting interval.

              The  interval  argument  is  specified as a sequence of one or more elements of the
              form
                        number[units]
              where number is an integer or floating point constant (parsed using strtod(3))  and
              the optional units is one of: seconds, second, secs, sec, s, minutes, minute, mins,
              min, m, hours, hour, h, days, day and d.  If the unit is empty, second is assumed.

              In addition, the upper case (or mixed case) version of any of  the  above  is  also
              acceptable.

              Spaces  anywhere  in  the  interval  are  ignored,  so  4  days 6 hours 30 minutes,
              4day6hour30min, 4d6h30m and 4d6.5h are all equivalent.

              Multiple specifications are  additive,  for  example  ``1hour  15mins  30secs''  is
              interpreted as 3600+900+30 seconds.

       -A align, --align=align
              By default samples are not necessarily aligned on any natural unit of time.  The -A
              or --align option may be used to force the initial sample  to  be  aligned  on  the
              boundary  of  a natural time unit.  For example -A 1sec, -A 30min and --align 1hour
              specify alignment on whole seconds, half and whole hours respectively.

              The align argument follows the syntax for an interval argument described above  for
              the -t or --interval option.

              Note  that  alignment  occurs  by  advancing  the time as required, and that -A (or
              --align) acts as a modifier to advance both the start of the time window  (see  the
              next section) and the origin time (if the -O or --origin option is specified).

TIME WINDOW SPECIFICATION

       Many  PCP  tools  are  designed to operate in some time window of interest, for example to
       define a termination time for real-time monitoring or to  define  a  start  and  end  time
       within a set of PCP archive logs.

       In  the  absence of the -O (or --origin) and -A (or --align) options to specify an initial
       sample time origin and time alignment (see above), the PCP application will  retrieve  the
       first sample at the start of the time window.

       The following options may be used to specify a time window of interest.

       -S starttime, --start=starttime
              By  default  the  time window commences immediately in real-time mode, or coincides
              with time at the start of the set of PCP archive logs in archive mode.  The  -S  or
              --start  option  may  be  used  to  specify  a later time for the start of the time
              window.

              The starttime parameter may be given in one of three forms (interval is the same as
              for the -t or --interval option as described above, datetime is described below):

              interval
                     To  specify  an  offset  from  the  current  time (in real-time mode) or the
                     beginning of a set of PCP archives (in  archive  mode)  simply  specify  the
                     interval  of  time as the argument.  For example -S 30min will set the start
                     of the time window to be exactly 30 minutes from now in real-time  mode,  or
                     exactly 30 minutes from the start of a set of PCP archives.

              -interval
                     To  specify  an offset from the end of a set of PCP archive logs, prefix the
                     interval argument with a minus sign.  In this case, the start  of  the  time
                     window  precedes  the  time  at  the end of the set of archives by the given
                     interval.  For example -S -1hour will set the start of the time window to be
                     exactly  one hour before the time of the last sample in a set of PCP archive
                     logs.

              @datetime
                     To specify the calendar date and time (local time in the reporting timezone)
                     for  the start of the time window, use the datetime syntax preceded by an at
                     sign.  Refer to the datetime description below for detailed information.

       -T endtime, --finish=endtime
              By default the end of the time window is unbounded (in real-time mode)  or  aligned
              with the time at the end of a set of PCP archive logs (in archive mode).  The -T or
              --finish option may be used to specify an earlier time for  the  end  of  the  time
              window.

              The  endtime  parameter may be given in one of three forms (interval is the same as
              for the -t or --interval option as described above, datetime is described below):

              interval
                     To specify an offset from the start  of  the  time  window  simply  use  the
                     interval  of time as the argument.  For example -T 2h30m will set the end of
                     the time window to be 2 hours and 30 minutes after the  start  of  the  time
                     window.

              -interval
                     To  specify  an offset back from the time at the end of a set of PCP archive
                     logs, prefix the interval argument with a minus sign.  For example  -T  -90m
                     will  set the end of the time window to be 90 minutes before the time of the
                     last sample in a set of PCP archive logs.

              @datetime
                     To specify the calendar date and time (local time in the reporting timezone)
                     for  the  end  of the time window, use the datetime syntax preceded by an at
                     sign.  Refer to the datetime description below for detailed information.

       -O origin, --origin=origin
              By default samples are fetched from the start of the time window  (see  description
              of  -S  or  --start option) to the end of the time window (see description of -T or
              --finish option).  The -O or --origin option allows the specification of an  origin
              within  the  time window to be used as the initial sample time.  This is useful for
              interactive use of a PCP tool with the pmtime(1) VCR replay facility.

              The origin argument accepted by -O (or --origin) conforms to the  same  syntax  and
              semantics as the starttime argument for the -T (or --finish) option.

              For example --origin -0 specifies that the initial position should be at the end of
              the time window; this is most useful when wishing to  replay  ``backwards''  within
              the time window.

       The  datetime  argument  for  the  -O  (or --origin), -S (or --start) and -T (or --finish)
       options consists of:
                 date time zone day relative
       A date can be one of: YY-MM-DD, MM/DD/YY, DD Month YYYY, or Month DD YYYY.  A time can  be
       one  of: HH:MM:SS, HH:MM.  HH:MM can use either the 12 hour (via an am or pm suffix) or 24
       hour convention.  A day of the week can be a spelled  out  day  of  the  week,  optionally
       preceded  by  an  ordinal  number  such as second tuesday.  A zone is a time zone value as
       specified by the tzselect(8) command.  A relative  time  can  be  a  time  unit  that  is:
       preceded  by  a  cardinal  number  such as 1 year or 2 months, preceded by one of the time
       words this or last, or succeeded by the time word ago.  A relative time can also be one of
       the  time  words:  yesterday,  today,  tomorrow,  now.   Examples of datetime strings are:
       1996-03-04 13:07:47 EST Mon, 1996-03-05 14:07:47 EST -1hour, Mon Mar  4 13:07:47 1996, Mar
       4 1996, Mar 4, Mar, 13:07:50 or 13:08.

       For any missing low order fields, the default value of 0 is assumed for hours, minutes and
       seconds, 1 for day of the month and Jan for months.  Hence, the following are  equivalent:
       --start '@ Mar 1996' and --start '@ Mar 1 00:00:00 1996'.

       If  any high order fields are missing, they are filled in by starting with the year, month
       and day from the current time (real-time mode) or the time at the beginning of the set  of
       PCP  archive  logs  (archive mode) and advancing the time until it matches the fields that
       are specified.  So, for example if the time window  starts  by  default  at  ``Mon  Mar  4
       13:07:47  1996'',  then  --start  @13:10 corresponds to 13:10:00 on Mon Mar 4, 1996, while
       --start @10:00 corresponds to 10:00:00 on Tue Mar 5, 1996  (note  this  is  the  following
       day).

       For  greater  precision  than  afforded  by  datetime(3),  the  seconds component may be a
       floating point number.

       If a timezone is not included in  a  datetime  then  there  ares  several  interpretations
       available  depending  on  the  other command line options used.  The default is to use the
       local timezone on the system where the PCP tool is being run.  A -Z or  --timezone  option
       specifies an explicit timezone, else a -z or --hostzone option changes the timezone to the
       local timezone at the host that is the source of the performance metrics.

PERFORMANCE METRICS - IDENTIFIERS, NAMES, VALUES

       The number of performance metric names supported by PCP on most platforms ranges from many
       hundreds  to  several  thousand.   The  PCP  libraries  and  applications  use an internal
       identification scheme that unambiguously associates  a  single  integer  with  each  known
       performance  metric.  This integer is known as the Performance Metric Identifier, or PMID.
       Although not a requirement, PMIDs tend to have global consistency across all systems, so a
       particular performance metric usually has the same PMID.

       For  all users and most applications, direct use of the PMIDs would be inappropriate (this
       would limit the range of accessible metrics, make the code hard  to  maintain,  force  the
       user  interface  to be particularly baroque, and so on).  Hence a Performance Metrics Name
       Space (PMNS) is used to  provide  external  names  and  a  hierarchic  classification  for
       performance  metrics.   A  PMNS is represented as a tree, with each node having a label, a
       pointer to either a PMID (for leaf nodes) or a set of descendent nodes in  the  PMNS  (for
       non-leaf nodes).

       A  node label must begin with an alphabetic character, followed by zero or more characters
       drawn from the alphabetics, the digits and character ``_'' (underscore).   For  alphabetic
       characters in a node label, upper and lower case are distinguished.

       By  convention,  the  name  of a performance metric is constructed by concatenation of the
       node labels on a path through the PMNS from the root node to a leaf node, with a ``.''  as
       a  separator.   The  root node in the PMNS is unlabeled, so all names begin with the label
       associated with one of the descendent nodes below the root node of the PMNS,  for  example
       kernel.percpu.syscall.   Typically  (although this is not a requirement) there would be at
       most one name for each PMID in a PMNS.  For example kernel.all.cpu.idle and  disk.dev.read
       are the unique names for two distinct performance metrics, each with a unique PMID.

       Groups  of  related  PMIDs  may  be  named by naming a non-leaf node in the PMNS tree, for
       example disk.

       The default local PMNS used by pmcd  is  located  at  $PCP_VAR_DIR/pmns/root  however  the
       environment  variable  PMNS_DEFAULT  may  be  set to the full pathname of a different PMNS
       which will then be used as the default local PMNS.

       Most applications do not use the local PMNS directly, but rather import parts of the  PMNS
       as  required  from  the same place that performance metrics are fetched, i.e. from pmcd(1)
       for live monitoring or from a set of PCP archives for retrospective monitoring.

       To explore the PMNS use pminfo(1), or if the PCP GUI package is installed  the  New  Chart
       and Metric Search windows within pmchart(1).

       Some  performance  metrics  have  a  singular value.  For example, the available memory or
       number of context switches have one value per performance  metric  source,  that  is,  one
       value per host.  The metric descriptor (metadata) for each metric makes this fact known to
       applications that process values for these single-valued metrics.

       Some performance  metrics  have  a  set  of  values  or  instances  in  each  implementing
       performance  metric  domain.   For  example,  one  value for each disk, one value for each
       process, one value for each CPU, or one value for each activation of a given  application.
       When  a  metric  has multiple instances, the PMNS does not represent this in metric names;
       rather, a single metric may have  an  associated  set  of  values.   Multiple  values  are
       associated  with  the  members of an instance domain, such that each instance has a unique
       instance identifier within the associated instance domain.  For example, the  ''per  CPU´´
       instance  domain  may  use  the instance identifiers 0, 1, 2, 3, and so on to identify the
       configured processors in the system.  Internally,  instance  identifiers  are  encoded  as
       binary  values,  but each performance metric domain also supports corresponding strings as
       external names for the instance  identifiers,  and  these  names  are  used  at  the  user
       interface to the PCP utilities.

       Multiple performance metrics may be associated with a single instance domain.

       PCP  arranges  for  information  describing  instance  domains  to  be  exported  from the
       performance  metric  domains  to  the  applications   that   require   this   information.
       Applications may also choose to retrieve values for all instances of a performance metric,
       or some arbitrary subset of the available instances.

       Metric names and the instance domain concept provides two-dimensions for the modelling  of
       performance metrics.  This is a clear and simple model, however on some occassions it does
       not suffice.  For example, a metric may wish to represent higher dimensional data such  as
       ``per  CPU''  counters  for each running process.  In these cases it is common to create a
       compound instance, where the name is composed of  each  component  with  a  separator  in-
       between (for example, ``87245::cpu7'' might be used to separate process ID from CPU ID) to
       create flattened instance names.  Additionally, such cases benefit from the use of  metric
       instances  labels  to explicitly show the separate components (continuing the example from
       above, labels ``{"pid":87245,"cpu":7}'' might be used).

PERFORMANCE METRIC SPECIFICATIONS

       In  configuration  files  and  (to  a  lesser  extent)  command   line   options,   metric
       specifications adhere to the following syntax rules by most tools.

       If the source of performance metrics is real-time from pmcd(1) then the accepted syntax is
                 host:metric[instance1,instance2,...]

       If the source of performance metrics is a set of PCP archive logs then the accepted syntax
       is
                 archive/metric[instance1,instance2,...]

       The host:, archive/ and [instance1,instance2,...]  components are all optional.

       The , delimiter in the list of instance names may be replaced by white space.

       Special characters in instance names may be escaped by  surrounding  the  name  in  double
       quotes or preceding the character with a backslash.

       White space is ignored everywhere except within a quoted instance name.

       An  empty  instance  is  silently  ignored,  and  in  particular  ``[]'' is the same as no
       instance, while ``[one,,,two]'' is parsed as specifying just the two instances ``one'' and
       ``two''.

       As  a  special  case,  if  the  host  is  the single character ``@'' then this refers to a
       PM_CONTEXT_LOCAL source, see pmNewContext(3).

SECURE PMCD CONNECTIONS

       Since PCP version 3.6.11, a monitor can  explicitly  request  a  secure  connection  to  a
       collector host running pmcd(1) or pmproxy(1) using the PM_CTXFLAG_SECURE context flag.  If
       the PCP Collector host supports this feature - refer to the pmcd.feature.secure metric for
       confirmation  of  this  -  a  TLS/SSL  (Transport  Layer Security or Secure Sockets Layer)
       connection can be established which uses public key cryptography and  related  techniques.
       These  features  aim  to  prevent  eavesdropping and data tampering from a malicious third
       party, as well as providing server-side  authentication  (confident  identification  of  a
       server by a client) which can be used to guard against man-in-the-middle attacks.

       A  secure  pmcd connection requires use of certificate-based authentication.  The security
       features offered by pmcd and pmproxy are implemented using the Network  Security  Services
       (NSS)  APIs  and  utilities.   The  NSS  certutil  tool can be used to create certificates
       suitable for establishing trust between PCP monitor and collector hosts.

       A complete description is beyond the scope of this document, refer to the PCP ENVIRONMENT,
       FILES and SEE ALSO sections for detailed information.  This includes links to tutorials on
       the steps involved in setting up the available security features.

PMCD HOST SPECIFICATION

       In the absence of an explicit hostname specification, most tools will default to the local
       host  in  live  update  mode.   In  PCP  releases  since  3.8.4 onward, this results in an
       efficient local protocol being selected - typically a Unix domain socket.  If this  option
       is used (which can also be explicitly requested via the unix: host specification described
       below), it is important to note that all connections will be automatically  authenticated.
       In  other  words, the credentials of the user invoking a client tool will automatically be
       made available to pmcd(1) and all of its PMDAs, on the users behalf, such that results can
       be customized to the privilege levels of individual users.

       Names of remote hosts running the pmcd(1) daemon can of course also be provided to request
       a remote host be used.  The most basic form of pmcd host specification is  a  simple  host
       name, possibly including the domain name if necessary.  However, this can be extended in a
       number of ways to further refine attributes of the connection made to pmcd.

       The pmcd port number and also optional pmproxy(1) hostname and its  port  number,  can  be
       given  as  part  of  the  host specification, since PCP version 3.0.  These supersede (and
       override) the old-style PMCD_PORT, PMPROXY_HOST and PMPROXY_PORT environment variables.

       The following are valid hostname specifications that specify connections to pmcd  on  host
       nas1.acme.com with/without a list of ports, with/without a pmproxy(1) connection through a
       firewall, and with IPv6 and IPv4 addresses as shown.

            $ pcp --host nas1.acme.com:44321,4321@firewall.acme.com:44322
            $ pcp --host nas1.acme.com:44321@firewall.acme.com:44322
            $ pcp --host nas1.acme.com:44321@firewall.acme.com
            $ pcp --host nas1.acme.com@firewall.acme.com
            $ pcp --host nas1.acme.com:44321
            $ pcp --host [fe80::2ad2:44ff:fe88:e4f1%p2p1]
            $ pcp --host 192.168.0.103

       In addition, ``connection attributes'' can also be  specified.   These  include  username,
       password  (can  be  given  interactively  and  may  depend on the authentication mechanism
       employed), whether  to  target  a  specific  running  container,  whether  to  use  secure
       (encrypted)  or  native (naked) protocol, and so on.  The previous examples all default to
       native protocol, and use no authentication.  This can be  altered,  as  in  the  following
       examples.

            $ pcp --host pcps://app2.acme.com?container=cae8e6edc0d5
            $ pcp --host pcps://nas1.acme.com:44321?username=tanya&method=gssapi
            $ pcp --host pcps://nas2.acme.com@firewalls.r.us?method=plain
            $ pcp --host pcp://nas3.acme.com
            $ pcp --host 192.168.0.103?container=cae8e6edc0d5,method=digest-md5
            $ pcp --host unix:
            $ pcp --host local:

       The  choice  of  authentication  method,  and  other  resulting  parameters like username,
       optionally password, etc, depends on the SASL2 configuration used by each  (remote)  pmcd.
       Tutorials  are  available  specifying  various  aspects  of configuring the authentication
       module(s) used, these fine details are outside the scope of this document.

       In all situations, host names can be used interchangeably with  IPv4  or  IPv6  addressing
       (directly),  as  shown  above.   In  the case of an IPv6 address, the full address must be
       enclosed by square brackets and the scope (interface) must also be specified.

       The final local: example above is now the default for most tools.  This connection  is  an
       automatically  authenticated  local  host  connection  on  all platforms that support Unix
       domain sockets.  No password is required and authentication is automatic.   This  is  also
       the most efficient (lowest overhead) communication channel.

       The  difference between unix: and local: is that the former is a strict Unix domain socket
       specification (connection fails if it cannot connect that way), whereas the latter  has  a
       more  forgiving fallback to using localhost (i.e. a regular Inet socket connection is used
       when Unix domain socket connections are unavailable).

ENVIRONMENT

       In addition to the PCP run-time environment and configuration variables described  in  the
       PCP   ENVIRONMENT  section  below,  the  following  environment  variables  apply  to  all
       installations.

       Note that most uses of these environment variables are optimized to check the  environment
       only the first time the variable might be used.  As the environment usually is not checked
       again, the only safe strategy is to ensure all PCP-related environment variables  are  set
       before the first call into any of the PCP libraries.

       PCP_ALLOW_BAD_CERT_DOMAIN
              When set, allow clients to accept certificates with mismatched domain names with no
              prompt  when  they  are  sent  by   pmcd   or   other   server   components.    See
              PCP_SECURE_SOCKETS.

       PCP_ALLOW_SERVER_SELF_CERT
              When set, allow clients to accept self-signed certificates with no prompt when they
              are sent by pmcd or other server components.  See PCP_SECURE_SOCKETS.

       PCP_CONSOLE
              When set, this changes the default console from /dev/tty  (on  Unix)  or  CON:  (on
              Windows)  to  be  the  specified console.  The special value of none can be used to
              indicate no console is available for use.  This is used in  places  where  console-
              based  tools  need  to  interact  with  the  user,  and  in particular is used when
              authentication is being performed.

       PCP_DEBUG
              When set, this variable provides  an  alternate  to  the  -D  command  line  option
              described  above  to  initialize  the  diagnostic and debug options.  The value for
              $PCP_DEBUG is the same as for the -D command line option, namely a  comma-separated
              list  of  debugging  option  name(s),  and/or  decimal integers, see pmdbg(1) for a
              description of the supported option names and values.

       PCP_DERIVED_CONFIG
              When set, this variable defines a colon separated list of files and/or  directories
              (the  syntax  is the same as for the $PATH variable for sh(1)).  The components are
              expanded into a list of files as follows: if a component of $PCP_DERIVED_CONFIG  is
              a  file,  then  that  file is added to the list, else if a component is a directory
              then recursive descent is used to enumerate all  files  below  that  directory  and
              these are added to the list.

              Each  file  in  the  resulting  list  is  assumed to contain definitions of derived
              metrics as per the syntax described in pmLoadDerivedConfig(3), and these are loaded
              in order.

              Derived  metrics  may  be  used  to extend the available metrics with new (derived)
              metrics using simple arithmetic expressions.

              If  PCP_DERIVED_CONFIG  is  set,  the  derived  metric  definitions  are  processed
              automatically  as  each new source of performance metrics is established (i.e. each
              time a pmNewContext(3) is called) or when requests are made against the PMNS.

              Any component in the $PCP_DERIVED_CONFIG list or the expanded list of files that is
              not a file, or is not a directory or is not accessible (due to permissions or a bad
              symbolic link) will be silently ignored.

       PCP_IGNORE_MARK_RECORDS
              When PCP archives logs are created there  may  be  temporal  gaps  associated  with
              discontinuities  in  the  time  series  of logged data, for example when pmcd(1) is
              restarted or when multiple archive  logs  are  concatenated  with  pmlogextract(1).
              These  discontinuities are internally noted with a <mark> record in the PCP archive
              logs, and value interpolation as described in pmSetMode(3) is not supported  across
              <mark>  records  (because  the  values  before  and  after  a <mark> record are not
              necessarily from a continuous time series).  Sometimes  the  user  knows  the  data
              semantics    are    sound    in   the   region   of   the   <mark>   records,   and
              $PCP_IGNORE_MARK_RECORDS may be used to suppress the default behaviour.

              If PCP_IGNORE_MARK_RECORDS is set (but has no value) then all <mark>  records  will
              be ignored.  Otherwise the value $PCP_IGNORE_MARK_RECORDS follows the syntax for an
              interval argument described above for the -t option, and  <mark>  records  will  be
              ignored  if  the  time  gap between the last record before the <mark> and the first
              record after the <mark> is not more than interval.

       PCP_SECURE_SOCKETS
              When set, this variable forces any monitor tool connections to be established using
              the  certificate-based  secure  sockets  feature.   If  the  connections  cannot be
              established securely, they will fail.

       PCP_SECURE_DB_METHOD
              With secure socket connections, the certificate and key database  is  stored  using
              the sql: method by default.  Use PCP_SECURE_DB_METHOD to override the default, most
              usually setting the value to the empty string (for the older database methods).

       PCP_SECURE_DB_PATH
              When set, this variable specifies an alternate certficate database path for  client
              tools.  Similar to the action of the -C option for pmcd(1) and pmproxy(1).

       PCP_STDERR
              Many  PCP  tools  support the environment variable PCP_STDERR, which can be used to
              control where error messages are sent.  When unset, the default  behavior  is  that
              ``usage''  messages and option parsing errors are reported on standard error, other
              messages after initial startup are sent to the default destination  for  the  tool,
              i.e. standard error for ASCII tools, or a dialog for GUI tools.

              If  PCP_STDERR  is  set  to  the  literal  value  DISPLAY then all messages will be
              displayed in a dialog.  This is used for any tools  launched  from  the  a  Desktop
              environment.

              If PCP_STDERR is set to any other value, the value is assumed to be a filename, and
              all messages will be written there.

       PMCD_CONNECT_TIMEOUT
              When attempting to connect to a remote pmcd(1) on a machine that  is  booting,  the
              connection attempt could potentially block for a long time until the remote machine
              finishes its initialization.  Most PCP applications and some  of  the  PCP  library
              routines  will abort and return an error if the connection has not been established
              after some specified interval has elapsed.  The  default  interval  is  5  seconds.
              This  may  be modified by setting PMCD_CONNECT_TIMEOUT in the environment to a real
              number of seconds for the desired timeout.  This is most useful in cases where  the
              remote  host  is  at  the  end  of  a  slow  network, requiring longer latencies to
              establish the connection correctly.

       PMCD_RECONNECT_TIMEOUT
              When a monitor  or  client  application  loses  a  connection  to  a  pmcd(1),  the
              connection  may  be re-established by calling a service routine in the PCP library.
              However, attempts to reconnect are controlled  by  a  back-off  strategy  to  avoid
              flooding  the  network with reconnection requests.  By default, the back-off delays
              are 5, 10, 20, 40 and 80 seconds  for  consecutive  reconnection  requests  from  a
              client  (the last delay will be repeated for any further attempts after the fifth).
              Setting the environment variable PMCD_RECONNECT_TIMEOUT to a comma  separated  list
              of  positive  integers  will  re-define  the  back-off  delays, for example setting
              PMCD_RECONNECT_TIMEOUT to ``1,2'' will back-off for 1 second, then attempt  another
              connection request every 2 seconds thereafter.

       PMCD_REQUEST_TIMEOUT
              For  monitor or client applications connected to pmcd(1), there is a possibility of
              the application "hanging" on a request for performance metrics or metadata or  help
              text.   These  delays  may become severe if the system running pmcd crashes, or the
              network   connection   is   lost.    By   setting    the    environment    variable
              PMCD_REQUEST_TIMEOUT  to  a  number of seconds, requests to pmcd will timeout after
              this number of seconds.  The default behavior is to be willing to wait  10  seconds
              for a response from every pmcd for all applications.

       PMCD_WAIT_TIMEOUT
              When  pmcd(1)  is  started  from  $PCP_RC_DIR/pcp  then  the  primary  instance  of
              pmlogger(1) will be started if the configuration flag pmlogger is  chkconfig(8)  or
              systemctl(1) enabled and pmcd is running and accepting connections.

              The  check  on pmcd's readiness will wait up to PMCD_WAIT_TIMEOUT seconds.  If pmcd
              has a long startup time (such as on a very large  system),  then  PMCD_WAIT_TIMEOUT
              can be set to provide a maximum wait longer than the default 60 seconds.

       PMNS_DEFAULT
              If  set, then interpreted as the full pathname to be used as the default local PMNS
              for  pmLoadNameSpace(3).   Otherwise,  the  default  local  PMNS  is   located   at
              $PCP_VAR_DIR/pcp/pmns/root for base PCP installations.

       PCP_COUNTER_WRAP
              Many  of  the  performance  metrics  exported from PCP agents have the semantics of
              counter meaning they are expected  to  be  monotonically  increasing.   Under  some
              circumstances,  one  value of these metrics may smaller than the previously fetched
              value.  This can happen when a counter of finite precision overflows, or  when  the
              PCP  agent  has  been reset or restarted, or when the PCP agent is exporting values
              from  some  underlying  instrumentation  that  is  subject  to  some   asynchronous
              discontinuity.

              The  environment  variable  PCP_COUNTER_WRAP  may  be set to indicate that all such
              cases of a decreasing ``counter'' should be treated  as  a  counter  overflow,  and
              hence  the  values  are  assumed  to  have  wrapped  once  in  the interval between
              consecutive samples.  This ``wrapping'' behavior was the  default  in  earlier  PCP
              versions, but by default has been disabled in PCP release from version 1.3 on.

       PMDA_PATH
              The  PMDA_PATH  environment  variable may be used to modify the search path used by
              pmcd(1) and pmNewContext(3) (for PM_CONTEXT_LOCAL contexts) when  searching  for  a
              daemon  or  DSO  PMDA.   The  syntax  follows  that for PATH in sh(1), i.e. a colon
              separated   list   of   directories,   and   the    default    search    path    is
              ``/var/pcp/lib:/usr/pcp/lib'',  (or ``/var/lib/pcp/lib'' on Linux, depending on the
              value of the $PCP_VAR_DIR environment variable).

       PMCD_PORT
              The TPC/IP port(s) used by pmcd(1) to create the socket  for  incoming  connections
              and  requests,  was  historically  4321 and more recently the officially registered
              port 44321; in the current release, both port numbers are  used  by  default  as  a
              transitional  arrangement.   This  may  be  over-ridden  by  setting PMCD_PORT to a
              different port number, or a comma-separated list of port numbers.  If a non-default
              port  is used when pmcd is started, then every monitoring application connecting to
              that pmcd must also have PMCD_PORT set in their  environment  before  attempting  a
              connection.

       The  following  environment  variables are relevant to installations in which pmlogger(1),
       the PCP archive logger, is used.

       PMLOGGER_PORT
              The environment variable PMLOGGER_PORT may be used to change the base  TCP/IP  port
              number used by pmlogger(1) to create the socket to which pmlc(1) instances will try
              and connect.  The default base port  number  is  4330.   When  used,  PMLOGGER_PORT
              should be set in the environment before pmlogger is executed.

       PMLOGGER_REQUEST_TIMEOUT
              When  pmlc(1)  connects  to  pmlogger(1),  there  is  a  remote possibility of pmlc
              "hanging" on a request for information as a consequence of a failure of the network
              or  pmlogger.   By  setting  the environment variable PMLOGGER_REQUEST_TIMEOUT to a
              number of seconds, requests to pmlogger will timeout after this number of  seconds.
              The  default  behavior  is  to  be willing to wait forever for a response from each
              request to a pmlogger.  When used, PMLOGGER_REQUEST_TIMEOUT should be  set  in  the
              environment before pmlc is executed.

       If  you  have  the  PCP  product  installed,  then the following environment variables are
       relevant to the Performance Metrics Domain Agents (PMDAs).

       PMDA_LOCAL_PROC
              Use this variable has been deprecated and it is now ignored.  If the ``proc''  PMDA
              is  configured  as  a  DSO  for  use with pmcd(1) on the local host then all of the
              ``proc'' metrics  will  be  available  to  applications  using  a  PM_CONTEXT_LOCAL
              context.

              The  previous  behaviour  was  that  if  this  variable  was  set,  then  a context
              established with the type of PM_CONTEXT_LOCAL will have access to the ``proc'' PMDA
              to retrieve performance metrics about individual processes.

       PMDA_LOCAL_SAMPLE
              Use  this  variable  has  been deprecated and it is now ignored.  If the ``sample''
              PMDA is configured as a DSO for use with pmcd(1) on the local host then all of  the
              ``sample''  metrics  will  be  available  to  applications using a PM_CONTEXT_LOCAL
              context.

              The previous  behaviour  was  that  if  this  variable  was  set,  then  a  context
              established  with  the  type of PM_CONTEXT_LOCAL will have access to the ``sample''
              PMDA if this optional PMDA has been installed locally.

       PMIECONF_PATH
              If set, pmieconf(1) will form its pmieconf(5) specification (set  of  parameterized
              pmie(1) rules) using all valid pmieconf files found below each subdirectory in this
              colon-separated  list  of   subdirectories.    If   not   set,   the   default   is
              $PCP_VAR_DIR/config/pmieconf.

FILES

       /etc/pcp.conf
                 Configuration file for the PCP runtime environment, see pcp.conf(5).
       /etc/pki/nssdb
                 Optionally  contains a Network Security Services database with a "PCP Collector"
                 certificate providing trusted identification for the collector host.
       $HOME/.pcp
                 User-specific directories containing configuration files  for  customisation  of
                 the various monitor tools, such as pmchart(1).
       $HOME/.pki/nssdb
                 A  shared Network Security Services (NSS) database directory containing per-user
                 certificates identifying known valid  remote  pmcd  collector  hosts.   The  NSS
                 certutil tool is one of several that can be used to maintain this database.
       $PCP_RC_DIR/pcp
                 Script for starting and stopping pmcd(1).
       $PCP_PMCDCONF_PATH
                 Control file for pmcd(1).
       $PCP_PMCDOPTIONS_PATH
                 Command  line options passed to pmcd(1) when it is started from $PCP_RC_DIR/pcp.
                 All the command line option lines should  start  with  a  hyphen  as  the  first
                 character.  This file can also contain environment variable settings of the form
                 "VARIABLE=value".
       $PCP_BINADM_DIR
                 Location of PCP utilities for collecting and maintaining PCP archives, PMDA help
                 text, PMNS files etc.
       $PCP_PMDAS_DIR
                 Parent  directory  of the installation directory for Dynamic Shared Object (DSO)
                 PMDAs.
       $PCP_RUN_DIR/pmcd.pid
                 If pmcd is running, this file contains an ascii decimal  representation  of  its
                 process ID.
       $PCP_LOG_DIR/pmcd
                 Default  location of log files for pmcd(1), current directory for running PMDAs.
                 Archives generated by pmlogger(1) are generally below $PCP_LOG_DIR/pmlogger.
       $PCP_LOG_DIR/pmcd/pmcd.log
                 Diagnostic and status log for the current running pmcd(1)  process.   The  first
                 place to look when there are problems associated with pmcd.
       $PCP_LOG_DIR/pmcd/pmcd.log.prev
                 Diagnostic and status log for the previous pmcd(1) instance.
       $PCP_LOG_DIR/NOTICES
                 Log of pmcd(1) and PMDA starts, stops, additions and removals.
       $PCP_VAR_DIR/config
                 Contains directories of configuration files for several PCP tools.
       $PCP_SYSCONF_DIR/pmcd/rc.local
                 Local script for controlling PCP boot, shutdown and restart actions.
       $PCP_VAR_DIR/pmns
                 Directory containing the set of PMNS files for all installed PMDAs.
       $PCP_VAR_DIR/pmns/root
                 The ASCII pmns(5) exported by pmcd(1) by default.  This PMNS is be the super set
                 of all other PMNS files installed in $PCP_VAR_DIR/pmns.
       In addition, if the PCP product is installed  the  following  files  and  directories  are
       relevant.
       $PCP_LOG_DIR/NOTICES
              In addition to the pmcd(1) and PMDA activity, may be used to log alarms and notices
              from pmie(1) via pmpost(1).
       $PCP_PMLOGGERCONTROL_PATH
              Control file for pmlogger(1) instances launched from $PCP_RC_DIR/pcp and/or managed
              by  pmlogger_check(1)  and  pmlogger_daily(1)  as  part of a production PCP archive
              collection setup.

PCP ENVIRONMENT

       Environment variables with the prefix PCP_ are used to parameterize the file and directory
       names used by PCP.  On each installation, the file /etc/pcp.conf contains the local values
       for these variables.  The $PCP_CONF  variable  may  be  used  to  specify  an  alternative
       configuration file, as described in pcp.conf(5).

SEE ALSO

       pmcd(1),  pmie(1),  pmie_daily(1),  pminfo(1),  pmlc(1),  pmlogger(1),  pmlogger_daily(1),
       pmrep(1),  pmstat(1),  pmval(1),  pcp(1),   systemctl(1),   LOGIMPORT(3),   LOGARCHIVE(5),
       pcp.conf(5), pcp.env(5), pmns(5) and chkconfig(8).

       If the PCP GUI package is installed, then the following entries are also relevant:
       pmchart(1), pmtime(1), and pmdumptext(1).

       If the secure sockets extensions have been enabled, then the following references are also
       relevant:
       https://pcp.io/documentation.html
       http://www.mozilla.org/projects/security/pki/nss/#documentation
       http://www.mozilla.org/projects/security/pki/nss/tools/certutil.html

       Also refer to  the  books  Performance  Co-Pilot  User's  and  Administrator's  Guide  and
       Performance Co-Pilot Programmer's Guide which can be found at https://pcp.io/