Provided by: pcp_3.5.11_amd64 bug


       pmlogger - create archive log for performance metrics


       $PCP_BINADM_DIR/pmlogger  [-c  configfile]  [-h host] [-l logfile] [-L] [-n pmnsfile] [-P]
       [-r] [-s endsize] [-t interval] [-T endtime]  [-u]  [-v  volsize]  [-V  version]  [-x  fd]


       pmlogger creates the archive logs of performance metric values that may be ``played back''
       by other Performance Co-Pilot (see PCPIntro(1)) tools.  These logs form the basis  of  the
       VCR paradigm and retrospective performance analysis services common to the PCP toolkit.

       The  mandatory argument archive is the base name for the physical files that constitute an
       archive log.

       The -V option specifies whether a version 1 or version 2 archive is generated.  A  version
       2  archive  also stores the associated Performance Metrics Name Space (PMNS). By default a
       version 2 archive is generated.

       Unless directed to another host by the -h option, pmlogger will  contact  the  Performance
       Metrics Collector Daemon (PMCD) on the local host and use that as the source of the metric
       values to be logged.

       To support the required flexibility and control over what is  logged  and  when,  pmlogger
       maintains  an  independent  two  level logging state for each instance of each performance
       metric.  At the first (mandatory) level, logging is allowed to be on (with  an  associated
       interval  between  samples),  or  off or maybe.  In the latter case, the second (advisory)
       level logging is allowed to be on (with an associated interval between samples), or off.

       The mandatory level allows universal specification that some metrics must  be  logged,  or
       must  not  be  logged.   The  default state for all instances of all metrics when pmlogger
       starts is mandatory maybe and advisory off.

       Use pmlc(1) to interrogate and change the logging state once pmlogger is running.

       If a metric's state is mandatory (on or off) and  a  request  is  made  to  change  it  to
       mandatory  maybe,  the new state is mandatory maybe and advisory off.  If a metric's state
       is already advisory (on or off) and a request is made to change it to mandatory maybe, the
       current state is retained.

       It is not possible for pmlogger to log specific instances of a metric and all instances of
       the same metric concurrently.  If specific instances are being logged and a request to log
       all  instances  is  made, then all instances of the metric will be logged according to the
       new request, superseding any prior logging request for the metric.  A request to  log  all
       instances of a metric will supersede any previous request to log all instances.  A request
       to log specific instances of a metric when all  instances  are  already  being  logged  is
       refused.   To do this one must turn off logging for all instances of the metric first.  In
       each case, the validity of the request is checked first; for example a request to change a
       metric's  logging  state  to  advisory  on  when  it  is  currently mandatory off is never
       permitted (it is necessary to change the state to mandatory maybe first).

       Optionally, each system running pmcd(1) may  also  be  configured  to  run  a  ``primary''
       pmlogger  instance.   Like pmcd(1), this pmlogger instance is launched by $PCP_RC_DIR/pcp,
       and is affected by the  files  /etc/config/pmlogger  (use  chkconfig(1M)  to  activate  or
       disable the primary pmlogger instance), /etc/config/pmlogger.options (command line options
       passed to  the  primary  pmlogger)  and  $PCP_VAR_DIR/config/pmlogger/config.default  (the
       default initial configuration file for the primary pmlogger).

       The  primary  pmlogger  instance is identified by the -P option.  There may be at most one
       ``primary'' pmlogger instance on each system with an active pmcd(1).  The primary pmlogger
       instance (if any) must be running on the same host as the pmcd(1) to which it connects, so
       the -h and -P options are mutually exclusive.

       When  launched  as  a  non-primary  instance,  pmlogger  will  exit  immediately  if   the
       configuration file causes no metric logging to be scheduled.  The -L option overrides this
       behavior, and causes a non-primary pmlogger instance  to  ``linger'',  presumably  pending
       some  future  dynamic  re-configuration  and state change via pmlc(1).  pmlogger will also
       linger without the -L option being used if all the metrics to be logged are logged as once
       only  metrics.  When  the  once  only  metrics have been logged, a warning message will be
       generated stating that the event queue is empty and no more events will be scheduled.

       By default all diagnostics and errors from pmlogger are written to the  file  pmlogger.log
       in  the  directory  where pmlogger is launched.  The -l option may be used to override the
       default behavior.  If the log file cannot be created or is not writable, output is written
       to standard error instead.

       If  specified,  the  -s  option  instructs  pmlogger  to terminate after a certain size in
       records, bytes or time units has been accumulated.  If endsize is an integer then  endsize
       records  will be written to the log.  If endsize is an integer suffixed by b or bytes then
       endsize bytes of the archive data will be written out (note,  however,  that  archive  log
       record  boundaries will not be broken and so this limit may be slightly surpassed).  Other
       viable file size units include: K, Kb, Kbyte, Kilobyte for kilobytes  and  M,  Mb,  Mbyte,
       Megabyte  for  megabytes  and  G,  Gb,  Gbyte, Gigabyte for gigabytes.  These units may be
       optionally suffixed by an s and may be of mixed case.  Alternatively  endsize  may  be  an
       integer  or a floating point number suffixed using a time unit as described in PCPIntro(1)
       for the interval argument (to the standard PCP -t command line option).
       Some examples of different formats:
          -s 100
          -s 100bytes
          -s 100K
          -s 100Mb
          -s 10Gbyte
          -s 10mins
          -s 1.5hours
       The default is for pmlogger to run forever.

       The -r option causes the size of the physical record(s) for each group of metrics and  the
       expected  contribution  of  the  group  to the size of the PCP archive for one full day of
       collection to be reported in the log file.  This information is reported  the  first  time
       each group is successfully written to the archive.

       The  log file is potentially a multi-volume data set, and the -v option causes pmlogger to
       start a new volume after a certain  size  in  records,  bytes,  or  time  units  has  been
       accumulated for the current volume.  The format of this size specification is identical to
       that of the -s option (see above).  The default is for pmlogger to create a single  volume
       log.  Additional volume switches can also be forced asynchronously by either using pmlc(1)
       or sending pmlogger a SIGHUP signal (see below). Note, if a scheduled volume switch is  in
       operation  due  to  the  -v  option, then its counters will be reset after an asynchronous

       Independent of any -v option, each volume of an archive is limited to no  more  than  2^31
       bytes,  so  pmlogger  will  automatically  create a new volume for the archive before this
       limit is reached.

       Normally pmlogger operates on the  distributed  Performance  Metrics  Name  Space  (PMNS),
       however  if  the  -n option is specified an alternative local PMNS is loaded from the file

       Under normal circumstances, pmlogger will run  forever  (except  for  a  -s  option  or  a
       termination  signal).   The  -T  option  may be used to limit the execution time using the
       format of time as prescribed by PCPIntro(1).
       Some examples of different formats:
          -T 10mins
          -T '@ 11:30'
       From this it can be seen that -T 10mins and -s 10mins perform identical actions.

       When pmlogger receives a SIGHUP signal, the current volume of the log is closed, and a new
       volume  is  opened.  This mechanism (or the alternative mechanism via pmlc(1)) may be used
       to manage the growth of the log files - once a log volume is  closed,  that  file  may  be
       archived  without  ill-effect  on  the  continued  operation of pmlogger.  See also the -v
       option above.

       The buffers for the current log may be flushed to disk using the flush command of pmlc(1),
       or by sending pmlogger a SIGUSR1 signal or by using the -u option (the latter forces every
       log write to be unbuffered).  This is useful when the log needs to be read while  pmlogger
       is still running.

       When  launched  with  the -x option, pmlogger will accept asynchronous control requests on
       the file descriptor fd.  This option is  only  expected  to  be  used  internally  by  PCP
       applications that support ``live record mode''.


       The  configuration  file may be specified with the -c option.  If it is not, configuration
       specifications are read from standard input.

       If  configfile  does   not   exist,   then   a   search   is   made   in   the   directory
       $PCP_VAR_DIR/config/pmlogger  for a file of the same name, and if found that file is used,
       e.g.  if  config.mumble  does  not  exist  in  the  current   directory   and   the   file
       $PCP_VAR_DIR/config/pmlogger/config.mumble  does  exist,  then  -c  config.mumble  and  -c
       $PCP_VAR_DIR/config/pmlogger/config.mumble are equivalent.

       The syntax for the configuration file is as follows.

       1.     Words are separated by white space (space, tab or newline).

       2.     The symbol ``#'' (hash) introduces a comment, and all text up to the  next  newline
              is ignored.

       3.     Keywords (shown in bold below) must appear literally (i.e. in lower case).

       4.     Each  specification  begins  with  the optional keyword log, followed by one of the
              states mandatory on, mandatory off, mandatory maybe, advisory on or advisory off.

       5.     For the on states, a logging interval must follow using  the  syntax  ``once'',  or
              ``default'', or ``every N timeunits'', or simply ``N timeunits'' - N is an unsigned
              integer, and timeunits is one of the keywords msec, millisecond, sec, second,  min,
              minute, hour or the plural form of one of the above.
              Internal  limitations  require  the  interval to be smaller than (approximately) 74
              hours.  An interval value of zero is a synonym for once.  An  interval  of  default
              means  to use the default logging interval of 60 seconds; this default value may be
              changed to interval with the -t command line option.

              The interval argument follows the syntax  described  in  PCPIntro(1),  and  in  the
              simplest  form  may  be  an  unsigned  integer  (the implied units in this case are

       6.     Following the state and possible interval specifications comes a ``{'', followed by
              a list of one or more metric specifications and a closing ``}''.  The list is white
              space (or comma) separated.  If there is only one metric specification in the list,
              the braces are optional.

       7.     A  metric  specification  consists of a metric name optionally followed by a set of
              instance names.  The metric name follows the standard PCP naming  conventions,  see
              pmns(4),  and if the metric name is a non-leaf node in the PMNS (see pmns(4)), then
              pmlogger will recursively descend the PMNS and apply the logging  specification  to
              all  descendent  metric names that are leaf nodes in the PMNS.  The set of instance
              names is a ``['', followed by a list of one or  more  space  (or  comma)  separated
              names,  numbers  or  strings,  and  a closing ``]''.  Elements in the list that are
              numbers are assumed to be internal instance identifiers, other elements are assumed
              to be external instance identifiers - see pmGetInDom(3) for more information.

              If  no  instances  are  given,  then  the  logging  specification is applied to all
              instances of the associated metric.

       8.     There may be an arbitrary number of logging specifications.

       9.     Following all of the logging  specifications,  there  may  be  an  optional  access
              control  section, introduced by the literal token [access].  Thereafter come access
              control rules of the form ``allow hostlist : operation ;'' and ``disallow  hostlist
              : operation ;''.

              The  base  operations  are  advisory, mandatory and enquire.  In all other aspects,
              these access control statements follow the syntactic and semantic rules defined for
              the access control mechanisms used by PMCD and are fully documented in pmcd(1).


       For  each PCP utility, there is a sample pmlogger configuration file that could be used to
       create an archive log suitable for replaying with that tool  (i.e.  includes  all  of  the
       performance  metrics  used  by the tool).  For a tool named foo this configuration file is
       located in $PCP_VAR_DIR/config/pmlogger/

       The following is a simple default configuration file for a primary pmlogger instance,  and
       demonstrates most of the capabilities of the configuration specification language.

            log mandatory on once { hinv.ncpu hinv.ndisk }
            log mandatory on every 10 minutes {
       [ "et0" ]
                network.interface.out.packets [ "et0" ]
                nfs.server.reqs [ "lookup" "getattr" "read" "write" ]

            log advisory on every 30 minutes {

            disallow * : all except enquire;
            allow localhost : mandatory, advisory;


       It  is often useful for pmlogger processes (other than the primary instance) to be started
       and stopped when the local host is booted or shutdown.   The  script  $PCP_RC_DIR/pcplocal
       and  the  necessary  soft-links are provided, and can be modified by root to run PCP tools
       automatically.  If you want to find out more before starting, read the  manual  pages  for
       rc2(1), rc0(1), shutdown(1) and the file /etc/init.d/README.

       For example, changing $PCP_RC_DIR/pcplocal so that it contains:

           # Add startup actions here
           ($PCP_BINADM_DIR/pmlogger_check &)

           # Add shutdown actions here
           ($PCP_BINADM_DIR/pmsignal -a -s TERM pmlogger &)

       will  start  pmlogger  instances  at boot time and terminate them in an orderly fashion at
       system shutdown.

       This script runs as root, so any pmlogger instances it launches are also run as root.   To
       run  some  pmlogger instances as a particular user, create your own archive logger control
       file (see pmlogger_check(1)) and use the su(1) command. e.g.

           # Add startup actions here
           (su tanya -c "$PCP_BINADM_DIR/pmlogger_check -c /usr/people/tanya/ctl" &)

       at boot time will start the pmlogger instances described in /usr/people/tanya/ctl, running
       as user tanya.


                 metadata (metric descriptions, instance domains, etc.) for the archive log
       archive.0 initial volume of metrics values (subsequent volumes have suffixes 1, 2, ...)
                 temporal  index to support rapid random access to the other files in the archive
                 pmlogger maintains the files in this directory as the map between the process id
                 of  the  pmlogger  instance  and  the  IPC port that may be used to control each
                 pmlogger instance (as used by pmlc(1))
                 chkconfig(1M)  control   flag,   to   control   launching   of   pmlogger   from
                 command line options to pmlogger when launched from $PCP_RC_DIR/pcp
                 default  configuration  file  for  the  primary  logger  instance  launched from
                 assorted configuration files suitable for creating logs that may be subsequently
                 replayed with the PCP visualization and monitoring tools
                 Default  directory for PCP archive files for performance metric values collected
                 from the host hostname.
                 (or $PCP_LOG_DIR/pmlogger/hostname/pmlogger.log when  started  automatically  by
                 either  $PCP_RC_DIR/pcp  or  one  of  the pmlogger(1) monitoring scripts such as
                 all messages and diagnostics are directed here
                 contains ``hooks'' to enable automatic restart at system boot time


       Normally pmlogger creates a socket to receive control messages from pmlc(1) on  the  first
       available TCP/IP port numbered 4330 or higher.  The environment variable PMLOGGER_PORT may
       be used to specify an alternative starting port number.


       Environment variables with the prefix PCP_ are used to parameterize the file and directory
       names used by PCP.  On each installation, the file /etc/pcp.conf contains the local values
       for these variables.  The $PCP_CONF  variable  may  be  used  to  specify  an  alternative
       configuration file, as described in pcp.conf(4).


       PCPIntro(1),  pmcd(1),  pmdumplog(1),  pmlc(1), pmlogger_check(1), pcp.conf(4), pcp.env(4)
       and pmns(4).


       The archive logs are sufficiently precious that pmlogger will  not  truncate  an  existing
       physical file.  A message of the form
        __pmLogNewFile: "foo.index" already exists, not over-written
        __pmLogCreate: File exists
       indicates  this  situation  has  arisen.   You must explicitly remove the files and launch
       pmlogger again.

       There may be at most one primary pmlogger instance per monitored host; attempting to  bend
       this rule produces the error:
        pmlogger: there is already a primary pmlogger running

       Various   other   messages   relating   to  the  creation  and/or  deletion  of  files  in
       $PCP_TMP_DIR/pmlogger suggest a permission problem on this directory, or some feral  files
       have appeared therein.