Provided by: pcp_3.8.12ubuntu1_amd64 
NAME
pmlogger - create archive log for performance metrics
SYNOPSIS
pmlogger [-c configfile] [-h host] [-l logfile] [-L] [-m note] [-n pmnsfile] [-P] [-r] [-s
endsize] [-t interval] [-T endtime] [-u] [-U username] [-v volsize] [-V version] [-x fd]
archive
DESCRIPTION
pmlogger creates the archive logs of performance metric values that may be ``played back''
by other Performance Co-Pilot (see PCPIntro(1)) tools. These logs form the basis of the
VCR paradigm and retrospective performance analysis services common to the PCP toolkit.
The mandatory argument archive is the base name for the physical files that constitute an
archive log.
The -V option specifies the version for the archive that is generated. By default a
version 2 archive is generated, and the only value currently supported for version is 2.
Unless directed to another host by the -h option, pmlogger will contact the Performance
Metrics Collector Daemon (PMCD) on the local host and use that as the source of the metric
values to be logged.
To support the required flexibility and control over what is logged and when, pmlogger
maintains an independent two level logging state for each instance of each performance
metric. At the first (mandatory) level, logging is allowed to be on (with an associated
interval between samples), or off or maybe. In the latter case, the second (advisory)
level logging is allowed to be on (with an associated interval between samples), or off.
The mandatory level allows universal specification that some metrics must be logged, or
must not be logged. The default state for all instances of all metrics when pmlogger
starts is mandatory maybe and advisory off.
Use pmlc(1) to interrogate and change the logging state once pmlogger is running.
If a metric's state is mandatory (on or off) and a request is made to change it to
mandatory maybe, the new state is mandatory maybe and advisory off. If a metric's state
is already advisory (on or off) and a request is made to change it to mandatory maybe, the
current state is retained.
It is not possible for pmlogger to log specific instances of a metric and all instances of
the same metric concurrently. If specific instances are being logged and a request to log
all instances is made, then all instances of the metric will be logged according to the
new request, superseding any prior logging request for the metric. A request to log all
instances of a metric will supersede any previous request to log all instances. A request
to log specific instances of a metric when all instances are already being logged is
refused. To do this one must turn off logging for all instances of the metric first. In
each case, the validity of the request is checked first; for example a request to change a
metric's logging state to advisory on when it is currently mandatory off is never
permitted (it is necessary to change the state to mandatory maybe first).
Optionally, each system running pmcd(1) may also be configured to run a ``primary''
pmlogger instance. Like pmcd(1), this pmlogger instance is launched by $PCP_RC_DIR/pcp,
and is affected by the files $PCP_SYSCONF_DIR/pmlogger (use chkconfig(1M) to activate or
disable the primary pmlogger instance), $PCP_SYSCONF_DIR/pmlogger/pmlogger.options
(command line options passed to the primary pmlogger) and
$PCP_SYSCONF_DIR/pmlogger/config.default (the default initial configuration file for the
primary pmlogger).
The primary pmlogger instance is identified by the -P option. There may be at most one
``primary'' pmlogger instance on each system with an active pmcd(1). The primary pmlogger
instance (if any) must be running on the same host as the pmcd(1) to which it connects, so
the -h and -P options are mutually exclusive.
When launched as a non-primary instance, pmlogger will exit immediately if the
configuration file causes no metric logging to be scheduled. The -L option overrides this
behavior, and causes a non-primary pmlogger instance to ``linger'', presumably pending
some future dynamic re-configuration and state change via pmlc(1). pmlogger will also
linger without the -L option being used if all the metrics to be logged are logged as once
only metrics. When the once only metrics have been logged, a warning message will be
generated stating that the event queue is empty and no more events will be scheduled.
By default all diagnostics and errors from pmlogger are written to the file pmlogger.log
in the directory where pmlogger is launched. The -l option may be used to override the
default behavior. If the log file cannot be created or is not writable, output is written
to standard error instead.
If specified, the -s option instructs pmlogger to terminate after a certain size in
records, bytes or time units has been accumulated. If endsize is an integer then endsize
records will be written to the log. If endsize is an integer suffixed by b or bytes then
endsize bytes of the archive data will be written out (note, however, that archive log
record boundaries will not be broken and so this limit may be slightly surpassed). Other
viable file size units include: K, Kb, Kbyte, Kilobyte for kilobytes and M, Mb, Mbyte,
Megabyte for megabytes and G, Gb, Gbyte, Gigabyte for gigabytes. These units may be
optionally suffixed by an s and may be of mixed case. Alternatively endsize may be an
integer or a floating point number suffixed using a time unit as described in PCPIntro(1)
for the interval argument (to the standard PCP -t command line option).
Some examples of different formats:
-s 100
-s 100bytes
-s 100K
-s 100Mb
-s 10Gbyte
-s 10mins
-s 1.5hours
The default is for pmlogger to run forever.
The -r option causes the size of the physical record(s) for each group of metrics and the
expected contribution of the group to the size of the PCP archive for one full day of
collection to be reported in the log file. This information is reported the first time
each group is successfully written to the archive.
The -U option specifies the user account under which to run pmlogger. The default is the
current user account for interactive use. When run as a daemon, the unprivileged "pcp"
account is used in current versions of PCP, but in older versions the superuser account
("root") was used by default.
The log file is potentially a multi-volume data set, and the -v option causes pmlogger to
start a new volume after a certain size in records, bytes, or time units has been
accumulated for the current volume. The format of this size specification is identical to
that of the -s option (see above). The default is for pmlogger to create a single volume
log. Additional volume switches can also be forced asynchronously by either using pmlc(1)
or sending pmlogger a SIGHUP signal (see below). Note, if a scheduled volume switch is in
operation due to the -v option, then its counters will be reset after an asynchronous
switch.
Independent of any -v option, each volume of an archive is limited to no more than 2^31
bytes, so pmlogger will automatically create a new volume for the archive before this
limit is reached.
Normally pmlogger operates on the distributed Performance Metrics Name Space (PMNS),
however if the -n option is specified an alternative local PMNS is loaded from the file
pmnsfile.
Under normal circumstances, pmlogger will run forever (except for a -s option or a
termination signal). The -T option may be used to limit the execution time using the
format of time as prescribed by PCPIntro(1).
Some examples of different formats:
-T 10mins
-T '@ 11:30'
From this it can be seen that -T 10mins and -s 10mins perform identical actions.
When pmlogger receives a SIGHUP signal, the current volume of the log is closed, and a new
volume is opened. This mechanism (or the alternative mechanism via pmlc(1)) may be used
to manage the growth of the log files - once a log volume is closed, that file may be
archived without ill-effect on the continued operation of pmlogger. See also the -v
option above.
The buffers for the current log may be flushed to disk using the flush command of pmlc(1),
or by sending pmlogger a SIGUSR1 signal or by using the -u option (the latter forces every
log write to be unbuffered). This is useful when the log needs to be read while pmlogger
is still running.
When launched with the -x option, pmlogger will accept asynchronous control requests on
the file descriptor fd. This option is only expected to be used internally by PCP
applications that support ``live record mode''.
The -m option allows the string note to be appended to the map file for this instance of
pmlogger in the $PCP_TMP_DIR/pmlogger directory. This is currently used internally to
document the file descriptor (fd) when the -x option is used, or to indicate that this
pmlogger instance was started under the control of pmlogger_check(1).
CONFIGURATION FILE SYNTAX
The configuration file may be specified with the -c option. If it is not, configuration
specifications are read from standard input.
If configfile does not exist, then a search is made in the directory
$PCP_SYSCONF_DIR/pmlogger for a file of the same name, and if found that file is used,
e.g. if config.mumble does not exist in the current directory and the file
$PCP_SYSCONF_DIR/pmlogger/config.mumble does exist, then -c config.mumble and -c
$PCP_SYSCONF_DIR/pmlogger/config.mumble are equivalent.
The syntax for the configuration file is as follows.
1. Words are separated by white space (space, tab or newline).
2. The symbol ``#'' (hash) introduces a comment, and all text up to the next newline
is ignored.
3. Keywords (shown in bold below) must appear literally (i.e. in lower case).
4. Each specification begins with the optional keyword log, followed by one of the
states mandatory on, mandatory off, mandatory maybe, advisory on or advisory off.
5. For the on states, a logging interval must follow using the syntax ``once'', or
``default'', or ``every N timeunits'', or simply ``N timeunits'' - N is an unsigned
integer, and timeunits is one of the keywords msec, millisecond, sec, second, min,
minute, hour or the plural form of one of the above.
Internal limitations require the interval to be smaller than (approximately) 74
hours. An interval value of zero is a synonym for once. An interval of default
means to use the default logging interval of 60 seconds; this default value may be
changed to interval with the -t command line option.
The interval argument follows the syntax described in PCPIntro(1), and in the
simplest form may be an unsigned integer (the implied units in this case are
seconds).
6. Following the state and possible interval specifications comes a ``{'', followed by
a list of one or more metric specifications and a closing ``}''. The list is white
space (or comma) separated. If there is only one metric specification in the list,
the braces are optional.
7. A metric specification consists of a metric name optionally followed by a set of
instance names. The metric name follows the standard PCP naming conventions, see
pmns(5), and if the metric name is a non-leaf node in the PMNS (see pmns(5)), then
pmlogger will recursively descend the PMNS and apply the logging specification to
all descendent metric names that are leaf nodes in the PMNS. The set of instance
names is a ``['', followed by a list of one or more space (or comma) separated
names, numbers or strings, and a closing ``]''. Elements in the list that are
numbers are assumed to be internal instance identifiers, other elements are assumed
to be external instance identifiers - see pmGetInDom(3) for more information.
If no instances are given, then the logging specification is applied to all
instances of the associated metric.
8. There may be an arbitrary number of logging specifications.
9. Following all of the logging specifications, there may be an optional access
control section, introduced by the literal token [access]. Thereafter come access
control rules of the form ``allow hostlist : operation ;'' and ``disallow hostlist
: operation ;''.
The base operations are advisory, mandatory and enquire. In all other aspects,
these access control statements follow the syntactic and semantic rules defined for
the access control mechanisms used by PMCD and are fully documented in pmcd(1).
EXAMPLES
For each PCP utility, there is a sample pmlogger configuration file that could be used to
create an archive log suitable for replaying with that tool (i.e. includes all of the
performance metrics used by the tool). For a tool named foo this configuration file is
located in $PCP_SYSCONF_DIR/pmlogger/config.foo.
The following is a simple default configuration file for a primary pmlogger instance, and
demonstrates most of the capabilities of the configuration specification language.
log mandatory on once { hinv.ncpu hinv.ndisk }
log mandatory on every 10 minutes {
disk.all.write
disk.all.read
network.interface.in.packets [ "et0" ]
network.interface.out.packets [ "et0" ]
nfs.server.reqs [ "lookup" "getattr" "read" "write" ]
}
log advisory on every 30 minutes {
environ.temp
pmcd.pdu_in.total
pmcd.pdu_out.total
}
[access]
disallow * : all except enquire;
allow localhost : mandatory, advisory;
FILES
archive.meta
metadata (metric descriptions, instance domains, etc.) for the archive log
archive.0 initial volume of metrics values (subsequent volumes have suffixes 1, 2, ...)
archive.index
temporal index to support rapid random access to the other files in the archive
log
$PCP_TMP_DIR/pmlogger
pmlogger maintains the files in this directory as the map between the process id
of the pmlogger instance and the IPC port that may be used to control each
pmlogger instance (as used by pmlc(1))
$PCP_SYSCONF_DIR/pmlogger/config.default
default configuration file for the primary logger instance launched from
$PCP_RC_DIR/pcp
$PCP_SYSCONF_DIR/pmlogger/config.*
assorted configuration files suitable for creating logs that may be subsequently
replayed with the PCP visualization and monitoring tools
$PCP_LOG_DIR/pmlogger/hostname
Default directory for PCP archive files for performance metric values collected
from the host hostname.
./pmlogger.log
(or $PCP_LOG_DIR/pmlogger/hostname/pmlogger.log when started automatically by
either $PCP_RC_DIR/pcp or one of the pmlogger(1) monitoring scripts such as
pmlogger_check(1))
all messages and diagnostics are directed here
$PCP_RC_DIR/pcplocal
contains ``hooks'' to enable automatic restart at system boot time
ENVIRONMENT
Normally pmlogger creates a socket to receive control messages from pmlc(1) on the first
available TCP/IP port numbered 4330 or higher. The environment variable PMLOGGER_PORT may
be used to specify an alternative starting port number.
PCP ENVIRONMENT
Environment variables with the prefix PCP_ are used to parameterize the file and directory
names used by PCP. On each installation, the file /etc/pcp.conf contains the local values
for these variables. The $PCP_CONF variable may be used to specify an alternative
configuration file, as described in pcp.conf(5).
SEE ALSO
PCPIntro(1), pmcd(1), pmdumplog(1), pmlc(1), pmlogger_check(1), pcp.conf(5), pcp.env(5)
and pmns(5).
DIAGNOSTICS
The archive logs are sufficiently precious that pmlogger will not truncate an existing
physical file. A message of the form
__pmLogNewFile: "foo.index" already exists, not over-written
__pmLogCreate: File exists
indicates this situation has arisen. You must explicitly remove the files and launch
pmlogger again.
There may be at most one primary pmlogger instance per monitored host; attempting to bend
this rule produces the error:
pmlogger: there is already a primary pmlogger running
Various other messages relating to the creation and/or deletion of files in
$PCP_TMP_DIR/pmlogger suggest a permission problem on this directory, or some feral files
have appeared therein.