Ubuntu Manpage: xymond_rrd - xymond worker module for updating Xymon RRD files

NAME

       xymond_rrd - xymond worker module for updating Xymon RRD files

SYNOPSIS

       xymond_channel --channel=status xymond_rrd [options]
       xymond_channel --channel=data xymond_rrd [options]

DESCRIPTION

       xymond_rrd  is  a  worker  module  for  xymond,  and  as  such  it is normally run via the
       xymond_channel(8) program. It receives "status" and "data" messages from xymond via stdin,
       and updates the RRD databases used to generate trend-graphs.

       Clients can send data to Xymon using both status- and data- messages. So you will normally
       run two instances of this module, once for the "status" channel and once  for  the  "data"
       channel.

       xymond_rrd  understands  data  sent  by the LARRD 0.43c client-side scripts (the so-called
       "bottom-feeder" scripts). So you still want to install the  LARRD  bottom-feeders  on  the
       clients you monitor.

       Note:  For  certain types of data, the RRD files used by Xymon are imcompatible with those
       generated by the Big Brother LARRD add-on.  See the COMPATIBILITY section below.

OPTIONS

--debug
Enable debugging output.

--rrddir=DIRECTORY
Defines the directory where the RRD-files are stored. xymond_rrd will use the
location pointed to by the XYMONRRDS environment if this option is not present.

--no-cache
xymond_rrd by default caches updates to the RRD files, to reduce the disk I/O
needed for storing the RRD data. Data is collected for a 30 minute period before
being committed to disk in one update. This option disables caching of the data,
so that data is stored on disk immediately.

--processor=FILENAME
xymond_rrd can send a parallel copy of all RRD updates to a single external process
as a stream on its STDIN. The data will be in a format similar to that used by
rrdupdate(1): <rrdtemplate> ts:<rrdvalue(s)> host <rrdparameters>

If the process exits, xymond_rrd will re-launch it.

--extra-script=FILENAME
Defines the script that is run to get the RRD data for tests that are not built
into xymond_rrd. You must also specify which tests are handled by the external
script in the --extra-tests option. This option can only be given once, so the
script must handle all of the external test-data. See the CUSTOM RRD DATA section
below. Note that this is NOT needed if your custom graphs are generated by the NCV
(Name Colon Value) module described below, it is only required for data where you
have a custom script to parse the status message and extract the data that is put
into the graph.

--extra-tests=TEST[,TEST]
List of testnames that are handled by the external script. See the CUSTOM RRD DATA
section below. Note that NCV graphs should NOT be listed here, but in the TEST2RRD
environment variable - see below.

--no-rrd
Disable the actual writing of RRD files. This is only really useful if you send all
of the data destined for the RRD files to an external processor (the --extra-script
or --processor options).

ENVIRONMENT

       TEST2RRD
              Defines  the  mapping  between  a  status-log  columnname and the corresponding RRD
              database format. This is normally defined in the xymonserver.cfg(5) file.

       XYMONRRDS
              Default directory where RRD files are stored.

       NCV_testname
              Defines the types of data collected by the "ncv" module in xymond_rrd.   See  below
              for more information.

       SPLITNCV_testname
              The  same  as  NCV_testname,  but  keeps  the data into separate files. That is, it
              creates one rrd file per "NAME : value" line found in the  status  message.  It  is
              useful when the list of NCV lines is varying.

       TRACKMAX
              Comma-separated  list  of  columnname for which you want to keep the maximum values
              along with the default average values. This only works
               for the NCV backend.

COLLECTED DATA

The following RRD-file datasets are generated by xymond_rrd:

la Records the CPU load average. Data is collected from the "cpu" status report.
Requires that a Xymon client is running on the monitored server.

disk Records the disk utilization. Data is collected from the "disk" status report.
Requires that a Xymon-compatible client is running on the monitored server.

memory Records memory- and swap-utilization. Data is collected from the "memory" status
report. If no "memory" status is reported, it will use the data from the Win32
client "cpu" status report to generate this dataset. Requires that a Xymon-
compatible client is running on the monitored server.

netstat
Records TCP and UDP statistics. Data is collected from the "netstat" status report;
however, this data is often sent via the Xymon "data" protocol, so there need not
be a "netstat" column visible on the Xymon display. To get these data, the LARRD
netstat bottom-feeder script must be running on the monitored server.

vmstat Records system performance metrics from the "vmstat" command. Data is collected
from the "vmstat" status report; however, this data is often sent via the Xymon
"data" protocol, so there need not be a "vmstat" column visible on the Xymon
display. To get these data, the LARRD vmstat bottom-feeder script must be running
on the monitored server.

tcp Response-time metrics from all of the Xymon network tests are recorded in the "tcp"
RRD.

apache Apache server performance metrics, taken from the "apache" data report. See the
description of the apache keyword in hosts.cfg(5) for details.

sendmail
Sendmail server performance metrics, taken from the "mailstats" output. To get
these data, the LARRD sendmail bottom-feeder script must be running on the
monitored server.

mailq Mail queue size. To get these data, the LARRD nmailq bottom-feeder script must be
running on the monitored server.

bea BEA Weblogic performance data. This is an experimental set of data collected from
BEA Weblogic servers via SNMP, by the "beastats" tool included with Xymon.

iishealth
IIS webserver performance data, collected by the "iishealth" script. This script
is a client-side add-on available from the www.deadcat.net archive.

temperature
Temperature data, collected with the temperature script from www.deadcat.net. To
get these data, the temperature script must be running on the monitored server.

ntpstat
Tracks the deviation between the local system time and an NTP server, using the
output from the "ntpq -c rv" command. A simple script to collect these data is
included in the Xymon contrib/ directory.

citrix Tracks the number of active sessions on a Citrix server using the "query session"
command. An extension for the BBNT client that generates data for this graph is in
the Xymon contrib/ directory.

CUSTOM RRD DATA IN NAME-COLON-VALUE (NCV) FORMAT

Many data-collection scripts report data in the form "NAME : value" or "NAME = value". So
a generic module in xymond_rrd allows for easy tracking of this type of data.

The "ncv" module will automatically detect all occurrences of a "NAME : value" or "NAME =
value" string in a status message, and generate an RRD file holding all of the name/value
data found in the message (unless you use SPLITNCV, see above). The colon- or equal-sign
must be present - if there is only whitespace, this module will fail.

Only the valid letters (A-Z, a-z) and digits (0-9) are used in the dataset names;
whitespace and other characters are stripped off automatically. Only the first 19
characters of a dataset name are used (this is an RRD limitation). Underscore '_' is not
allowed, even though RRDtool permits this, and will be stripped from the name.

When using the alternative SPLITNCV_testname, the dataset name is not limited in length,
and non-valid characters are changed to underscores instead of being stripped off. The
dataset inside the resulting rrd file is always "lambda".

Note that each "NAME : value" must be on a line by itself. If you have a custom script
generating the status- or data-message that is fed into the NCV handler, make sure it
inserts a newline before each of the data-items you want to track.

Any lines in the status message prepended with a "" will be skipped by
the module. This can be used to prevent unneeded RRD files from an existing dataset from
being created.

A line prepended with a "" will be ignored, along with all
subsequent lines until a line starting with "" is found, at which
point processing will resume. This can be used to ignore explanatory or other text with a
mostly-ncv message.

"" can be used to ignore certain text at the beginning of a line, up
until a closing '</-->' tag on the same line, at which point the line will continue to be
processed as usual. Wrapping is not supported; but skipstart/skipend can be used to handle
multiple lines.

A bare "" on its own line will stop further NCV processing of that
message.

All of these ncv_ terms are case-sensitive. Note that if you have full control over your
NCV output, it is most efficient to have NCV data near the top of your message and use
"" once your data is complete.

To enable the ncv module for a status, add a "COLUMNNAME=ncv" to the TEST2RRD setting and
the COLUMNNAME to the GRAPHS setting in xymonserver.cfg(5) , then restart Xymon. Xymon
will now send all status-messages for the column COLUMNNAME through the xymond_rrd ncv-
handler.

The name of the RRD file will be COLUMNNAME.rrd. When using SPLITNCV, the name of the RRD
file will be COLUMNAME,DATASETNAME.rrd.

By default, all of the datasets are generated as the RRD type "DERIVE" which works for all
types of monotonically increasing counters. If you have data that are of the type GAUGE,
you can override the default via an environment variable NCV_COLUMNNAME (or
SPLITNCV_COLUMNAME).

E.g. if you are using the bb-mysqlstatus script from www.deadcat.net to collect data about
your MySQL server, it generates a report in the column called "mysql". One data item is
the average number of queries/second, which must be logged in the RRD file as type
"GAUGE". To do that, add the following to xymonserver.cfg:
NCV_mysql="Queriespersecondavg:GAUGE"
If you have multiple datasets that you myst define, add them to the environment variable
separated by commas, e.g.
NCV_mysql="Uptime:NONE,Queriespersecondavg:GAUGE"

The dataset type "NONE" used above causes xymond_rrd to ignore this data, it is not
included in the RRD file.

You can use "*" as the dataset name to match all datasets not listed. E.g.
NCV_weather="Rain:DERIVE,*:GAUGE"
will cause the "Rainfall" dataset to be of type DERIVE, and all others of type GAUGE. If
you want to track only a few of the variables in your data, you can use "*:NONE" to drop
any dataset not explicitly listed.

For a more detailed "how to" description, see the on-line HTML documentation of "How to
create graph custom data" available in the Help menu section on your Xymon server.

SENDING METRIC DATA TO AN ADDITIONAL PROCESS

       xymond_rrd  provides  a  mechanism  to  send  a  copy  of isolated metric data to a single
       external processor for further processing. This can be used to  inject  metric  data  that
       xymond_rrd  has  prepared into other storage systems, such as OpenTSDB, graphite, etc. The
       data is printed in a format nearly suitable for injection using  rrdupdate(1)  and  easily
       transformable  to  other  formats.  If  the  process  exits,  xymond_rrd will re-launch it
       automatically.

CUSTOM RRD DATA VIA SCRIPTS

xymond_rrd provides a simple mechanism for adding custom graphs to the set of data
collected on your Xymon server. By adding the "--extra-script" and "--extra-tests"
options, data reported to Xymon from selected tests are passed to an external script,
which can define the RRD data-sets to store in an RRD file.

NOTE: For performance reasons, you should not use this mechanism for large amounts of
data. The overhead involved in storing the received message to disk and launching the
script is significantly larger than the normal xymond_rrd overhead. So if you have a large
number of reports for a given test, you should consider implementing it in C and including
it in the xymond_rrd tool or writing a separate stream listener that injects appropriate
"trends" data messages back to xymond.

Apart from writing the script, You must also add a section to graphs.cfg(5) so that
showgraph.cgi(1) knows how to generate the graph from the data stored in the RRD file. To
make the graphs actually show up on the status-page and/or the "trends" page, add the name
of the new graph to the TEST2RRD and/or GRAPHS setting in xymonserver.cfg(5).

The script is invoked for each message that arrives, where the test-name matches one of
the testnames given in the "--extra-tests" option. The script receives three command-line
parameters:

Hostname
The name of the host reporting the data.

Testname
The name of the test being reported.

Filename
File containing the data that was reported. This file is generated for you by
xymond_rrd, and is also deleted automatically after your script is finished with
it.

The script must process the data that is reported, and generate the following output:

RRD data-set definitions
For each dataset that the RRD file holds, a line beginning with "DS:" must be
output. If multiple data-sets are used, print one line for each dataset.
Data-set definitions are described in the rrdcreate(1) documentation, but a common
definition for e.g. tracking the number of users logged on would be
"DS:users:GAUGE:600:0:U". "users" is the name of the dataset, "GAUGE" is the
datatype, "600" is the longest time allowed between updates for the data to be
valid, "0" is the minimum value, and "U" is the maximum value (a "U" means
"unknown").

RRD filename
The name of the RRD file where the data is stored. Note that Xymon stores all RRD
files in host-specific directories, so unlike LARRD you should not include the
hostname in the name of the RRD file.

RRD values
One line, with all of the data values collected by the script. Data-items are
colon-delimited and must appear in the same sequence as your data-set definitions,
e.g. if your RRD has two datasets with the values "5" and "0.4" respectively, then
the script must output "5:0.4" as the RRD values.
In some cases it may be useful to define a dataset even though you will not always
have data for it. In that case, use "U" (unknown) for the value.

If you want to store the data in multiple RRD files, the script can just print out
more sequences of data-set definitions, RRD filenames and RRD values. If the data-
set definitions are identical to the previous definition, you need not print the
data-set definitions again - just print a new RRD filename and value.

The following sample script for tracking weather data shows how to use this mechanism. It
assumes the status message include lines like these:

green Weather in Copenhagen is FAIR

Temperature: 21 degrees Celsius
Wind: 4 m/s
Humidity: 72 %
Rainfall: 5 mm since 6:00 AM

A shell-script to track all of these variables could be written like this:

#!/bin/sh

# Input parameters: Hostname, testname (column), and messagefile
HOSTNAME="$1"
TESTNAME="$2"
FNAME="$3"

if [ "$TESTNAME" = "weather" ]
then
# Analyze the message we got
TEMP=`grep "^Temperature:" $FNAME | awk '{print $2}'`
WIND=`grep "^Wind:" $FNAME | awk '{print $2}'`
HMTY=`grep "^Humidity:" $FNAME | awk '{print $2}'`
RAIN=`grep "^Rainfall:" $FNAME | awk '{print $2}'`

# The RRD dataset definitions
echo "DS:temperature:GAUGE:600:-30:50"
echo "DS:wind:GAUGE:600:0:U"
echo "DS:humidity:GAUGE:600:0:100"
echo "DS:rainfall:DERIVE:600:0:100"

# The filename
echo "weather.rrd"

# The data
echo "$TEMP:$WIND:$HMTY:$RAIN"
fi

exit 0

COMPATIBILITY

       Some of the RRD files generated by xymond_rrd are incompatible with the files generated by
       the Big Brother LARRD add-on:

       vmstat The vmstat files with data from Linux based systems are  incompatible  due  to  the
              addition  of  a  number  of  new  data-items  that  LARRD  0.43 do not collect, but
              xymond_rrd does. This is due to  changes  in  the  output  from  the  Linux  vmstat
              command, and changes in the way e.g.  system load metrics are reported.

       netstat
              All  netstat  files  from LARRD 0.43 are incompatible with xymond_rrd.  The netstat
              data collected by LARRD is  quite  confusing:  For  some  types  of  systems  LARRD
              collects  packet-counts,  for  others  it  collects byte- counts. xymond_rrd uses a
              different RRD file-format with separate counters for packets and bytes  and  tracks
              whatever data the system is reporting.