bionic (8) xymond_rrd.8.gz

Provided by: xymon_4.3.28-3build1_amd64 bug

NAME

       xymond_rrd - xymond worker module for updating Xymon RRD files

SYNOPSIS

       xymond_channel --channel=status xymond_rrd [options]
       xymond_channel --channel=data xymond_rrd [options]

DESCRIPTION

       xymond_rrd  is  a  worker  module  for  xymond,  and as such it is normally run via the xymond_channel(8)
       program. It receives "status" and "data" messages from xymond via stdin, and updates  the  RRD  databases
       used to generate trend-graphs.

       Clients  can  send  data  to  Xymon  using  both status- and data- messages. So you will normally run two
       instances of this module, once for the "status" channel and once for the "data" channel.

       xymond_rrd understands data sent by the LARRD 0.43c client-side scripts  (the  so-called  "bottom-feeder"
       scripts). So you still want to install the LARRD bottom-feeders on the clients you monitor.

       Note: For certain types of data, the RRD files used by Xymon are imcompatible with those generated by the
       Big Brother LARRD add-on.  See the COMPATIBILITY section below.

OPTIONS

       --debug
              Enable debugging output.

       --rrddir=DIRECTORY
              Defines the directory where the RRD-files are stored. xymond_rrd will use the location pointed  to
              by the XYMONRRDS environment if this option is not present.

       --no-cache
              xymond_rrd  by  default caches updates to the RRD files, to reduce the disk I/O needed for storing
              the RRD data. Data is collected for a 30 minute period before  being  committed  to  disk  in  one
              update.  This option disables caching of the data, so that data is stored on disk immediately.

       --processor=FILENAME
              xymond_rrd can send a parallel copy of all RRD updates to a single external process as a stream on
              its STDIN. The data will be in a format similar to that used by  rrdupdate(1):       <rrdtemplate>
              ts:<rrdvalue(s)> host <rrdparameters>

              If the process exits, xymond_rrd will re-launch it.

       --extra-script=FILENAME
              Defines  the  script that is run to get the RRD data for tests that are not built into xymond_rrd.
              You must also specify which tests are handled by the external script in the --extra-tests  option.
              This  option  can only be given once, so the script must handle all of the external test-data. See
              the CUSTOM RRD DATA section below. Note that  this  is  NOT  needed  if  your  custom  graphs  are
              generated by the NCV (Name Colon Value) module described below, it is only required for data where
              you have a custom script to parse the status message and extract the data that  is  put  into  the
              graph.

       --extra-tests=TEST[,TEST]
              List  of testnames that are handled by the external script. See the CUSTOM RRD DATA section below.
              Note that NCV graphs should NOT be listed here, but in the TEST2RRD  environment  variable  -  see
              below.

       --no-rrd
              Disable  the  actual  writing of RRD files. This is only really useful if you send all of the data
              destined for the RRD files to an external processor (the --extra-script or --processor options).

ENVIRONMENT

       TEST2RRD
              Defines the mapping between a status-log columnname and the  corresponding  RRD  database  format.
              This is normally defined in the xymonserver.cfg(5) file.

       XYMONRRDS
              Default directory where RRD files are stored.

       NCV_testname
              Defines  the  types  of  data  collected  by  the  "ncv" module in xymond_rrd.  See below for more
              information.

       SPLITNCV_testname
              The same as NCV_testname, but keeps the data into separate files. That is, it creates one rrd file
              per  "NAME  :  value" line found in the status message. It is useful when the list of NCV lines is
              varying.

       TRACKMAX
              Comma-separated list of columnname for which you want to keep the maximum values  along  with  the
              default average values. This only works
               for the NCV backend.

COLLECTED DATA

       The following RRD-file datasets are generated by xymond_rrd:

       la     Records  the  CPU  load  average.  Data is collected from the "cpu" status report. Requires that a
              Xymon client is running on the monitored server.

       disk   Records the disk utilization. Data is collected from the "disk" status  report.  Requires  that  a
              Xymon-compatible client is running on the monitored server.

       memory Records  memory-  and  swap-utilization.  Data is collected from the "memory" status report. If no
              "memory" status is reported, it will use the data from the Win32 client  "cpu"  status  report  to
              generate this dataset. Requires that a Xymon-compatible client is running on the monitored server.

       netstat
              Records  TCP and UDP statistics. Data is collected from the "netstat" status report; however, this
              data is often sent via the Xymon "data" protocol, so there need not be a "netstat" column  visible
              on the Xymon display. To get these data, the LARRD netstat bottom-feeder script must be running on
              the monitored server.

       vmstat Records system performance metrics from the "vmstat" command.  Data is collected from the "vmstat"
              status  report;  however, this data is often sent via the Xymon "data" protocol, so there need not
              be a "vmstat" column visible on the Xymon display. To get these data,  the  LARRD  vmstat  bottom-
              feeder script must be running on the monitored server.

       tcp    Response-time metrics from all of the Xymon network tests are recorded in the "tcp" RRD.

       apache Apache server performance metrics, taken from the "apache" data report. See the description of the
              apache keyword in hosts.cfg(5) for details.

       sendmail
              Sendmail server performance metrics, taken from the "mailstats" output. To  get  these  data,  the
              LARRD sendmail bottom-feeder script must be running on the monitored server.

       mailq  Mail  queue  size. To get these data, the LARRD nmailq bottom-feeder script must be running on the
              monitored server.

       bea    BEA Weblogic performance data. This is an experimental set of data  collected  from  BEA  Weblogic
              servers via SNMP, by the "beastats" tool included with Xymon.

       iishealth
              IIS webserver performance data, collected by the "iishealth" script.  This script is a client-side
              add-on available from the www.deadcat.net archive.

       temperature
              Temperature data, collected with the temperature script from www.deadcat.net. To get  these  data,
              the temperature script must be running on the monitored server.

       ntpstat
              Tracks  the  deviation  between the local system time and an NTP server, using the output from the
              "ntpq -c rv" command.  A simple script to collect these data is included  in  the  Xymon  contrib/
              directory.

       citrix Tracks  the  number  of  active  sessions on a Citrix server using the "query session" command. An
              extension for the BBNT client that generates  data  for  this  graph  is  in  the  Xymon  contrib/
              directory.

CUSTOM RRD DATA IN NAME-COLON-VALUE (NCV) FORMAT

       Many  data-collection  scripts  report  data  in  the form "NAME : value" or "NAME = value". So a generic
       module in xymond_rrd allows for easy tracking of this type of data.

       The "ncv" module will automatically detect all occurrences of a "NAME : value" or "NAME =  value"  string
       in  a  status  message,  and generate an RRD file holding all of the name/value data found in the message
       (unless you use SPLITNCV, see above). The colon- or equal-sign  must  be  present  -  if  there  is  only
       whitespace, this module will fail.

       Only  the  valid  letters (A-Z, a-z) and digits (0-9) are used in the dataset names; whitespace and other
       characters are stripped off automatically. Only the first 19 characters of a dataset name are used  (this
       is  an  RRD  limitation).  Underscore  '_'  is not allowed, even though RRDtool permits this, and will be
       stripped from the name.

       When using the alternative SPLITNCV_testname, the dataset name is not limited in  length,  and  non-valid
       characters are changed to underscores instead of being stripped off. The dataset inside the resulting rrd
       file is always "lambda".

       Note that each "NAME : value" must be on a line by itself. If you have a  custom  script  generating  the
       status-  or  data-message that is fed into the NCV handler, make sure it inserts a newline before each of
       the data-items you want to track.

       Any lines in the status message prepended with a "<!-- ncv_skip -->" will be skipped by the module.  This
       can be used to prevent unneeded RRD files from an existing dataset from being created.

       A line prepended with a "<!-- ncv_skipstart -->" will be ignored, along with all subsequent lines until a
       line starting with "<!-- ncv_skipend -->" is found, at which point processing will resume.  This  can  be
       used to ignore explanatory or other text with a mostly-ncv message.

       "<!--  ncv_ignore  -->" can be used to ignore certain text at the beginning of a line, up until a closing
       '</-->' tag on the same line, at which point the line will continue to be processed as usual. Wrapping is
       not supported; but skipstart/skipend can be used to handle multiple lines.

       A bare "<!-- ncv_end -->" on its own line will stop further NCV processing of that message.

       All  of  these ncv_ terms are case-sensitive. Note that if you have full control over your NCV output, it
       is most efficient to have NCV data near the top of your message and use "<!-- ncv_end -->" once your data
       is complete.

       To  enable the ncv module for a status, add a "COLUMNNAME=ncv" to the TEST2RRD setting and the COLUMNNAME
       to the GRAPHS setting in xymonserver.cfg(5) , then restart Xymon. Xymon will now send all status-messages
       for the column COLUMNNAME through the xymond_rrd ncv-handler.

       The  name  of  the RRD file will be COLUMNNAME.rrd. When using SPLITNCV, the name of the RRD file will be
       COLUMNAME,DATASETNAME.rrd.

       By default, all of the datasets are generated as the RRD type "DERIVE"  which  works  for  all  types  of
       monotonically  increasing  counters.  If  you  have data that are of the type GAUGE, you can override the
       default via an environment variable NCV_COLUMNNAME (or SPLITNCV_COLUMNAME).

       E.g. if you are using the bb-mysqlstatus script from www.deadcat.net to collect  data  about  your  MySQL
       server,  it  generates  a  report  in  the  column called "mysql". One data item is the average number of
       queries/second, which must be logged in the RRD file as type "GAUGE". To do that, add  the  following  to
       xymonserver.cfg:
           NCV_mysql="Queriespersecondavg:GAUGE"
       If  you  have  multiple  datasets that you myst define, add them to the environment variable separated by
       commas, e.g.
           NCV_mysql="Uptime:NONE,Queriespersecondavg:GAUGE"

       The dataset type "NONE" used above causes xymond_rrd to ignore this data, it is not included in  the  RRD
       file.

       You can use "*" as the dataset name to match all datasets not listed.  E.g.
           NCV_weather="Rain:DERIVE,*:GAUGE"
       will  cause  the  "Rainfall"  dataset  to be of type DERIVE, and all others of type GAUGE. If you want to
       track only a few of the variables in your data, you can use "*:NONE" to drop any dataset  not  explicitly
       listed.

       For  a  more  detailed  "how  to" description, see the on-line HTML documentation of "How to create graph
       custom data" available in the Help menu section on your Xymon server.

SENDING METRIC DATA TO AN ADDITIONAL PROCESS

       xymond_rrd provides a mechanism to send a copy of isolated metric data to a single external processor for
       further  processing.  This  can  be  used  to  inject metric data that xymond_rrd has prepared into other
       storage systems, such as OpenTSDB, graphite, etc. The data is printed in a  format  nearly  suitable  for
       injection  using rrdupdate(1) and easily transformable to other formats. If the process exits, xymond_rrd
       will re-launch it automatically.

CUSTOM RRD DATA VIA SCRIPTS

       xymond_rrd provides a simple mechanism for adding custom graphs to the set  of  data  collected  on  your
       Xymon  server.  By  adding  the "--extra-script" and "--extra-tests" options, data reported to Xymon from
       selected tests are passed to an external script, which can define the RRD data-sets to store  in  an  RRD
       file.

       NOTE:  For performance reasons, you should not use this mechanism for large amounts of data. The overhead
       involved in storing the received message to disk and launching the script is  significantly  larger  than
       the  normal  xymond_rrd  overhead.  So if you have a large number of reports for a given test, you should
       consider implementing it in C and including it in the  xymond_rrd  tool  or  writing  a  separate  stream
       listener that injects appropriate "trends" data messages back to xymond.

       Apart  from  writing  the  script,  You must also add a section to graphs.cfg(5) so that showgraph.cgi(1)
       knows how to generate the graph from the data stored in the RRD file.  To make the graphs  actually  show
       up  on  the  status-page  and/or  the "trends" page, add the name of the new graph to the TEST2RRD and/or
       GRAPHS setting in xymonserver.cfg(5).

       The script is invoked for each message that arrives, where the test-name matches  one  of  the  testnames
       given in the "--extra-tests" option. The script receives three command-line parameters:

       Hostname
              The name of the host reporting the data.

       Testname
              The name of the test being reported.

       Filename
              File  containing  the data that was reported. This file is generated for you by xymond_rrd, and is
              also deleted automatically after your script is finished with it.

       The script must process the data that is reported, and generate the following output:

       RRD data-set definitions
              For each dataset that the RRD file holds, a line beginning with "DS:" must be output.  If multiple
              data-sets are used, print one line for each dataset.
              Data-set  definitions are described in the rrdcreate(1) documentation, but a common definition for
              e.g. tracking the number of users logged on would be "DS:users:GAUGE:600:0:U". "users" is the name
              of the dataset, "GAUGE" is the datatype, "600" is the longest time allowed between updates for the
              data to be valid, "0" is the minimum value, and "U" is the maximum value (a "U" means "unknown").

       RRD filename
              The name of the RRD file where the data is stored. Note that Xymon stores all RRD files  in  host-
              specific  directories,  so unlike LARRD you should not include the hostname in the name of the RRD
              file.

       RRD values
              One line, with all of the data values collected by the script. Data-items are colon-delimited  and
              must  appear  in the same sequence as your data-set definitions, e.g. if your RRD has two datasets
              with the values "5" and "0.4" respectively, then the script must output "5:0.4" as the RRD values.
              In some cases it may be useful to define a dataset even though you will not always have  data  for
              it. In that case, use "U" (unknown) for the value.

              If  you want to store the data in multiple RRD files, the script can just print out more sequences
              of data-set definitions, RRD filenames and RRD values. If the data-set definitions  are  identical
              to  the  previous definition, you need not print the data-set definitions again - just print a new
              RRD filename and value.

       The following sample script for tracking weather data shows how to use this  mechanism.  It  assumes  the
       status message include lines like these:

              green Weather in Copenhagen is FAIR

              Temperature: 21 degrees Celsius
              Wind: 4 m/s
              Humidity: 72 %
              Rainfall: 5 mm since 6:00 AM

       A shell-script to track all of these variables could be written like this:

              #!/bin/sh

              # Input parameters: Hostname, testname (column), and messagefile
              HOSTNAME="$1"
              TESTNAME="$2"
              FNAME="$3"

              if [ "$TESTNAME" = "weather" ]
              then
                   # Analyze the message we got
                   TEMP=`grep "^Temperature:" $FNAME | awk '{print $2}'`
                   WIND=`grep "^Wind:" $FNAME | awk '{print $2}'`
                   HMTY=`grep "^Humidity:" $FNAME | awk '{print $2}'`
                   RAIN=`grep "^Rainfall:" $FNAME | awk '{print $2}'`

                   # The RRD dataset definitions
                   echo "DS:temperature:GAUGE:600:-30:50"
                   echo "DS:wind:GAUGE:600:0:U"
                   echo "DS:humidity:GAUGE:600:0:100"
                   echo "DS:rainfall:DERIVE:600:0:100"

                   # The filename
                   echo "weather.rrd"

                   # The data
                   echo "$TEMP:$WIND:$HMTY:$RAIN"
              fi

              exit 0

COMPATIBILITY

       Some  of  the  RRD  files  generated  by  xymond_rrd are incompatible with the files generated by the Big
       Brother LARRD add-on:

       vmstat The vmstat files with data from Linux based systems are incompatible due  to  the  addition  of  a
              number  of  new  data-items  that  LARRD  0.43 do not collect, but xymond_rrd does. This is due to
              changes in the output from the Linux vmstat command, and changes in  the  way  e.g.   system  load
              metrics are reported.

       netstat
              All netstat files from LARRD 0.43 are incompatible with xymond_rrd.  The netstat data collected by
              LARRD is quite confusing: For some types of systems LARRD collects packet-counts,  for  others  it
              collects  byte-  counts.  xymond_rrd  uses  a different RRD file-format with separate counters for
              packets and bytes and tracks whatever data the system is reporting.

SEE ALSO

       xymond_channel(8), xymond(8), xymonserver.cfg(5), xymon(7)