xenial (1) calamaris.1.gz

Provided by: calamaris_2.99.4.5-1_all bug

NAME

       calamaris - generate text and graphical statistics out of log files from Proxy-Cache-Servers

SYNOPSIS

       cat log | calamaris [ --config-file .../calamaris.conf ] [ switches ]

DESCRIPTION

       Calamaris is used to produce statistical output from Squid, NetCache, Inktomi Traffic Server, Oops! proxy
       server, Compaq Tasksmart, Cisco Content Engines or related Proxy log files.  The resulting output can  be
       ascii or html with or without graphic and with or without frames. It is possible to cache calculated data
       in a file to use them in later runs.

       This manual page describes the options of Calamaris and gives a few examples.

OPTIONS

   Configuration File
       --config-file file

              Not all reports and modification can be made through command-line-switches.  To use all options of
              Calamaris  you´ll  have  to  use the configuration file.  You'll find the configuration-directives
              below, always inside of braces.  Examples are in the calamaris.conf which should  come  with  this
              package.

   Reports
       --all-useful-reports|-a
              extracts  all  useful reports available, --all-useful-reports equals --size-distribution-report 10
              --domain-report 20 --performance-report 60 --requester-report 20 --status-report --type-report  20
              --response-time-report --errorcode-distribution-report

       --domain-report|-d n ($domain_report)
              switches  the  top  level  and the second level report on.  The data is derived from the URL.  The
              output is limited by n. (-1 is unlimited)

       --domain-report-limit n ($domain_report_limit)
              limit the domain-report to lines which have n or more requests.

       --domain-report-n-level|-N n ($domain_report_n_level)
              All URL-Host reports will be switched from 2nd-level to n-level-reports. (-1 shows a full urlhost-
              report)

              Note: This option is only useful with activated domain-report.

       --errorcode-distribution-report ($errorcode_distribution_report)
              shows the Response code distribution over all objects

       ($object_freshness_report)
              shows  the freshness of objects in your cache.  Calamaris looks for freshness tags like 'TCP_HIT',
              'TCP_REFRESH_MISS', ...  and make statistics on it. With this information  you  can  optimize  the
              caching  behaviour  of  your cache depending on the objects content type.  E.g. squid admins could
              use this information to configure the refresh_pattern. This option needs more configuration in the
              configuration-file.

       --peak-report|-p type ($peak_report)
              Measures the peaks of the Proxy usage in requests per second, per minute and per hour.  It reports
              peaks for TCP, UDP and ALL requests. If set to 'old' these values were  calculated  with  the  old
              slow method, if set to 'new' the new faster (but still slow) method is used.

       --performance-report|-P n ($performance_report)
              Shows the throughput of TCP requests for every n minutes.

       --performance-report-adjust|-T n ($performance_report_adjust)
              Time: Adjust the Performancereport in minutes for non GMT-Timezoners.

       --requester-report|-r n ($requester_report)
              Switches the UDP and TCP requester reports on.  The output is limited by n.  (-1 is unlimited)

       --requester-report-no-dns-lookup|-n ($requester_report_no_dns_lookup)
              Switches the IP lookup for the requesters off.

       --requester-report-use-user-info|-u ($requester_report_use_user_info)
              Switches the usage of eventually available ident information for requester reports on.

              Warning: This breaks the privacy of your users! (see PRIVACY-Section below)

       --requester-report-with-targets|-R n ($requester_report_with_targets)
              adds  to each line of the requester report the requested URLs. The output is limited by n.  (-1 is
              unlimited, and can result in very very long reports.)

              Warning: Using this option breaks the privacy of your users! (see PRIVACY-Section below)

       --response-time-report ($response_time_report)
              sums up the time distribution over all objects

       (@response_time_report_interval)
              This array defines the time steps, which should be reported in the response-time-report.

       --size-distribution-report|-D n ($size_distribution_report)
              shows size-based distribution of  requested  objects,  smaller  numbers  result  in  more  verbose
              reports. (choose 2, 10 or 100 for useful output.)

       --status-report|-s ($status_report)
              alters the default behaviour of Calamaris and makes the status reports more verbose.

       --type-report|-t n ($type_report)
              switches  the  content type and the file extension report on.  The output is limited by n.  (-1 is
              unlimited)

       --type-report-ignore-case|-c ($type_report_ignore_case)
              Switch to case-insensitive. This is useful for the 'Requested extensions' report.

   Input
       --input-format|-f type ($input_format)
              sets the type of input logfiles. If set to

              'auto' Calamaris tries to guess the input file format.  This is the Default.

              Note: If the first line of your input file is corrupted, Calamaris will stop with an error.

              'squid' Calamaris expects native logfile derived from Squid V1.1.beta26-V2.x or OOPS.

              'squid-extended' Calamaris  expects  native  logfile  derived  from  Squid  V1.1.alpha1-V2.x  with
              log_mime_hdrs enabled or Squid with Smartfilter-Patch or squid-style logfiles out of Cisco Content
              Engines.  (This only enables parsing of these  kind  of  logfile,  the  additional  data  will  be
              ignored.)

              (Logging of MIME-Headers breaks the privacy of your users! (see PRIVACY-Section below)

              'squid-old' Calamaris expects native logfile derived from Squid V1.1.alpha1-V1.1.beta25.

              'nc' Calamaris expects Logfiles from NetCache up to V4.x. (Please see the README on this.)

              'its' Calamaris expects Logfiles from Inktomi Traffic Server.

              'elff'  Calamaris  expects Logfiles in Extended Logfile Format (i.e. from Compaq Tasksmart, Novell
              Internet Caching System or NetCache V5.x)

              'nse' Calamaris expects Logfiles in Netscape Extended-1  or  Netscape  Extended-2  Logfile  Format
              (from Netscape/iPlanet/SunOne Proxy-Server )

       --ipfilter-exclude IP/range ($ipfilter_exclude)
              all     IPs     are    analyzed,    except    IP/range.     Format:    1.1.1.1/32:1.1.2.0/24    or
              1.1.1.1/255.255.255.255:1.1.2.0/255.255.255.0

              IP list separated by ':' This switch needs the perl Module NetAddr::IP.

              Warning: This breaks the privacy of your users! (see PRIVACY-Section below)

       --ipfilter-include IP/range ($ipfilter_include)
              no IPs are analyzed, except IP/range. Format: see --ipfilter-exclude

              Warning: This breaks the privacy of your users! (see PRIVACY-Section below)

       --no-input|-z ($no_input)
              Switches reading from standard input off.  You can use this to merge many cache files to one  (see
              --cache-input-file and --cache-output-file) or to generate a report out of cache files.

       --time-interval|-I t-t ($time_interval)
              defines which time-interval should be parsed.  t has to be the format yyyymmddhhmmss (localtime)

              Note: omitting the beginning or ending date is allowed.

   Output
       Standard output format is plain ascii with 80 chars width.

       ($column1_color)
              ($column2_color) defines the colors for the columns in graphics. (only useful with --output-format
              graph)

       ($formats[n])
              Through the config-file you are able to modify the width of the report and alter the culomns  that
              are displayed in the reports. n is the number of the report, as displayed by --help in the --show-
              reports-option.

       --hostname|-H name ($hostname)
              The name for the title or subject of the output.  If set to 'lookup' Calamaris looks up  the  host
              name of the system its been run on.

       --image-type ($image_type)
              Sets  the  image type to gif, png, jpeg, gd or gd2. Only useful when --output-format graph is set.
              The available images types are dependend on your GD::Graph installation. Default is 'png'.

       --logo|-l string ($logo)
              add a custom string to a HTML-Report. It'll be added to a table on the top of the output.  -l  '<A
              HREF="http://cord.de/"><IMG         BORDER=0        SRC="http://cord.de/Images/cord/cordlog2n.gif"
              ALT="Cord"></A>' will add my logo with a link to the Report.

              Note: --logo works only in combination with --output-format html or html-frame

       --meta|-M string ($meta)
              Meta: adds a custom string or the content of a file into the <HEAD> of a  HTML-Report.  Useful  if
              you want to add Stylesheets or something to the Report.

              Note: --meta works only in combination with --output-format html or html-frame

       --output-format|-F type[,type[,type[,...]]] ($output_format)
              Format: sets the format of the output-report. If set to

              'mail' adds a subject header to the beginning of the report.

              'html' all output is given in html with tables.  Can be combined with 'mail' to send html mails.

              'html-frame' all output is given in html frames with tables.

              'html-embed' all output is given in html with tables without HTML-Headers. Useful for Server-Side-
              Includes.

              'graph' enables graphics for html, html-embed or html-frame.

              'unformatted' gives out the raw numbers separated by spaces. Useful for  re-using  the  output  in
              other  scripts.  If  you use this along with -U, the byte values are calculated in the given Unit,
              and displayed without indication along with the numbers. the indication moves up to the header  of
              the report.

       --output-path ($output_path)
              output  calamaris  statistics to /path. In case of graph output, the graphics destination is /path
              and the filename is index.html, else it is calamaris.txt.  If  --output-path  is  not  given,  all
              graphics are written to the working directory.

       --output-file ($output_file)
              alter the filename of --output-path.

       --output-file-prefix ($output_file_prefix)
              adds  a  prefix to --output-file %t is replaced by the timerange of the report, %h by the hostname
              (see --hostname )

       --show-reports|-S n[,n[,n[,...]]] ($show_reports)
              Show: Shows only the defined reports in the specified order. Default is to display the reports  as
              they are defined through the report-switches above. The following numbers are defined:

              0  Summary
              1  Incoming request peak per protocol
              2  Incoming transfer volume peak per protocol
              3  Incoming requests by method
              4  Incoming UDP-requests by status
              5  Incoming TCP-requests by status
              6  Outgoing requests by status
              7  Outgoing requests by destination
              8  Request-destinations by 2ndlevel-domain
              9  Request-destinations by toplevel-domain
              10 TCP-Request-protocol
              11 Requested content-type
              12 Requested extensions
              13 Incoming UDP-requests by host
              14 Incoming TCP-requests by host
              15 Size Distribution Diagram
              16 Performance in n minute steps
              17 UDP-Request duration distribution in msec
              18 TCP-Request duration distribution in msec
              19 UDP Response code distribution
              20 TCP Response code distribution

       Note: Using this doesn't make Calamaris any faster, the internal calculations will be done as the report-
       switches were set (see above).

       --sort-order|-O ($sort_order)
              Changes the sort order in the reports to request size, default is sorting by number of requests.

       ($text_color)
              defines the colors for text/axis/legend/labels in  graphics.  (only  useful  with  --output-format
              graph )

       --unit|-U string ($unit)
              You can define this to K(ilo), M(ega), G(iga) or T(era) for the Byte-units.

       ($width)
              defines  the  width of the graphics. height is calculated from this with a 3:2-ratio. (only useful
              with --output-format graph )

       ($x_scale)
              defines how many datasets should be drawn on the graph. 30 is a good value, but you can play  with
              this. if $x_scale gets to big, you're on your own ;-)

       --generate-index ($generate_index)
              generates an index for all reports that match --output-file-prefix.

   Caching
       --cache-input-file|-i file ($cache_input_file)
              You  can  reuse  a  cache  file  generated  with --cache-output-file file to add old data to a new
              report.  Several files can be separated with a ':'.

              Note: if you use more than one cache file, make  sure  they  are  chronologicaly  ordered  (oldest
              first).

              Note:  if you reuse cache-files, which were not created with -d -1 -r -1 -t -1 -R -1 the number of
              'others' would be wrong everywhere. In this case the number of 'others' are omitted.

       --cache-output-file|-o file ($cache_output_file)
              Calamaris stores a summary of the computed information in file and you can reuse  it  at  a  later
              time with --cache-input-file

              Note:  The  output  file can be the same as the input file: it is simply overwritten after reading
              the data.  It is not recommended to change the options between different runs if you include older
              data as this can result in strange measurements.

   Misc
       --benchmark|-b n ($benchmark)
              benchmark: A switch for the impatient as it prints a '#' for every n parsed lines.

       --copyright|-C
              Prints the copyright information of Calamaris

       --help|-h
              Prints a brief description of the command line options.

       --version|-V
              Prints out the Version-Number.

   Debug
       --dump-loop|-L
              prints the internal loop to STDERR. (for Debugging)

       ($test)
              activates some small tests for the programmer.

       --verbose|-v ($verbose)
              print more information about what is Calamaris is doing and believing.

EXAMPLES

       This example mails the daily statistics to root:
              cat  /var/log/squid/access.log  |  nice -39 calamaris --all-useful-reports --hostname "daily worf"
              --output-format mail | mail root

       This one only caches a summary for later use:
              cat /var/log/squid/access.log |  calamaris  --all-useful-reports  --cache-output-file  daily.`date
              +"%w"` > /dev/null

       You can then use the caches to have weekly statistics:

       if [ $DAYOFWEEK = "0" ]; then
              calamaris                          --all-useful-reports                         --cache-input-file
              daily.1:daily.2:daily.3:daily.4:daily.5:daily.6:daily.0 --no-input --output-format mail --hostname
              "weekly worf" | mail root ;

       fi

BUGS

       If you have a problem with Calamaris , please make sure that you use the recent version of Calamaris (see
       VERSION below). Also check that your proxy works correctly and doesn't produce invalid Logfiles. (see the
       README for buglist and pointers.)

       If you're sure that you've encountered a bug in Calamaris please report it to Calamaris-bug@cord.de. This
       also applies if Calamaris itself says 'please report this'.

PRIVACY

       Calamaris can be (mis-)used to track what users are requesting.

       So please read the following and think about it, before using Calamaris to be the Big Brother.

       -      If you don't trust your users than there is something more wrong than the loss of productivity.

       -      Squid has some nice acl-mechanisms. If you think that your users don't use the net properly, don't
              let them use it. (You can also open the net at specific times or to specific sites, if you want.)

       -      If you still want to use Calamaris that way, let your vict^Wusers know, that they'll be monitored.
              (in Germany you have to let them know!)

SEE ALSO

       squid(8)

AUTHOR

       Cord Beermann <Cord@Wunder-Nett.org>, Michael Pophal <michael.pophal@nefkom.net>.  There are also  a  lot
       of people who contributed code, gave ideas or requested features. Look them up in the executable.

       This  man  page  was  written  by  Philipp  Frauenfelder  <pfrauenf@debian.org>, maintainer of the Debian
       package.  Maintenance is now taken over by Cord Beermann.

VERSION

       Version of this manpage: $Id: calamaris.1,v 3.1 2006-03-19 17:52:48 cord Exp $

       It describes the usage of Calamaris V3.0 and later.

       Information about new releases, mailing lists, and other related issues can be found from  the  Calamaris
       home page at URL http://Calamaris.Cord.de/

WARRANTY

       Calamaris comes with "absolutely no warranty".

       Copyright (C) 1997-2006, 2013, 2015 Cord Beermann

       This  program  is  free  software;  you  can  redistribute it and/or modify it under the terms of the GNU
       General Public License as published by the Free Software Foundation; either version 2 of the License,  or
       (at your option) any later version.

       (If  you modify and want to publish it under the name Calamaris , please ask me.  I don't want to confuse
       the 'audience' with many different versions of the same name and/or Version number.  (This is not part of
       the license, it is only a favour i asked of you.))

       This  program  is  distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even
       the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General  Public
       License for more details.

       You  should have received a copy of the GNU General Public License along with this program; if not, write
       to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.