Provided by: calamaris_2.99.4.7-1.1_all bug

NAME

       calamaris  -  generate  text  and  graphical statistics out of log files from Proxy-Cache-
       Servers

SYNOPSIS

       cat log | calamaris [ --config-file .../calamaris.conf ] [ switches ]

DESCRIPTION

       Calamaris is used to produce statistical output  from  Squid,  NetCache,  Inktomi  Traffic
       Server,  Oops!  proxy server, Compaq Tasksmart, Cisco Content Engines or related Proxy log
       files.  The resulting output can be ascii or html with or  without  graphic  and  with  or
       without  frames.  It  is  possible to cache calculated data in a file to use them in later
       runs.

       This manual page describes the options of Calamaris and gives a few examples.

OPTIONS

   Configuration File
       --config-file file

              Not all reports and modification can be made through command-line-switches.  To use
              all  options  of  Calamaris you´ll have to use the configuration file.  You'll find
              the configuration-directives below, always inside of braces.  Examples are  in  the
              calamaris.conf which should come with this package.

   Reports
       --all-useful-reports|-a
              extracts   all   useful  reports  available,  --all-useful-reports  equals  --size-
              distribution-report 10  --domain-report  20  --performance-report  60  --requester-
              report  20  --status-report  --type-report  20  --response-time-report --errorcode-
              distribution-report

       --domain-report|-d n ($domain_report)
              switches the top level and the second level report on.  The data  is  derived  from
              the URL.  The output is limited by n. (-1 is unlimited)

       --domain-report-limit n ($domain_report_limit)
              limit the domain-report to lines which have n or more requests.

       --domain-report-n-level|-N n ($domain_report_n_level)
              All  URL-Host reports will be switched from 2nd-level to n-level-reports. (-1 shows
              a full urlhost-report)

              Note: This option is only useful with activated domain-report.

       --errorcode-distribution-report ($errorcode_distribution_report)
              shows the Response code distribution over all objects

       ($object_freshness_report)
              shows the freshness of objects in your cache.  Calamaris looks for  freshness  tags
              like  'TCP_HIT',  'TCP_REFRESH_MISS',  ...   and  make  statistics on it. With this
              information you can optimize the caching behaviour of your cache depending  on  the
              objects  content  type.   E.g. squid admins could use this information to configure
              the refresh_pattern. This option needs more  configuration  in  the  configuration-
              file.

       --peak-report|-p type ($peak_report)
              Measures  the  peaks  of the Proxy usage in requests per second, per minute and per
              hour.  It reports peaks for TCP, UDP and ALL requests. If set to 'old' these values
              were calculated with the old slow method, if set to 'new' the new faster (but still
              slow) method is used.

       --performance-report|-P n ($performance_report)
              Shows the throughput of TCP requests for every n minutes.

       --performance-report-adjust|-T n ($performance_report_adjust)
              Time: Adjust the Performancereport in minutes for non GMT-Timezoners.

       --requester-report|-r n ($requester_report)
              Switches the UDP and TCP requester reports on.  The output is limited by n.  (-1 is
              unlimited)

       --requester-report-no-dns-lookup|-n ($requester_report_no_dns_lookup)
              Switches the IP lookup for the requesters off.

       --requester-report-use-user-info|-u ($requester_report_use_user_info)
              Switches  the usage of eventually available ident information for requester reports
              on.

              Warning: This breaks the privacy of your users! (see PRIVACY-Section below)

       --requester-report-with-targets|-R n ($requester_report_with_targets)
              adds to each line of the requester report the requested URLs. The output is limited
              by n.  (-1 is unlimited, and can result in very very long reports.)

              Warning:  Using  this option breaks the privacy of your users! (see PRIVACY-Section
              below)

       --response-time-report ($response_time_report)
              sums up the time distribution over all objects

       (@response_time_report_interval)
              This array defines the time steps, which should be reported in  the  response-time-
              report.

       --size-distribution-report|-D n ($size_distribution_report)
              shows  size-based distribution of requested objects, smaller numbers result in more
              verbose reports. (choose 2, 10 or 100 for useful output.)

       --status-report|-s ($status_report)
              alters the default behaviour  of  Calamaris  and  makes  the  status  reports  more
              verbose.

       --type-report|-t n ($type_report)
              switches  the content type and the file extension report on.  The output is limited
              by n.  (-1 is unlimited)

       --type-report-ignore-case|-c ($type_report_ignore_case)
              Switch to case-insensitive. This is useful for the 'Requested extensions' report.

   Input
       --input-format|-f type ($input_format)
              sets the type of input logfiles. If set to

              'auto' Calamaris tries to guess the input file format.  This is the Default.

              Note: If the first line of your input file is corrupted, Calamaris will  stop  with
              an error.

              'squid'  Calamaris  expects  native  logfile derived from Squid V1.1.beta26-V2.x or
              OOPS.

              'squid-extended'   Calamaris   expects   native   logfile   derived   from    Squid
              V1.1.alpha1-V2.x  with  log_mime_hdrs  enabled  or  Squid with Smartfilter-Patch or
              squid-style logfiles out of Cisco Content Engines.  (This only enables  parsing  of
              these kind of logfile, the additional data will be ignored.)

              (Logging  of  MIME-Headers  breaks  the privacy of your users! (see PRIVACY-Section
              below)

              'squid-old'   Calamaris    expects    native    logfile    derived    from    Squid
              V1.1.alpha1-V1.1.beta25.

              'nc' Calamaris expects Logfiles from NetCache up to V4.x. (Please see the README on
              this.)

              'its' Calamaris expects Logfiles from Inktomi Traffic Server.

              'elff' Calamaris expects Logfiles in Extended  Logfile  Format  (i.e.  from  Compaq
              Tasksmart, Novell Internet Caching System or NetCache V5.x)

              'nse'  Calamaris  expects  Logfiles  in  Netscape Extended-1 or Netscape Extended-2
              Logfile Format (from Netscape/iPlanet/SunOne Proxy-Server )

       --ipfilter-exclude IP/range ($ipfilter_exclude)
              all  IPs  are  analyzed,  except  IP/range.    Format:   1.1.1.1/32:1.1.2.0/24   or
              1.1.1.1/255.255.255.255:1.1.2.0/255.255.255.0

              IP list separated by ':' This switch needs the perl Module NetAddr::IP.

              Warning: This breaks the privacy of your users! (see PRIVACY-Section below)

       --ipfilter-include IP/range ($ipfilter_include)
              no IPs are analyzed, except IP/range. Format: see --ipfilter-exclude

              Warning: This breaks the privacy of your users! (see PRIVACY-Section below)

       --no-input|-z ($no_input)
              Switches  reading  from  standard  input off.  You can use this to merge many cache
              files to one (see --cache-input-file and  --cache-output-file)  or  to  generate  a
              report out of cache files.

       --time-interval|-I t-t ($time_interval)
              defines   which   time-interval   should  be  parsed.   t  has  to  be  the  format
              yyyymmddhhmmss (localtime)

              Note: omitting the beginning or ending date is allowed.

   Output
       Standard output format is plain ascii with 80 chars width.

       ($column1_color)
              ($column2_color) defines the colors for the columns in graphics. (only useful  with
              --output-format graph)

       ($formats[n])
              Through  the  config-file  you are able to modify the width of the report and alter
              the culomns that are displayed in the reports. n is the number of  the  report,  as
              displayed by --help in the --show-reports-option.

       --hostname|-H name ($hostname)
              The  name  for  the  title  or subject of the output.  If set to 'lookup' Calamaris
              looks up the host name of the system its been run on.

       --image-type ($image_type)
              Sets the image type to gif, png, jpeg, gd or gd2. Only useful when  --output-format
              graph  is  set.  The  available  images  types  are  depending  on  your  GD::Graph
              installation. Default is 'png'.

       --logo|-l string ($logo)
              add a custom string to a HTML-Report. It'll be added to a table on the top  of  the
              output.          -l         '<A        HREF="https://cord.de/"><IMG        BORDER=0
              SRC="https://cord.de/sites/default/files/cordlog2n.gif" ALT="Cord"></A>'  will  add
              my logo with a link to the Report.

              Note: --logo works only in combination with --output-format html or html-frame

       --meta|-M string ($meta)
              Meta:  adds  a  custom  string  or the content of a file into the <HEAD> of a HTML-
              Report. Useful if you want to add Stylesheets or something to the Report.

              Note: --meta works only in combination with --output-format html or html-frame

       --output-format|-F type[,type[,type[,...]]] ($output_format)
              Format: sets the format of the output-report. If set to

              'mail' adds a subject header to the beginning of the report.

              'html' all output is given in html with tables.  Can be  combined  with  'mail'  to
              send html mails.

              'html-frame' all output is given in html frames with tables.

              'html-embed'  all  output is given in html with tables without HTML-Headers. Useful
              for Server-Side-Includes.

              'graph' enables graphics for html, html-embed or html-frame.

              'unformatted' gives out the raw numbers separated by spaces.  Useful  for  re-using
              the  output  in  other  scripts. If you use this along with -U, the byte values are
              calculated in the given Unit, and  displayed  without  indication  along  with  the
              numbers. the indication moves up to the header of the report.

       --output-path ($output_path)
              output  calamaris  statistics  to  /path.  In  case  of  graph output, the graphics
              destination is /path and the filename is index.html, else it is calamaris.txt.   If
              --output-path is not given, all graphics are written to the working directory.

       --output-file ($output_file)
              alter the filename of --output-path.

       --output-file-prefix ($output_file_prefix)
              adds a prefix to --output-file %t is replaced by the timerange of the report, %h by
              the hostname (see --hostname )

       --show-reports|-S n[,n[,n[,...]]] ($show_reports)
              Show: Shows only the defined reports in the specified order. Default is to  display
              the  reports  as  they are defined through the report-switches above. The following
              numbers are defined:

              0  Summary
              1  Incoming request peak per protocol
              2  Incoming transfer volume peak per protocol
              3  Incoming requests by method
              4  Incoming UDP-requests by status
              5  Incoming TCP-requests by status
              6  Outgoing requests by status
              7  Outgoing requests by destination
              8  Request-destinations by 2ndlevel-domain
              9  Request-destinations by toplevel-domain
              10 TCP-Request-protocol
              11 Requested content-type
              12 Requested extensions
              13 Incoming UDP-requests by host
              14 Incoming TCP-requests by host
              15 Size Distribution Diagram
              16 Performance in n minute steps
              17 UDP-Request duration distribution in msec
              18 TCP-Request duration distribution in msec
              19 UDP Response code distribution
              20 TCP Response code distribution

       Note: Using this doesn't make Calamaris any faster, the internal calculations will be done
       as the report-switches were set (see above).

       --sort-order|-O ($sort_order)
              Changes the sort order in the reports to request size, default is sorting by number
              of requests.

       ($text_color)
              defines the colors for  text/axis/legend/labels  in  graphics.  (only  useful  with
              --output-format graph )

       --unit|-U string ($unit)
              You can define this to K(ilo), M(ega), G(iga) or T(era) for the Byte-units.

       ($width)
              defines the width of the graphics. height is calculated from this with a 3:2-ratio.
              (only useful with --output-format graph )

       ($x_scale)
              defines how many datasets should be drawn on the graph. 30 is a good value, but you
              can play with this. if $x_scale gets to big, you're on your own ;-)

       --generate-index ($generate_index)
              generates an index for all reports that match --output-file-prefix.

   Caching
       --cache-input-file|-i file ($cache_input_file)
              You  can reuse a cache file generated with --cache-output-file file to add old data
              to a new report.  Several files can be separated with a ':'.

              Note: if you use more than one  cache  file,  make  sure  they  are  chronologicaly
              ordered (oldest first).

              Note: if you reuse cache-files, which were not created with -d -1 -r -1 -t -1 -R -1
              the number of 'others' would be wrong  everywhere.  In  this  case  the  number  of
              'others' are omitted.

       --cache-output-file|-o file ($cache_output_file)
              Calamaris stores a summary of the computed information in file and you can reuse it
              at a later time with --cache-input-file

              Note: The output file can be the same as the input file: it is  simply  overwritten
              after  reading  the  data.   It  is  not  recommended to change the options between
              different  runs  if  you  include  older  data  as  this  can  result  in   strange
              measurements.

   Misc
       --benchmark|-b n ($benchmark)
              benchmark: A switch for the impatient as it prints a '#' for every n parsed lines.

       --copyright|-C
              Prints the copyright information of Calamaris

       --help|-h
              Prints a brief description of the command line options.

       --version|-V
              Prints out the Version-Number.

   Debug
       --dump-loop|-L
              prints the internal loop to STDERR. (for Debugging)

       ($test)
              activates some small tests for the programmer.

       --verbose|-v ($verbose)
              print more information about what is Calamaris is doing and believing.

EXAMPLES

       This example mails the daily statistics to root:
              cat  /var/log/squid/access.log | nice -39 calamaris --all-useful-reports --hostname
              "daily worf" --output-format mail | mail root

       This one only caches a summary for later use:
              cat /var/log/squid/access.log | calamaris --all-useful-reports  --cache-output-file
              daily.`date +"%w"` > /dev/null

       You can then use the caches to have weekly statistics:

       if [ $DAYOFWEEK = "0" ]; then
              calamaris                  --all-useful-reports                  --cache-input-file
              daily.1:daily.2:daily.3:daily.4:daily.5:daily.6:daily.0 --no-input  --output-format
              mail --hostname "weekly worf" | mail root ;

       fi

BUGS

       If you have a problem with Calamaris , please make sure that you use the recent version of
       Calamaris (see VERSION below). Also check that your  proxy  works  correctly  and  doesn't
       produce invalid Logfiles. (see the README for buglist and pointers.)

       If  you're  sure that you've encountered a bug in Calamaris please report it to Calamaris-
       bug@cord.de. This also applies if Calamaris itself says 'please report this'.

PRIVACY

       Calamaris can be (mis-)used to track what users are requesting.

       So please read the following and think about it, before using  Calamaris  to  be  the  Big
       Brother.

       -      If  you  don't trust your users than there is something more wrong than the loss of
              productivity.

       -      Squid has some nice acl-mechanisms. If you think that your users don't use the  net
              properly, don't let them use it. (You can also open the net at specific times or to
              specific sites, if you want.)

       -      If you still want to use Calamaris  that  way,  let  your  vict^Wusers  know,  that
              they'll be monitored. (in Germany you have to let them know!)

SEE ALSO

       squid(8)

AUTHOR

       Cord  Beermann  <Cord@Wunder-Nett.org>, Michael Pophal <michael.pophal@nefkom.net>.  There
       are also a lot of people who contributed code, gave ideas or requested features. Look them
       up in the executable.

       This man page was written by Philipp Frauenfelder <pfrauenf@debian.org>, maintainer of the
       Debian package.  Maintenance is now taken over by Cord Beermann.

VERSION

       Version of this manpage: $Id: calamaris.1,v 3.1 2006-03-19 17:52:48 cord Exp $

       It describes the usage of Calamaris V3.0 and later.

       Information about new releases, mailing lists, and other related issues can be found  from
       the Calamaris home page at URL https://Calamaris.Cord.de/

WARRANTY

       Calamaris comes with "absolutely no warranty".

COPYRIGHT

       Copyright (C) 1997-2006, 2013, 2015, 2017, 2021 Cord Beermann

       This program is free software; you can redistribute it and/or modify it under the terms of
       the GNU General Public License as  published  by  the  Free  Software  Foundation;  either
       version 2 of the License, or (at your option) any later version.

       (If  you  modify and want to publish it under the name Calamaris , please ask me.  I don't
       want to confuse the 'audience' with many  different  versions  of  the  same  name  and/or
       Version number.  (This is not part of the license, it is only a favour i asked of you.))

       This  program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
       without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR  PURPOSE.
       See the GNU General Public License for more details.

       You should have received a copy of the GNU General Public License along with this program;
       if not, write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330,  Boston,
       MA 02111-1307, USA.