Ubuntu Manpage: varnishncsa - Display Varnish logs in Apache / NCSA combined log format

Provided by: varnish_5.2.1-1ubuntu0.1_amd64

NAME

       varnishncsa - Display Varnish logs in Apache / NCSA combined log format

SYNOPSIS

       varnishncsa   [-a]   [-b]  [-c]  [-C]  [-d]  [-D]  [-F  <format>]  [-f  <formatfile>]  [-g
       <request|vxid>] [-h] [-L <limit>] [-n <dir>] [-P <file>] [-q <query>] [-r <filename>]  [-t
       <seconds|off>] [-V] [-w <filename>]

DESCRIPTION

The varnishncsa utility reads varnishd(1) shared memory logs and presents them in the
Apache / NCSA "combined" log format.

Each log line produced is based on a single Request type transaction gathered from the
shared memory log. The Request transaction is then scanned for the relevant parts in order
to output one log line. To filter the log lines produced, use the query language to select
the applicable transactions. Non-request transactions are ignored.

The following options are available:

-a When writing output to a file, append to it rather than overwrite it.

-b Log backend requests. If -c is not specified, then only backend requests will
trigger log lines.

-c Log client requests. This is the default. If -b is specified, then -c is needed to
also log client requests

-C Do all regular expression and string matching caseless.

-d Process log records at the head of the log and exit.

-D Daemonize.

-F <format>
Set the output log format string.

-f <formatfile>
Read output format from a file. Will read a single line from the specified file,
and use that line as the format.

-g <request|vxid>
The grouping of the log records. The default is to group by vxid.

-h Print program usage and exit

-L <limit>
Sets the upper limit of incomplete transactions kept before the oldest transaction
is force completed. A warning record is synthesized when this happens. This setting
keeps an upper bound on the memory usage of running queries. Defaults to 1000
transactions.

-n <dir>
Specify the varnishd working directory (also known as instance name) to get logs
from. If -n is not specified, the host name is used.

-P <file>
Write the process' PID to the specified file.

-q <query>
Specifies the VSL query to use.

-r <filename>
Read log in binary file format from this file. The file can be created with
varnishlog -w filename.

-t <seconds|off>
Timeout before returning error on initial VSM connection. If set the VSM connection
is retried every 0.5 seconds for this many seconds. If zero the connection is
attempted only once and will fail immediately if unsuccessful. If set to "off", the
connection will not fail, allowing the utility to start and wait indefinetely for
the Varnish instance to appear. Defaults to 5 seconds.

-V Print version information and exit.

-w <filename>
Redirect output to file. The file will be overwritten unless the -a option was
specified. If the application receives a SIGHUP in daemon mode the file will be
reopened allowing the old one to be rotated away. This option is required when
running in daemon mode.

MODES

       The default mode of varnishncsa is "client mode".  In this mode, the log will  be  similar
       to  what  a  web  server  would  produce  in  the  absence of varnish.  Client mode can be
       explicitly selected by using -c.

       If the -b switch is specified, varnishncsa will operate in "backend mode".  In this  mode,
       requests  generated  by  varnish  to  the  backends  will  be  logged.   Unless -c is also
       specified, client requests received by varnish will be ignored.

       When running varnishncsa in both backend and  client  mode,  it  is  strongly  advised  to
       include  the  format  specifier %{Varnish:side}x to distinguish between backend and client
       requests.

       Client requests that results in a pipe  (ie.  return(pipe)  in  vcl),  will  not  generate
       logging  in  backend mode. This is because varnish is not generating requests, but blindly
       passes on bytes in both directions.  However, a varnishncsa  instance  running  in  normal
       mode can see this case by using the formatter %{Varnish:handling}x, which will be 'pipe'.

       In  backend  mode,  some  of the fields in the format string get different meanings.  Most
       notably, the byte counting formatters (%b, %I, %O) considers varnish to be the client.

       It is possible to keep two varnishncsa instances running, one in backend mode, and one  in
       client mode, logging to different files.

FORMAT

Specify the log format to use. If no format is specified the default log format is used:

%h %l %u %t "%r" %s %b "%{Referer}i" "%{User-agent}i"

Escape sequences \n and \t are supported.

Supported formatters are:

%b In client mode, size of response in bytes, excluding HTTP headers. In backend
mode, the number of bytes received from the backend, excluding HTTP headers. In
CLF format, i.e. a '-' rather than a 0 when no bytes are sent.

%D In client mode, time taken to serve the request, in microseconds. In backend mode,
time from the request was sent to the entire body had been received.

%H The request protocol. Defaults to HTTP/1.0 if not known.

%h Remote host. Defaults to '-' if not known. In backend mode this is the IP of the
backend server.

%I In client mode, total bytes received from client. In backend mode, total bytes
sent to the backend.

%{X}i The contents of request header X. If the header appears multiple times in a single
transaction, the last occurrence is used.

%l Remote logname. Always '-'.

%m Request method. Defaults to '-' if not known.

%{X}o The contents of response header X. If the header appears multiple times in a single
transaction, the last occurrence is used.

%O In client mode, total bytes sent to client. In backend mode, total bytes received
from the backend.

%q The query string. Defaults to an empty string if not present.

%r The first line of the request. Synthesized from other fields, so it may not be the
request verbatim. See the NOTES section.

%s Status sent to the client. In backend mode, status received from the backend.

%t In client mode, time when the request was received, in HTTP date/time format. In
backend mode, time when the request was sent.

%{X}t In client mode, time when the request was received, in the format specified by X.
In backend mode, time when the request was sent. The time specification format is
the same as for strftime(3).

%T In client mode, time taken to serve the request, in seconds. In backend mode, time
from the request was sent to the entire body had been received.

%U The request URL without the query string. Defaults to '-' if not known.

%u Remote user from auth.

%{X}x Extended variables. Supported variables are:

Varnish:time_firstbyte
Time from when the request processing starts until the first byte is sent to
the client. For backend mode: Time from the request was sent to the backend
to the entire header had been received.

Varnish:hitmiss
One of the 'hit' or 'miss' strings, depending on whether the request was a
cache hit or miss. Pipe, pass and synth are considered misses.

Varnish:handling
One of the 'hit', 'miss', 'pass', 'pipe' or 'synth' strings indicating how
the request was handled.

Varnish:side
Backend or client side. One of two values, 'b' or 'c', depending on where
the request was made. In pure backend or client mode, this field will be
constant.

Varnish:vxid
The VXID of the varnish transaction.

VCL_Log:key
The value set by std.log("key:value") in VCL.

VSL:tag:record-prefix[field]
The value of the VSL entry for the given tag-record prefix-field
combination. Tag is mandatory, the other components are optional.

The record prefix will limit the matches to those records that have this
prefix as the first part of the record content followed by a colon.

The field will, if present, treat the log record as a white space separated
list of fields, and only the nth part of the record will be matched against.
Fields start counting at 1 and run up to 255.

Defaults to '-' when the tag is not seen, the record prefix does not match
or the field is out of bounds. If a tag appears multiple times in a single
transaction, the first occurrence is used.

SIGNALS

       SIGHUP Rotate the log file (see -w option).

       SIGUSR1
              Flush any outstanding transactions.

NOTES

       The %r formatter is equivalent to "%m http://%{Host}i%U%q %H". This differs from  apache's
       %r behavior, equivalent to "%m %U%q %H".  Furthermore, when using the %r formatter, if the
       Host header appears multiple times in a single transaction, the first occurrence is used.

EXAMPLE

       Log the second field of the  Begin  record,  corresponding  to  the  VXID  of  the  parent
       transaction:

          varnishncsa -F "%{VSL:Begin[2]}x"

       Log the entire Timestamp record associated with the processing length:

          varnishncsa -F "%{VSL:Timestamp:Process}x"

HISTORY

       The  varnishncsa  utility  was  developed by Poul-Henning Kamp in cooperation with Verdens
       Gang AS and Varnish Software AS. This manual page  was  initially  written  by  Dag-Erling
       Smørgrav  <des@des.no>,  and  later  updated  by  Martin  Blix  Grydeland  and Pål Hermunn
       Johansen.

COPYRIGHT

       This document is licensed under the same  licence  as  Varnish  itself.  See  LICENCE  for
       details.

       • Copyright (c) 2006 Verdens Gang AS

       • Copyright (c) 2006-2016 Varnish Software AS

                                                                                   VARNISHNCSA(1)