Ubuntu Manpage: logging.yaml - the logging.yaml file defines all custom log file formats, filters, and

Provided by: trafficserver_8.0.5+ds-3_amd64

NAME

       logging.yaml  -  the  logging.yaml  file defines all custom log file formats, filters, and
       processing options

       IMPORTANT:
          This configuration file replaces the XML based logs_xml.config,  as  well  as  the  Lua
          based  logging.config  from  past  Traffic Server releases. If you are upgrading from a
          Traffic Server release which used either the XML or the Lua configuration file  format,
          and  you  have  created custom log formats, filters, and destinations, you will need to
          update those settings to this format.

LOG DEFINITIONS

Custom logs are configured by the combination of three key elements: a format, an optional
filter, and a log destination.

A format defines how log lines will appear (as well as whether the logs using the format
will be event logs or summary logs).

A filter defines what events do, and what events don't, make it into the logs employing
the filter.

A log defines where the record of events or summaries ends up.

Formats
Custom logging formats may be provided directly to a log definition, or they may be
defined as a reusable variable in your logging.yaml for ease of reference, particularly
when you may have more than one log using the same format. Which approach you use is
entirely up to you, though it's strongly recommended to create an explicit format object
if you intend to reuse the same format for multiple log files.

Custom formats are defined by choosing a name to identify the given logging format, and a
format string, which defines the output format string for every event. An optional
interval attribute can be specified to define the aggregation interval for summary logs.

# A one-line-per-event format that just prints event timestamps.
formats:
- name: myformat
format: '%<cqtq>'

# An aggregation/summary format that prints the last event timestamp from
# the interval along with the total count of events in the same interval.
# (Doing so every 30 seconds.)
formats:
- name: mysummaryformat
format: '%<LAST(cqtq)> %<COUNT(*)>'
interval: 30

You may define as many and as varied a collection of format objects as you desire.

Format Specification
The format specification provided as the required format attribute of the objects listed
in formats is a simple string, containing whatever mixture of logging field variables and
literal characters meet your needs. Logging fields are discussed in great detail in the
admin-logging-fields section.

Flexible enough to not only emulate the logging formats of most other proxy and HTTP
servers, but also to provide even finer detail than many of them, the logging fields are
very easy to use. Within the format string, logging fields are indicated by enclosing
their name within angle brackets (< and >), preceded by a percent symbol (%). For example,
returning to the altogether too simple format shown earlier, the following format string:

'%<cqtq>'

Defines a format in which nothing but the value of the logging field cqtq is interpolated
for each event's entry in the log. We could include some literal characters in the log
output by updating the format specification as so:

'Event received at %<cqtq>'

Because the string "Event received at " (including the trailing space) is just a bunch of
characters, not enclosed in %<...>, it is repeated verbatim in the logging output.

Multiple logging fields may of course be used:

'%<cqtq> %<chi> %<cqhm> %<cqtx>'

Each logging field is separately enclosed in its own percent-brace set.

There are a small number of logging fields which extend this simple format, primarily
those dealing with request and response headers. Instead of defining a separate logging
field name for every single possible HTTP header (an impossible task, given that arbitrary
vendor/application headers may be present in both requests and responses), there are
instead single logging fields for each of the major stages of an event lifecycle that
permit access to named headers, such as:

'%<{User-Agent}cqh>'

Which emits to the log the value of the client request's User-Agent HTTP header. Other
stages of the event lifecycle have similar logging fields: pqh (proxy requests), ssh
(origin server responses), and psh (proxy responses).

You will find a complete listing of the available fields in admin-logging-fields.

Aggregation Interval
Every format may be given an optional interval value, specified as the number of seconds
over which events destined for a log using the format are aggregated and summarized. Logs
which use formats containing an aggregation interval do not behave like regular logs, with
a single line for every event. Instead, they emit a single line only every
interval-seconds.

These types of logs are described in more detail in admin-logging-type-summary.

Formats have no interval by default, and will generate event-based logs unless given one.

Filters
Two different type of filters are available: accept and reject. They may be used,
optionally, to accept or reject logging for matching events.

Filter objects are created by assigning them a name to be used later to refer to the
filter, as well as an action (either accept or reject). Accept and reject filters require
a condition against which to match all events. The condition fields must be in the
following format:

For example, the following snippet defines a filter that matches all POST requests:

filters:
- name: postfilter
action: accept
condition: cqhm MATCH POST

Filter Fields
The log fields have already been discussed in the Formats section above. For a reference
to the available log field names, see admin-logging-fields. Unlike with the log format
specification, you do not wrap the log field names in any additional markup.

Filter Operators
The operators describe how to perform the matching in the filter rule, and may be any one
of the following:

MATCH True if the values of field and value are identical. Case-sensitive.

CASE_INSENSITIVE_MATCH
True if the values of field and value are identical. Case-insensitive.

CONTAIN
True if the value of field contains value (i.e. value is a substring of the
contents of field). Case-sensitive.

CASE_INSENSITIVE_CONTAIN
True if the value of field contains value (i.e. value is a substring of the
contents of field). Case-insensitive.

Filter Values
The final component of a filter string specifies the value against which the name field
will be compared.

For integer matches, all of the operators are effectively equivalent and require the field
to be equal to the given integer. If you wish to match multiple integers, provide a comma
separated list like this:

<field> <operator> 4,5,6,7

String matches work similarly to integer matches. Multiple matches are also supported via
a comma separated list. For example:

<field> <operator> e1host,host2,hostz

For IP addresses, ranges may be specified by separating the first address and the last of
the range with a single - dash, as 10.0.0.0-10.255.255.255 which gives the ranges for the
10/8 network. Other network notations are not supported at this time.

NOTE:
It may be tempting to attach multiple Filters to a log object reject multiple log
fields (in lieu of providing a single comma separated list to a single Filter). Avoid
this temptation and use a comma separated list of reject objects instead. Remember that
you may not have multiple accept filter objects. Attaching multiple filters does the
opposite of what you'd expect. If, for example, we had 2 accept log filters, each
disjoint from the other, nothing will ever get logged on the given log object.

Logs
Up to this point, we've only described what events should be logged and what they should
look like in the logging output. Now we define where those logs should be sent.

Three options currently exist for the type of logging output: ascii, binary, and
ascii_pipe. Which type of logging output you choose depends largely on how you intend to
process the logs with other tools, and a discussion of the merits of each is covered
elsewhere, in admin-logging-ascii-v-binary.

The following subsections cover the attributes you should specify when creating your
logging object. Only filename and format are required.

┌─────────────────────┬──────────────────┬────────────────────────────────────────┐
│Name │ Type │ Description │
├─────────────────────┼──────────────────┼────────────────────────────────────────┤
│filename │ string │ The name of the logfile │
│ │ │ relative to the default │
│ │ │ logging directory (set │
│ │ │ with │
│ │ │ proxy.config.log.logfile_dir). │
├─────────────────────┼──────────────────┼────────────────────────────────────────┤
│format │ string │ a string with a valid named │
│ │ │ format specification. │
└─────────────────────┴──────────────────┴────────────────────────────────────────┘

│header │ string │ If present, emitted as the │
│ │ │ first line of each new log │
│ │ │ file. │
├─────────────────────┼──────────────────┼────────────────────────────────────────┤
│rolling_enabled │ see below │ Determines the type of log │
│ │ │ rolling to use (or whether to │
│ │ │ disable rolling). Overrides │
│ │ │ proxy.config.log.rolling_enabled. │
├─────────────────────┼──────────────────┼────────────────────────────────────────┤
│rolling_interval_sec │ number │ Interval in seconds between log │
│ │ │ file rolling. Overrides │
│ │ │ proxy.config.log.rolling_interval_sec. │
├─────────────────────┼──────────────────┼────────────────────────────────────────┤
│rolling_offset_hr │ number │ Specifies an hour (from 0 to 23) at │
│ │ │ which log rolling is guaranteed to │
│ │ │ align. Only has an effect if │
│ │ │ RollingIntervalSec is set to greater │
│ │ │ than one hour. Overrides │
│ │ │ proxy.config.log.rolling_offset_hr. │
├─────────────────────┼──────────────────┼────────────────────────────────────────┤
│rolling_size_mb │ number │ Size, in megabytes, at which log files │
│ │ │ are rolled. │
├─────────────────────┼──────────────────┼────────────────────────────────────────┤
│filters │ array of filters │ The optional list of filter objects │
│ │ │ which restrict the individual events │
│ │ │ logged. The array may only contain one │
│ │ │ accept filter. │
├─────────────────────┼──────────────────┼────────────────────────────────────────┤
│collation_hosts │ array of strings │ If present, one or more strings │
│ │ │ specifying the log collation hosts to │
│ │ │ which logs should be delivered, each │
│ │ │ in the form of "<ip>:<port>". │
│ │ │ admin-logging-collation for more │
│ │ │ information. NOTE: This is a │
│ │ │ deprecated feature, which will be │
│ │ │ removed in ATS v9.0.0. See the logging │
│ │ │ documentation (above) for more │
│ │ │ details. │
└─────────────────────┴──────────────────┴────────────────────────────────────────┘

Enabling log rolling may be done globally in records.config, or on a per-log basis by
passing appropriate values for the rolling_enabled key. The latter method may also be used
to effect different rolling settings for individual logs. The numeric values that may be
passed are the same as used by proxy.config.log.rolling_enabled. For convenience and
readability, the following predefined variables may also be used in logging.yaml:

log.roll.none
Disable log rolling.

log.roll.time
Roll at a certain time frequency, specified by RollingIntervalSec and
RollingOffsetHr.

log.roll.size
Roll when the size exceeds RollingSizeMb.

log.roll.both
Roll when either the specified rolling time is reached or the specified file size
is reached.

log.roll.any
Roll the log file when the specified rolling time is reached if the size of the
file equals or exceeds the specified size.

EXAMPLES

       The  following  is  an  example  of  a format that collects information using three common
       fields:

          formats:
          - name: minimalfmt
            format: '%<chi> : %<cqu> : %<pssc>'

       The following is an example of a format that uses aggregate operators to produce a summary
       log:

          formats:
          - name: summaryfmt
            format: '%<LAST(cqts)> : %<COUNT(*)> : %<SUM(psql)>'
            interval: 10

       The  following  is  an  example  of a filter that will cause only REFRESH_HIT events to be
       logged:

          filters:
          - name: refreshhitfilter
            action: accept
            condition: pssc MATCH REFRESH_HIT

       The following is an example of a log specification that creates a local log file  for  the
       minimal format defined earlier. The log filename will be minimal.log because we select the
       ASCII logging format.

          logs:
          - mode: ascii
            filename: minimal
            format: minimalfmt

       The following is an example of a log specification that creates a local log file using the
       summary  format from earlier, and only includes events that matched the REFRESH_HIT filter
       we created.

          logs:
          - mode: ascii
            filename: refreshhit_summary
            format: summaryfmt
            filters:
            - refreshhitfilter

COPYRIGHT

       2020, dev@trafficserver.apache.org