xenial (5) logs_xml.config.5.gz
NAME
logs_xml.config - Traffic Server log format configuration file
The logs_xml.config file defines the custom log file formats, filters, and processing options. The format
of this file is modeled after XML, the Extensible Markup Language.
FORMAT
The logs_xml.config file contains the specifications below:
• LogFormat specifies the fields to be gathered from each protocol event access.
• LogFilter specifies the filters that are used to include or exclude certain entries being logged based
on the value of a field within that entry.
• LogObject specifies an object that contains a particular format, a local filename, filters, and
collation servers.
The logs_xml.config file ignores extra white space, blank lines, and all comments.
LOGFORMAT
The following list shows LogFormat specifications.
<Name = valid_format_name />
Required Valid format names include any name except squid, common, extended, or extended2, which
are pre-defined formats. There is no default for this tag.
<Format = valid_format_specification />
Required A valid format specification is a printf-style string describing each log entry when
formatted for ASCII output.
The printf-style could accept Oct/Hex escape representation:
• \abc is Oct escape sequence, a,b,c should be one of [0-9], and (a*8^2 + b*8 + c) should be
greater than 0 and less than 255.
• \xab is Hex escape sequence, a,b should be one of [0-9, a-f, A-F], and (a*16 + b) should be
greater than 0 and less than 255.
Use %< field > as a placeholder for valid field names. For more information, refer to
custom-logging-fields.
The specified field can be one of the following types:
Simple. For example: %<cqu> A field within a container, such as an HTTP header or a statistic.
Fields of this type have the syntax:
%<{ field } container>
Aggregates, such as COUNT, SUM, AVG, FIRST, LAST. Fields of this type have the syntax: %<operator
( field )>
NOTE:
You cannot create a format specification that contains both aggregate operators and regular fields.
<Interval = aggregate_interval_secs />
Optional Use this tag when the format contains aggregate operators. The value
"aggregate_interval_secs" represents the number of seconds between individual aggregate values
being produced.
The valid set of aggregate operators are:
• COUNT
• SUM
• AVG
• FIRST
• LAST
LOGFILTER
The following list shows the LogFilter specifications.
<Name = valid_filter_name />
Required All filters must be uniquely named.
<Condition = valid_log_field valid_operator valid_comparison_value />
Required This field contains the following elements:
valid_log_field - the field that will be compared against the given value. For more
information, refer to logging-format-cross-reference.
valid_operator_field - any one of the following: MATCH, CASE_INSENSITIVE_MATCH, CONTAIN,
CASE_INSENSITIVE_CONTAIN.
• MATCH is true if the field and value are identical (case-sensitive).
• CASE_INSENSITIVE_MATCH is similar to MATCH, except that it is case-insensitive.
• CONTAIN is true if the field contains the value (the value is a substring of the field).
• CASE_INSENSITIVE_CONTAIN is a case-insensitive version of CONTAIN.
valid_comparison_value - any string or integer matching the field type. For integer values, all
of the operators are equivalent and mean that the field must be equal to the specified value.
For IP address fields, this can be a list of IP addresses and include ranges. A range is an IP
address, followed by a dash '-', and then another IP address of the same family. For instance, the
10/8 network can be represented by 10.0.0.0-10.255.255.255. Currently network specifiers are not
supported.
NOTE:
There are no negative comparison operators. If you want to specify a negative condition, then use the
Action field to REJECT the record.
<Action = valid_action_field />
Required: ACCEPT or REJECT or WIPE_FIELD_VALUE. ACCEPT or REJECT instructs Traffic Server to
either accept or reject records that satisfy the filter condition. WIPE_FIELD_VALUE wipes out the
values of the query params in the url fields specified in the Condition.
NOTES: 1. WIPE_FIELD_VALUE action is only applied to the parameters in the query part.
2. Multiple parameters can be listed in a single WIPE_FIELD_VALUE filter
3. If the same parameter appears more than once in the query part , only the value of the first
occurance is wiped
LOGOBJECT
The following list shows the LogObject specifications.
<Format = valid_format_name />
Required Valid format names include the predefined logging formats: squid, common, extended, and
extended2, as well as any previously-defined custom log formats. There is no default for this tag.
<Filename = file_name />
Required The filename to which the given log file is written on the local file system or on a
remote collation server. No local log file will be created if you fail to specify this tag. All
filenames are relative to the default logging directory.
If the name does not contain an extension (for example, squid), then the extension .log is
automatically appended to it for ASCII logs and .blog for binary logs (refer to Mode =
"valid_logging_mode").
If you do not want an extension to be added, then end the filename with a single (.) dot (for
example: squid. ).
<Mode = valid_logging_mode />
Optional Valid logging modes include ascii , binary , and ascii_pipe . The default is ascii .
• Use ascii to create event log files in human-readable form (plain ASCII).
• Use binary to create event log files in binary format. Binary log files generate lower system
overhead and occupy less space on the disk (depending on the information being logged). You must
use the traffic_logcat utility to translate binary log files to ASCII format before you can read
them.
• Use ascii_pipe to write log entries to a UNIX named pipe (a buffer in memory). Other processes
can then read the data using standard I/O functions. The advantage of using this option is that
Traffic Server does not have to write to disk, which frees disk space and bandwidth for other
tasks. In addition, writing to a pipe does not stop when logging space is exhausted because the
pipe does not use disk space.
If you are using a collation server, then the log is written to a pipe on the collation server. A
local pipe is created even before a transaction is processed, so you can see the pipe right after
Traffic Server starts. Pipes on a collation server, however, are created when Traffic Server
starts.
<Filters = list_of_valid_filter_names />
Optional A comma-separated list of names of any previously-defined log filters. If more than one
filter is specified, then all filters must accept a record for the record to be logged.
<Protocols = list_of_valid_protocols />
Optional A comma-separated list of the protocols this object should log. Valid protocol names for
this release are HTTP (FTP is deprecated).
<ServerHosts = list_of_valid_servers />
Optional A comma-separated list of valid hostnames.This tag indicates that only entries from the
named servers will be included in the file.
<CollationHosts = list_of_valid_hostnames:port|failover hosts />
Optional A comma-separated list of collation servers (with pipe delimited failover servers) to
which all log entries (for this object) are forwarded. Collation servers can be specified by name
or IP address. Specify the collation port with a colon after the name. For example, in
host1:5000|failhostA:5000|failhostB:6000, host2:6000 logs would be sent to host1 and host2, with
failhostA and failhostB acting as failover hosts for host1. When host1 disconnects, logs would be
sent to failhostA. If failhostA disconnects, log entries would be sent to failhostB until host1 or
failhostA comes back. Logs would also be sent to host2.
<Header = header />
Optional The header text you want the log files to contain. The header text appears at the
beginning of the log file, just before the first record.
<RollingEnabled = truth value />
Optional Enables or disables log file rolling for the LogObject. This setting overrides the value
for the proxy.config.log.rolling_enabled variable in the records.config file. Set truth value to
one of the following values:
• 0 to disable rolling for this particular LogObject.
• 1 to roll log files at specific intervals during the day (you specify time intervals with the
RollingIntervalSec and RollingOffsetHr fields).
• 2 to roll log files when they reach a certain size (you specify the size with the RollingSizeMb
field).
• 3 to roll log files at specific intervals during the day or when they reach a certain size
(whichever occurs first).
• 4 to roll log files at specific intervals during the day when log files reach a specific size
(at a specified time if the file is of the specified size).
<RollingIntervalSec = seconds />
Optional The seconds between log file rolling for the LogObject; enables you to specify different
rolling intervals for different LogObjects.
This setting overrides the value for proxy.config.log.rolling_interval_sec in the records.config
file.
<RollingOffsetHr = hour />
Optional Specifies an hour (from 0 to 23) at which rolling is guaranteed to align. Rolling might
start before then, but a rolled file will be produced only at that time. The impact of this
setting is only noticeable if the rolling interval is larger than one hour. This setting overrides
the configuration setting for proxy.config.log.rolling_offset_hr in the records.config file.
<RollingSizeMb = size_in_MB />
Optional The size at which log files are rolled.
EXAMPLES
The following is an example of a LogFormat specification that collects information using three common
fields:
<LogFormat>
<Name="minimal"/>
<Format = "%<chi> : %<cqu> : %<pssc>"/>
</LogFormat>
The following is an example of a LogFormat specification that uses aggregate operators:
<LogFormat>
<Name = "summary"/>
<Format = "%<LAST(cqts)> : %<COUNT(*)> : %<SUM(psql)>"/>
<Interval = "10"/>
</LogFormat>
The following is an example of a LogFilter that will cause only REFRESH_HIT entries to be logged:
<LogFilter>
<Name = "only_refresh_hits"/>
<Action = "ACCEPT"/>
<Condition = "%<pssc> MATCH REFRESH_HIT"/>
</LogFilter>
NOTE:
When specifying the field in the filter condition, you can omit the %<>. This means that the filter
below is equivalent to the example directly above:
<LogFilter>
<Name = "only_refresh_hits"/>
<Action = "ACCEPT"/>
<Condition = "pssc MATCH REFRESH_HIT"/>
</LogFilter>
The following is an example of a LogFilter that will cause the value of passwd field be wiped in cquc
<LogFilter>
<Name = "wipe_password"/>
<Condition = "cquc CONTAIN passwd"/>
<Action = "WIPE_FIELD_VALUE"/>
</LogFilter>
The following is an example of a LogObject specification that creates a local log file for the minimal
format defined earlier. The log filename will be minimal.log because this is an ASCII log file (the
default).:
<LogObject>
<Format = "minimal"/>
<Filename = "minimal"/>
</LogObject>
The following is an example of a LogObject specification that includes only HTTP requests served by hosts
in the domain company.com or by the specific server server.somewhere.com. Log entries are sent to port
4000 of the collation host logs.company.com and to port 5000 of the collation host 209.131.52.129.
<LogObject>
<Format = "minimal"/>
<Filename = "minimal"/>
<ServerHosts = "company.com,server.somewhere.com"/>
<Protocols = "http"/>
<CollationHosts = "logs.company.com:4000,209.131.52.129:5000"/>
</LogObject>
WELF
Traffic Server supports WELF (WebTrends Enhanced Log Format) so you can analyze Traffic Server log files
with WebTrends reporting tools. A predefined <LogFormat> that is compatible with WELF is provided in the
logs_xml.config file (shown below). To create a WELF format log file, create a <LogObject> that uses this
predefined format.
<LogFormat>
<Name = "welf"/>
<Format = "id=firewall time=\"%<cqtd> %<cqtt>\" fw=%<phn> pri=6
proto=%<cqus> duration=%<ttmsf> sent=%<psql> rcvd=%<cqhl>
src=%<chi> dst=%<shi> dstname=%<shn> user=%<caun> op=%<cqhm>
arg=\"%<cqup>\" result=%<pssc> ref=\"%<{Referer}cqh>\"
agent=\"%<{user-agent}cqh>\" cache=%<crc>"/>
</LogFormat>
COPYRIGHT
2014, dev@trafficserver.apache.org