Ubuntu Manpage: XDP-bench - a simple XDP benchmarking tool

Provided by: xdp-tools_1.3.0-2ubuntu2_amd64

NAME

       XDP-bench - a simple XDP benchmarking tool

SYNOPSIS

       XDP-bench  is  a benchmarking utility for exercising the different operation modes of XDP.
       It is intended to be a simple program demonstrating the  various  operating  modes;  these
       include  dropping  packets,  hairpin  forwarding  (using  the  XDP_TX  return  code),  and
       redirection using the various in-kernel packet redirection facilities.

       The drop and TX modes support various options to control whether packet  data  is  touched
       (read or written) before being dropped or transmitted. The redirection modes support using
       the simple ifindex-based bpf_redirect helper, the bpf_redirect_map helper using  a  cpumap
       as  its  target, bpf_redirect_map using a devmap as its target, and the devmap's broadcast
       mode which allows redirecting to multiple devices.

       There is more information on the meaning  of  the  output  in  both  default  (terse)  and
       extended output mode, in the Output Format Description section below.

   Running xdp-bench
       The syntax for running xdp-bench is:

              Usage: xdp-bench COMMAND [options]

              COMMAND can be one of:
                     drop           - Drop all packets on an interface
                     tx             - Transmit packets back out on an interface (hairpin forwarding)
                     redirect       - XDP redirect using the bpf_redirect() helper
                     redirect-cpu   - XDP CPU redirect using BPF_MAP_TYPE_CPUMAP
                     redirect-map   - XDP redirect using BPF_MAP_TYPE_DEVMAP
                     redirect-multi - XDP multi-redirect using BPF_MAP_TYPE_DEVMAP and the BPF_F_BROADCAST flag

       Each  command, and its options are explained below. Or use xdp-bench COMMAND --help to see
       the options for each command.

The DROP command

In this mode, xdp-bench installs an XDP program on an interface that simply drops all
packets. There are options to control what to do with the packet before dropping it (touch
the packet data or not), as well as which statistics to gather. This is a basic benchmark
for the baseline (best-case) performance of XDP on an interface.

The syntax for the drop command is:

xdp-bench drop [options] <ifname>

Where <ifname> is the name of the interface the XDP program should be installed on.

The supported options are:

-p, --packet-operation <ACTION>
Specify which operation should be taken on the packet before dropping it. The following
actions are available:

no-touch - Drop the packet without touching the packet data
touch - Read a field in the packet header before dropping
swap-macs - Swap the source and destination MAC addresses before dropping

Whether to touch the packet before dropping it can have a significant performance impact
as this requires bringing packet data into the CPU cache (and flushing it back out if
writing).

The default for this option is no-touch.

-r, --rxq-stats
If set, the XDP program will also gather statistics on which receive queue index each
packet was received on. This is displayed in the extended output mode along with per-CPU
data (which, depending on the hardware configuration may or may not be equivalent).

-i, --interval <SECONDS>
Set the polling interval for collecting all statistics and displaying them to the output.
The unit of interval is in seconds.

-e, --extended
Start xdp-bench in "extended" output mode. If not set, xdp-bench will start in "terse"
mode. The output mode can be switched by hitting C-$ while the program is running. See
also the Output Format Description section below.

-m, --mode
Selects the XDP program mode (native or skb). Note that native XDP mode is the default,
and loading the redirect program in skb manner is neither performant, nor recommended.
However, this option is useful if the interface driver lacks native XDP support, or when
simply testing the tool.

-v, --verbose
Enable verbose logging. Supply twice to enable verbose logging from the underlying libxdp
and libbpf libraries.

--version
Show the application version and exit.

-h, --help
Display a summary of the available options

The PASS command

In this mode, xdp-bench installs an XDP program on an interface that passes all packets to
the network stack after processing them (returning XDP_PASS). There are options to
control what to do with the packet before passing it (touch the packet data or not), as
well as which statistics to gather. This is a basic benchmark for the overhead of
installing an XDP program on an interface while still running the regular network stack.

The syntax for the pass command is:

xdp-bench pass [options] <ifname>

Where <ifname> is the name of the interface the XDP program should be installed on.

The supported options are:

-p, --packet-operation <ACTION>
Specify which operation should be taken on the packet before passing it. The following
actions are available:

no-touch - Pass the packet without touching the packet data
touch - Read a field in the packet header before passing
swap-macs - Swap the source and destination MAC addresses before passing

The default for this option is no-touch.

-i, --interval <SECONDS>
Set the polling interval for collecting all statistics and displaying them to the output.
The unit of interval is in seconds.

-v, --verbose
Enable verbose logging. Supply twice to enable verbose logging from the underlying libxdp
and libbpf libraries.

--version
Show the application version and exit.

-h, --help
Display a summary of the available options

The TX command

In this mode, xdp-bench installs an XDP program on an interface that performs so-called
"hairpin forwarding", which means each packet is transmitted back out the same interface
(using the XDP_TX return code).. There are options to control what to do with the packet
before transmitting it (touch the packet data or not), as well as which statistics to
gather.

The syntax for the tx command is:

xdp-bench tx [options] <ifname>

Where <ifname> is the name of the interface the XDP program should be installed on.

The supported options are:

-p, --packet-operation <ACTION>
Specify which operation should be taken on the packet before transmitting it. The
following actions are available:

no-touch - Transmit the packet without touching the packet data
touch - Read a field in the packet header before transmitting
swap-macs - Swap the source and destination MAC addresses before transmitting

To allow the packet to be successfully transmitted back to the sender, the MAC addresses
have to be swapped, so that the source MAC matches the network device. However, there is
a performance overhead in doing swapping, so this option allows this function to be turned
off.

The default for this option is swap-macs.

-i, --interval <SECONDS>
Set the polling interval for collecting all statistics and displaying them to the output.
The unit of interval is in seconds.

-v, --verbose
Enable verbose logging. Supply twice to enable verbose logging from the underlying libxdp
and libbpf libraries.

--version
Show the application version and exit.

-h, --help
Display a summary of the available options

The REDIRECT command

In this mode, xdp-bench sets up packet redirection between the two interfaces supplied on
the command line using the bpf_redirect BPF helper triggered on packet reception on the
ingress interface.

The syntax for the redirect command is:

xdp-bench redirect [options] <ifname_in> <ifname_out>

Where <ifname_in> is the name of the input interface from where packets will be redirect
to the output interface <ifname_out>.

The supported options are:

-i, --interval <SECONDS>
Set the polling interval for collecting all statistics and displaying them to the output.
The unit of interval is in seconds.

-s, --stats
Enable statistics for successful redirection. This option comes with a per packet tracing
overhead, for recording all successful redirections.

-v, --verbose
Enable verbose logging. Supply twice to enable verbose logging from the underlying libxdp
and libbpf libraries.

--version
Show the application version and exit.

-h, --help
Display a summary of the available options

The REDIRECT-CPU command

In this mode, xdp-bench sets up packet redirection using the bpf_redirect_map BPF helper
triggered on packet reception on the ingress interface, using a cpumap as its target.
Hence, this tool can be used to redirect packets on an interface from one CPU to another.
In addition to this, the tool then supports redirecting the packet to another output
device when it is processed on the target CPU.

The syntax for the redirect-cpu command is:

xdp-bench redirect-cpu [options] <ifname> -c 0 ... -c N

Where <ifname> is the name of the input interface from where packets will be redirect to
the target CPU list specified using -c.

The supported options are:

-c, --cpu <CPU>
Specify a possible target CPU index. This option must be passed at least once, and can be
passed multiple times to specify a list of CPUs. Which CPU is chosen for a given packet
depends on the value of the --program-mode option, described below.

-p, --program-mode <MODE>
Specify a program that embeds a predefined policy deciding how packets are redirected to
different CPUs. The following options are available:

no-touch - Redirect without touching packet data
touch - Read packet data before redirecting
round-robin - Cycle between target CPUs in a round-robin fashion (for each packet)
l4-proto - Choose the target CPU based on the layer-4 protocol of packet
l4-filter - Like l4-proto, but drop UDP packets with destination port 9 (used by pktgen)
l4-hash - Use source and destination IP hashing to pick target CPU

The no-touch and touch modes always redirect packets to the same CPU (the first value
supplied to --cpu). The round-robin and l4-hash modes distribute packets between all the
CPUs supplied as --cpu arguments, while l4-proto and l4-filter send TCP and unrecognised
packets to CPU index 0, UDP packets to CPU index 1 and ICMP packets to CPU index 2 (where
the index refers to the order the actual CPUs are given on the command line).

The default for this option is l4-hash.

-r --remote-action <ACTION>
If this option is set, a separate program is installed into the cpumap, which will be
invoked on the remote CPU after the packet is processed there. The action can be either
drop or pass which will drop the packet or pass it to the regular networking stack,
respectively. Or it can be redirect, which will cause the packet to be redirected to
another interface and transmitted out that interface on the remote CPU. If this option is
set to redirect the target device must be specified using --redirect-device.

The default for this option is disabled.

-r, --redirect-device <IFNAME>
Specify the device to redirect the packet to when it is received on the target CPU. Note
that this option can only be specified with --remote-action redirect.

-q, --qsize <PACKETS>
Set the queue size for the per-CPU cpumap ring buffer used for redirecting packets from
multiple CPUs to one CPU. The default value is 2048 packets.

-x, --stress-mode
Stress the cpumap implementation by deallocating and reallocating the cpumap ring buffer
on each polling interval.

-i, --interval <SECONDS>
Set the polling interval for collecting all statistics and displaying them to the output.
The unit of interval is in seconds.

-s, --stats
Enable statistics for successful redirection. This option comes with a per packet tracing
overhead, for recording all successful redirections.

-v, --verbose
Enable verbose logging. Supply twice to enable verbose logging from the underlying libxdp
and libbpf libraries.

--version
Show the application version and exit.

-h, --help
Display a summary of the available options

The REDIRECT-MAP command

In this mode, xdp-bench sets up packet redirection between two interfaces supplied on the
command line using the bpf_redirect_map() BPF helper triggered on packet reception on the
ingress interface, using a devmap as its target.

The syntax for the redirect-map command is:

xdp-bench redirect-map [options] <ifname_in> <ifname_out>

Where <ifname_in> is the name of the input interface from where packets will be redirect
to the output interface <ifname_out>.

The supported options are:

-X, --load-egress
Load a program in the devmap entry used for redirection, so that it is invoked after the
packet is redirected to the target device, before it is transmitted out of the output
interface. The remote program will update the packet data so its source MAC address
matches the one of the destination interface.

-i, --interval <SECONDS>
Set the polling interval for collecting all statistics and displaying them to the output.
The unit of interval is in seconds.

-s, --stats
Enable statistics for successful redirection. This option comes with a per packet tracing
overhead, for recording all successful redirections.

-v, --verbose
Enable verbose logging. Supply twice to enable verbose logging from the underlying libxdp
and libbpf libraries.

--version
Show the application version and exit.

-h, --help
Display a summary of the available options

The REDIRECT-MULTI command

In this mode, xdp-bench sets up one-to-many packet redirection between interfaces supplied
on the command line, using the bpf_redirect_map BPF helper triggered on packet reception
on the ingress interface, using a devmap as its target. The packet is broadcast to all
output interfaces specified on the command line, using devmap's packet broadcast feature.

The syntax for the redirect-multi command is:

xdp-bench redirect-multi [options] <ifname_in> <ifname_out1> ... <ifname_outN>

Where <ifname_in> is the name of the input interface from where packets will be redirect
to one or many output interface(s).

The supported options are:

-i, --interval <SECONDS>
Set the polling interval for collecting all statistics and displaying them to the output.
The unit of interval is in seconds.

-s, --stats
Enable statistics for successful redirection. This option comes with a per packet tracing
overhead, for recording all successful redirections.

-v, --verbose
Enable verbose logging. Supply twice to enable verbose logging from the underlying libxdp
and libbpf libraries.

--version
Show the application version and exit.

-h, --help
Display a summary of the available options

Output Format Description

       By default, redirect success statistics are disabled, use --stats to  enable.   The  terse
       output mode is default, extended output mode can be activated using the --extended command
       line option.

       SIGQUIT (Ctrl + \) can be used to switch the mode dynamically at runtime.

       Terse mode displays at most the following fields:
              rx/s      Number of packets received per second
              redir/s   Number of packets successfully redirected per second
              err,drop/s     Aggregated count of errors per second (including dropped packets when not using the drop command)
              xmit/s    Number of packets transmitted on the output device per second

       Extended output mode displays at most the following fields:
              FIELD            DESCRIPTION
              receive          Displays the number of packets received and errors encountered

                               Whenever an error or packet drop occurs, details of per CPU error
                               and drop statistics will be expanded inline in terse mode.
                                         pkt/s        - Packets received per second
                                         drop/s       - Packets dropped per second
                                         error/s      - Errors encountered per second
                                         redirect     - Displays the number of packets successfully redirected
                               Errors encountered are expanded under redirect_err field
                               Note that passing -s to enable it has a per packet overhead
                                         redir/s      - Packets redirected successfully per second

              redirect_err     Displays the number of packets that failed redirection

                               The errno is expanded under this field with per CPU count
                               The recognized errors are:
                                         EINVAL:      Invalid redirection
                                         ENETDOWN:    Device being redirected to is down
                                         EMSGSIZE:    Packet length too large for device
                                         EOPNOTSUPP:  Operation not supported
                                         ENOSPC:      No space in ptr_ring of cpumap kthread

                                         error/s      - Packets that failed redirection per second

              enqueue to cpu N Displays the number of packets enqueued to bulk queue of CPU N
                               Expands to cpu:FROM->N to display enqueue stats for each CPU enqueuing to CPU N
                               Received packets can be associated with the CPU redirect program is enqueuing
                               packets to.
                                         pkt/s        - Packets enqueued per second from other CPU to CPU N
                                         drop/s       - Packets dropped when trying to enqueue to CPU N
                                         bulk-avg     - Average number of packets processed for each event

              kthread          Displays the number of packets processed in CPUMAP kthread for each CPU
                               Packets consumed from ptr_ring in kthread, and its xdp_stats (after calling
                               CPUMAP bpf prog) are expanded below this. xdp_stats are expanded as a total and
                               then per-CPU to associate it to each CPU's pinned CPUMAP kthread.
                                         pkt/s        - Packets consumed per second from ptr_ring
                                         drop/s       - Packets dropped per second in kthread
                                         sched        - Number of times kthread called schedule()

                               xdp_stats (also expands to per-CPU counts)
                                         pass/s       - XDP_PASS count for CPUMAP program execution
                                         drop/s       - XDP_DROP count for CPUMAP program execution
                                         redir/s      - XDP_REDIRECT count for CPUMAP program execution

              xdp_exception    Displays xdp_exception tracepoint events

                               This can occur due to internal driver errors, unrecognized
                               XDP actions and due to explicit user trigger by use of XDP_ABORTED
                               Each action is expanded below this field with its count
                                         hit/s        - Number of times the tracepoint was hit per second

              devmap_xmit      Displays devmap_xmit tracepoint events

                               This tracepoint is invoked for successful transmissions on output
                               device but these statistics are not available for generic XDP mode,
                               hence they will be omitted from the output when using SKB mode
                                         xmit/s       - Number of packets that were transmitted per second
                                         drop/s       - Number of packets that failed transmissions per second
                                         drv_err/s    - Number of internal driver errors per second
                                         bulk-avg     - Average number of packets processed for each event

BUGS

       Please report any bugs on Github: https://github.com/xdp-project/xdp-tools/issues

AUTHOR

       Earlier xdp-redirect tools were written by Jesper Dangaard Brouer and John Fastabend. They
       were  then  rewritten to support more features by Kumar Kartikeya Dwivedi, who also ported
       them to xdp-tools together with Toke Høiland-Jørgensen.  This  man  page  was  written  by
       Kumar Kartikeya Dwivedi and Toke Høiland-Jørgensen.