Provided by: xdp-tools_1.4.2-1ubuntu4_amd64 
NAME
XDP-bench - a simple XDP benchmarking tool
SYNOPSIS
XDP-bench is a benchmarking utility for exercising the different operation modes of XDP. It is intended
to be a simple program demonstrating the various operating modes; these include dropping packets, hairpin
forwarding (using the XDP_TX return code), and redirection using the various in-kernel packet redirection
facilities.
The drop and TX modes support various options to control whether packet data is touched (read or written)
before being dropped or transmitted. The redirection modes support using the simple ifindex-based
bpf_redirect helper, the bpf_redirect_map helper using a cpumap as its target, bpf_redirect_map using a
devmap as its target, and the devmap's broadcast mode which allows redirecting to multiple devices.
There is more information on the meaning of the output in both default (terse) and extended output mode,
in the Output Format Description section below.
Running xdp-bench
The syntax for running xdp-bench is:
Usage: xdp-bench COMMAND [options]
COMMAND can be one of:
drop - Drop all packets on an interface
pass - Pass all packets to the network stack
tx - Transmit packets back out on an interface (hairpin forwarding)
redirect - XDP redirect using the bpf_redirect() helper
redirect-cpu - XDP CPU redirect using BPF_MAP_TYPE_CPUMAP
redirect-map - XDP redirect using BPF_MAP_TYPE_DEVMAP
redirect-multi - XDP multi-redirect using BPF_MAP_TYPE_DEVMAP and the BPF_F_BROADCAST flag
Each command, and its options are explained below. Or use xdp-bench COMMAND --help to see the options for
each command.
The DROP command
In this mode, xdp-bench installs an XDP program on an interface that simply drops all packets. There are
options to control what to do with the packet before dropping it (touch the packet data or not), as well
as which statistics to gather. This is a basic benchmark for the baseline (best-case) performance of XDP
on an interface.
The syntax for the drop command is:
xdp-bench drop [options] <ifname>
Where <ifname> is the name of the interface the XDP program should be installed on.
The supported options are:
-p, --packet-operation <ACTION>
Specify which operation should be taken on the packet before dropping it. The following actions are
available:
no-touch - Drop the packet without touching the packet data
touch - Read a field in the packet header before dropping
parse-ip - Parse the IP header field before dropping
swap-macs - Swap the source and destination MAC addresses before dropping
Whether to touch the packet before dropping it can have a significant performance impact as this requires
bringing packet data into the CPU cache (and flushing it back out if writing).
The default for this option is no-touch.
-l, --load-mode <MODE>
Specify which mechanism xdp-bench should use to load the packet data when parsing the IP header (used
with -p parse-ip). The following modes are available:
dpa - Use traditional Direct Packet Access from the XDP program
load-bytes - Use the xdp_load_bytes() helper function to load the data
This can be used to benchmark the various packet access modes supported by the kernel.
The default for this option is dpa.
-r, --rxq-stats
If set, the XDP program will also gather statistics on which receive queue index each packet was received
on. This is displayed in the extended output mode along with per-CPU data (which, depending on the
hardware configuration may or may not be equivalent).
-i, --interval <SECONDS>
Set the polling interval for collecting all statistics and displaying them to the output. The unit of
interval is in seconds.
-e, --extended
Start xdp-bench in "extended" output mode. If not set, xdp-bench will start in "terse" mode. The output
mode can be switched by hitting C-$ while the program is running. See also the Output Format Description
section below.
-m, --mode
Selects the XDP program mode (native or skb). Note that native XDP mode is the default, and loading the
redirect program in skb manner is neither performant, nor recommended. However, this option is useful if
the interface driver lacks native XDP support, or when simply testing the tool.
-v, --verbose
Enable verbose logging. Supply twice to enable verbose logging from the underlying libxdp and libbpf
libraries.
--version
Show the application version and exit.
-h, --help
Display a summary of the available options
The PASS command
In this mode, xdp-bench installs an XDP program on an interface that passes all packets to the network
stack after processing them (returning XDP_PASS). There are options to control what to do with the
packet before passing it (touch the packet data or not), as well as which statistics to gather. This is a
basic benchmark for the overhead of installing an XDP program on an interface while still running the
regular network stack.
The syntax for the pass command is:
xdp-bench pass [options] <ifname>
Where <ifname> is the name of the interface the XDP program should be installed on.
The supported options are:
-p, --packet-operation <ACTION>
Specify which operation should be taken on the packet before passing it. The following actions are
available:
no-touch - Pass the packet without touching the packet data
touch - Read a field in the packet header before passing
parse-ip - Parse the IP header field before passing
swap-macs - Swap the source and destination MAC addresses before passing
The default for this option is no-touch.
-l, --load-mode <MODE>
Specify which mechanism xdp-bench should use to load the packet data when parsing the IP header (used
with -p parse-ip). The following modes are available:
dpa - Use traditional Direct Packet Access from the XDP program
load-bytes - Use the xdp_load_bytes() helper function to load the data
This can be used to benchmark the various packet access modes supported by the kernel.
The default for this option is dpa.
-r, --rxq-stats
If set, the XDP program will also gather statistics on which receive queue index each packet was received
on. This is displayed in the extended output mode along with per-CPU data (which, depending on the
hardware configuration may or may not be equivalent).
-i, --interval <SECONDS>
Set the polling interval for collecting all statistics and displaying them to the output. The unit of
interval is in seconds.
-e, --extended
Start xdp-bench in "extended" output mode. If not set, xdp-bench will start in "terse" mode. The output
mode can be switched by hitting C-$ while the program is running. See also the Output Format Description
section below.
-m, --mode
Selects the XDP program mode (native or skb). Note that native XDP mode is the default, and loading the
redirect program in skb manner is neither performant, nor recommended. However, this option is useful if
the interface driver lacks native XDP support, or when simply testing the tool.
-v, --verbose
Enable verbose logging. Supply twice to enable verbose logging from the underlying libxdp and libbpf
libraries.
--version
Show the application version and exit.
-h, --help
Display a summary of the available options
The TX command
In this mode, xdp-bench installs an XDP program on an interface that performs so-called "hairpin
forwarding", which means each packet is transmitted back out the same interface (using the XDP_TX return
code).. There are options to control what to do with the packet before transmitting it (touch the packet
data or not), as well as which statistics to gather.
The syntax for the tx command is:
xdp-bench tx [options] <ifname>
Where <ifname> is the name of the interface the XDP program should be installed on.
The supported options are:
-p, --packet-operation <ACTION>
Specify which operation should be taken on the packet before transmitting it. The following actions are
available:
no-touch - Transmit the packet without touching the packet data
touch - Read a field in the packet header before transmitting
parse-ip - Parse the IP header field before transmitting
swap-macs - Swap the source and destination MAC addresses before transmitting
To allow the packet to be successfully transmitted back to the sender, the MAC addresses have to be
swapped, so that the source MAC matches the network device. However, there is a performance overhead in
doing swapping, so this option allows this function to be turned off.
The default for this option is swap-macs.
-l, --load-mode <MODE>
Specify which mechanism xdp-bench should use to load the packet data when parsing the IP header (used
with -p parse-ip). The following modes are available:
dpa - Use traditional Direct Packet Access from the XDP program
load-bytes - Use the xdp_load_bytes() helper function to load the data
This can be used to benchmark the various packet access modes supported by the kernel.
The default for this option is dpa.
-r, --rxq-stats
If set, the XDP program will also gather statistics on which receive queue index each packet was received
on. This is displayed in the extended output mode along with per-CPU data (which, depending on the
hardware configuration may or may not be equivalent).
-i, --interval <SECONDS>
Set the polling interval for collecting all statistics and displaying them to the output. The unit of
interval is in seconds.
-e, --extended
Start xdp-bench in "extended" output mode. If not set, xdp-bench will start in "terse" mode. The output
mode can be switched by hitting C-$ while the program is running. See also the Output Format Description
section below.
-m, --mode
Selects the XDP program mode (native or skb). Note that native XDP mode is the default, and loading the
redirect program in skb manner is neither performant, nor recommended. However, this option is useful if
the interface driver lacks native XDP support, or when simply testing the tool.
-v, --verbose
Enable verbose logging. Supply twice to enable verbose logging from the underlying libxdp and libbpf
libraries.
--version
Show the application version and exit.
-h, --help
Display a summary of the available options
The REDIRECT command
In this mode, xdp-bench sets up packet redirection between the two interfaces supplied on the command
line using the bpf_redirect BPF helper triggered on packet reception on the ingress interface.
The syntax for the redirect command is:
xdp-bench redirect [options] <ifname_in> <ifname_out>
Where <ifname_in> is the name of the input interface from where packets will be redirect to the output
interface <ifname_out>.
The supported options are:
-i, --interval <SECONDS>
Set the polling interval for collecting all statistics and displaying them to the output. The unit of
interval is in seconds.
-s, --stats
Enable statistics for successful redirection. This option comes with a per packet tracing overhead, for
recording all successful redirections.
-e, --extended
Start xdp-bench in "extended" output mode. If not set, xdp-bench will start in "terse" mode. The output
mode can be switched by hitting C-$ while the program is running. See also the Output Format Description
section below.
-m, --mode
Selects the XDP program mode (native or skb). Note that native XDP mode is the default, and loading the
redirect program in skb manner is neither performant, nor recommended. However, this option is useful if
the interface driver lacks native XDP support, or when simply testing the tool.
-v, --verbose
Enable verbose logging. Supply twice to enable verbose logging from the underlying libxdp and libbpf
libraries.
--version
Show the application version and exit.
-h, --help
Display a summary of the available options
The REDIRECT-CPU command
In this mode, xdp-bench sets up packet redirection using the bpf_redirect_map BPF helper triggered on
packet reception on the ingress interface, using a cpumap as its target. Hence, this tool can be used to
redirect packets on an interface from one CPU to another. In addition to this, the tool then supports
redirecting the packet to another output device when it is processed on the target CPU.
The syntax for the redirect-cpu command is:
xdp-bench redirect-cpu [options] <ifname> -c 0 ... -c N
Where <ifname> is the name of the input interface from where packets will be redirect to the target CPU
list specified using -c.
The supported options are:
-c, --cpu <CPU>
Specify a possible target CPU index. This option must be passed at least once, and can be passed multiple
times to specify a list of CPUs. Which CPU is chosen for a given packet depends on the value of the
--program-mode option, described below.
-p, --program-mode <MODE>
Specify a program that embeds a predefined policy deciding how packets are redirected to different CPUs.
The following options are available:
no-touch - Redirect without touching packet data
touch - Read packet data before redirecting
round-robin - Cycle between target CPUs in a round-robin fashion (for each packet)
l4-proto - Choose the target CPU based on the layer-4 protocol of packet
l4-filter - Like l4-proto, but drop UDP packets with destination port 9 (used by pktgen)
l4-hash - Use source and destination IP hashing to pick target CPU
l4-sport - Use modulo of source port to pick target CPU
l4-dport - Use modulo of destination port to pick target CPU
The no-touch and touch modes always redirect packets to the same CPU (the first value supplied to --cpu).
The round-robin and l4-hash modes distribute packets between all the CPUs supplied as --cpu arguments,
while l4-proto and l4-filter send TCP and unrecognised packets to CPU index 0, UDP packets to CPU index 1
and ICMP packets to CPU index 2 (where the index refers to the order the actual CPUs are given on the
command line).
The default for this option is l4-hash.
-r --remote-action <ACTION>
If this option is set, a separate program is installed into the cpumap, which will be invoked on the
remote CPU after the packet is processed there. The action can be either drop or pass which will drop the
packet or pass it to the regular networking stack, respectively. Or it can be redirect, which will cause
the packet to be redirected to another interface and transmitted out that interface on the remote CPU. If
this option is set to redirect the target device must be specified using --redirect-device.
The default for this option is disabled.
-r, --redirect-device <IFNAME>
Specify the device to redirect the packet to when it is received on the target CPU. Note that this
option can only be specified with --remote-action redirect.
-q, --qsize <PACKETS>
Set the queue size for the per-CPU cpumap ring buffer used for redirecting packets from multiple CPUs to
one CPU. The default value is 2048 packets.
-x, --stress-mode
Stress the cpumap implementation by deallocating and reallocating the cpumap ring buffer on each polling
interval.
-i, --interval <SECONDS>
Set the polling interval for collecting all statistics and displaying them to the output. The unit of
interval is in seconds.
-s, --stats
Enable statistics for successful redirection. This option comes with a per packet tracing overhead, for
recording all successful redirections.
-e, --extended
Start xdp-bench in "extended" output mode. If not set, xdp-bench will start in "terse" mode. The output
mode can be switched by hitting C-$ while the program is running. See also the Output Format Description
section below.
-m, --mode
Selects the XDP program mode (native or skb). Note that native XDP mode is the default, and loading the
redirect program in skb manner is neither performant, nor recommended. However, this option is useful if
the interface driver lacks native XDP support, or when simply testing the tool.
-v, --verbose
Enable verbose logging. Supply twice to enable verbose logging from the underlying libxdp and libbpf
libraries.
--version
Show the application version and exit.
-h, --help
Display a summary of the available options
The REDIRECT-MAP command
In this mode, xdp-bench sets up packet redirection between two interfaces supplied on the command line
using the bpf_redirect_map() BPF helper triggered on packet reception on the ingress interface, using a
devmap as its target.
The syntax for the redirect-map command is:
xdp-bench redirect-map [options] <ifname_in> <ifname_out>
Where <ifname_in> is the name of the input interface from where packets will be redirect to the output
interface <ifname_out>.
The supported options are:
-X, --load-egress
Load a program in the devmap entry used for redirection, so that it is invoked after the packet is
redirected to the target device, before it is transmitted out of the output interface. The remote program
will update the packet data so its source MAC address matches the one of the destination interface.
-i, --interval <SECONDS>
Set the polling interval for collecting all statistics and displaying them to the output. The unit of
interval is in seconds.
-s, --stats
Enable statistics for successful redirection. This option comes with a per packet tracing overhead, for
recording all successful redirections.
-e, --extended
Start xdp-bench in "extended" output mode. If not set, xdp-bench will start in "terse" mode. The output
mode can be switched by hitting C-$ while the program is running. See also the Output Format Description
section below.
-m, --mode
Selects the XDP program mode (native or skb). Note that native XDP mode is the default, and loading the
redirect program in skb manner is neither performant, nor recommended. However, this option is useful if
the interface driver lacks native XDP support, or when simply testing the tool.
-v, --verbose
Enable verbose logging. Supply twice to enable verbose logging from the underlying libxdp and libbpf
libraries.
--version
Show the application version and exit.
-h, --help
Display a summary of the available options
The REDIRECT-MULTI command
In this mode, xdp-bench sets up one-to-many packet redirection between interfaces supplied on the command
line, using the bpf_redirect_map BPF helper triggered on packet reception on the ingress interface, using
a devmap as its target. The packet is broadcast to all output interfaces specified on the command line,
using devmap's packet broadcast feature.
The syntax for the redirect-multi command is:
xdp-bench redirect-multi [options] <ifname_in> <ifname_out1> ... <ifname_outN>
Where <ifname_in> is the name of the input interface from where packets will be redirect to one or many
output interface(s).
The supported options are:
-X, --load-egress
Load a program in the devmap entry used for redirection, so that it is invoked after the packet is
redirected to the target device, before it is transmitted out of the output interface. The remote program
will update the packet data so its source MAC address matches the one of the destination interface.
-i, --interval <SECONDS>
Set the polling interval for collecting all statistics and displaying them to the output. The unit of
interval is in seconds.
-s, --stats
Enable statistics for successful redirection. This option comes with a per packet tracing overhead, for
recording all successful redirections.
-e, --extended
Start xdp-bench in "extended" output mode. If not set, xdp-bench will start in "terse" mode. The output
mode can be switched by hitting C-$ while the program is running. See also the Output Format Description
section below.
-m, --mode
Selects the XDP program mode (native or skb). Note that native XDP mode is the default, and loading the
redirect program in skb manner is neither performant, nor recommended. However, this option is useful if
the interface driver lacks native XDP support, or when simply testing the tool.
-v, --verbose
Enable verbose logging. Supply twice to enable verbose logging from the underlying libxdp and libbpf
libraries.
--version
Show the application version and exit.
-h, --help
Display a summary of the available options
Output Format Description
By default, redirect success statistics are disabled, use --stats to enable. The terse output mode is
default, extended output mode can be activated using the --extended command line option.
SIGQUIT (Ctrl + \) can be used to switch the mode dynamically at runtime.
Terse mode displays at most the following fields:
rx/s Number of packets received per second
redir/s Number of packets successfully redirected per second
err,drop/s Aggregated count of errors per second (including dropped packets when not using the drop command)
xmit/s Number of packets transmitted on the output device per second
Extended output mode displays at most the following fields:
FIELD DESCRIPTION
receive Displays the number of packets received and errors encountered
Whenever an error or packet drop occurs, details of per CPU error
and drop statistics will be expanded inline in terse mode.
pkt/s - Packets received per second
drop/s - Packets dropped per second
error/s - Errors encountered per second
redirect - Displays the number of packets successfully redirected
Errors encountered are expanded under redirect_err field
Note that passing -s to enable it has a per packet overhead
redir/s - Packets redirected successfully per second
redirect_err Displays the number of packets that failed redirection
The errno is expanded under this field with per CPU count
The recognized errors are:
EINVAL: Invalid redirection
ENETDOWN: Device being redirected to is down
EMSGSIZE: Packet length too large for device
EOPNOTSUPP: Operation not supported
ENOSPC: No space in ptr_ring of cpumap kthread
error/s - Packets that failed redirection per second
enqueue to cpu N Displays the number of packets enqueued to bulk queue of CPU N
Expands to cpu:FROM->N to display enqueue stats for each CPU enqueuing to CPU N
Received packets can be associated with the CPU redirect program is enqueuing
packets to.
pkt/s - Packets enqueued per second from other CPU to CPU N
drop/s - Packets dropped when trying to enqueue to CPU N
bulk-avg - Average number of packets processed for each event
kthread Displays the number of packets processed in CPUMAP kthread for each CPU
Packets consumed from ptr_ring in kthread, and its xdp_stats (after calling
CPUMAP bpf prog) are expanded below this. xdp_stats are expanded as a total and
then per-CPU to associate it to each CPU's pinned CPUMAP kthread.
pkt/s - Packets consumed per second from ptr_ring
drop/s - Packets dropped per second in kthread
sched - Number of times kthread called schedule()
xdp_stats (also expands to per-CPU counts)
pass/s - XDP_PASS count for CPUMAP program execution
drop/s - XDP_DROP count for CPUMAP program execution
redir/s - XDP_REDIRECT count for CPUMAP program execution
xdp_exception Displays xdp_exception tracepoint events
This can occur due to internal driver errors, unrecognized
XDP actions and due to explicit user trigger by use of XDP_ABORTED
Each action is expanded below this field with its count
hit/s - Number of times the tracepoint was hit per second
devmap_xmit Displays devmap_xmit tracepoint events
This tracepoint is invoked for successful transmissions on output
device but these statistics are not available for generic XDP mode,
hence they will be omitted from the output when using SKB mode
xmit/s - Number of packets that were transmitted per second
drop/s - Number of packets that failed transmissions per second
drv_err/s - Number of internal driver errors per second
bulk-avg - Average number of packets processed for each event
BUGS
Please report any bugs on Github: https://github.com/xdp-project/xdp-tools/issues
AUTHOR
Earlier xdp-redirect tools were written by Jesper Dangaard Brouer and John Fastabend. They were then
rewritten to support more features by Kumar Kartikeya Dwivedi, who also ported them to xdp-tools together
with Toke Høiland-Jørgensen. This man page was written by Kumar Kartikeya Dwivedi and Toke Høiland-
Jørgensen.
V1.4.2 OCTOBER 26, 2023 xdp-bench(8)