Ubuntu Manpage: stafd.conf - stafd(8) configuration file

NAME

       stafd.conf - stafd(8) configuration file

SYNOPSIS

       /etc/stas/stafd.conf

DESCRIPTION

       When stafd(8) starts up, it reads its configuration from stafd.conf.

CONFIGURATION FILE FORMAT

       stafd.conf is a plain text file divided into sections, with configuration entries in the
       style key=value. Spaces immediately before or after the = are ignored. Empty lines are
       ignored as well as lines starting with #, which may be used for commenting.

OPTIONS

[Global] section
The following options are available in the [Global] section:

tron=
Trace ON. Takes a boolean argument. If true, enables full code tracing. The trace will
be displayed in the system log such as systemd's journal. Defaults to false.

hdr-digest=
Enable Protocol Data Unit (PDU) Header Digest. Takes a boolean argument. NVMe/TCP
facilitates an optional PDU Header digest. Digests are calculated using the CRC32C
algorithm. If true, Header Digests are inserted in PDUs and checked for errors.
Defaults to false.

data-digest=
Enable Protocol Data Unit (PDU) Data Digest. Takes a boolean argument. NVMe/TCP
facilitates an optional PDU Data digest. Digests are calculated using the CRC32C
algorithm. If true, Data Digests are inserted in PDUs and checked for errors. Defaults
to false.

kato=
Keep Alive Timeout (KATO) in seconds. Takes an unsigned integer. This field specifies
the timeout value for the Keep Alive feature in seconds. Defaults to 30 seconds for
Discovery Controller connections and 120 seconds for I/O Controller connections.

ip-family=
Takes a string argument. With this you can specify whether IPv4, IPv6, or both are
supported when connecting to a Controller. Connections will not be attempted to IP
addresses (whether discovered or manually configured with controller=) disabled by
this option. If an invalid value is entered, then the default (see below) will apply.

Choices are ipv4, ipv6, or ipv4+ipv6.

Defaults to ipv4+ipv6.

queue-size=
Takes a value in the range 16...1024.

Overrides the default number of elements in the I/O queues created by the driver. This
option will be ignored for discovery, but will be passed on to the subsequent connect
call.

Note: This parameter is identical to that provided by nvme-cli.

Defaults to 128.

reconnect-delay=
Takes a value in the range 1 to N seconds.

Overrides the default delay before reconnect is attempted after a connect loss.

Note: This parameter is identical to that provided by nvme-cli.

Defaults to 10. Retry to connect every 10 seconds.

ctrl-loss-tmo=
Takes a value in the range -1, 0, ..., N seconds. -1 means retry forever. 0 means do
not retry.

Overrides the default controller loss timeout period (in seconds).

Note: This parameter is identical to that provided by nvme-cli.

Defaults to 600 seconds (10 minutes).

disable-sqflow=
Takes a boolean argument. Disables SQ flow control to omit head doorbell update for
submission queues when sending nvme completions.

Note: This parameter is identical to that provided by nvme-cli.

Defaults to false.

ignore-iface=
Takes a boolean argument. This option controls how connections with Discovery
Controllers (DC) are made.

DCs are automatically discovered using DNS-SD/mDNS. mDNS provides the DC's IP address
and the interface on which the DC was discovered.

There is no guarantee that there will be a route to reach that DC. However, we can use
the socket option SO_BINDTODEVICE to force the connection to be made on a specific
interface instead of letting the routing tables decide where to make the connection.

This option determines whether stafd will use SO_BINDTODEVICE to force connections on
an interface or just rely on the routing tables. The default is to use
SO_BINDTODEVICE, in other words, stafd does not ignore the interface by default.

Defaults to false.

pleo=
Port Local Entries Only. Takes a string argument enabled or disabled. This option is
sent in the LSP field (Log SPecific) of the Get Discovery Log Page (DLP) command. It
is used by stafd to tell Discovery Controllers (DC) whether the response to a Get DLP
command should contain all the NVM subsystems or only those reachable by the host on
the interface where the Get DLP command was issued by the host.

This parameter was introduced in TP8010. When pleo=enabled, then the DC shall return
records for only NVM subsystem ports that are presented through the same NVM subsystem
port that received the Get Log Page command. When pleo=disabled, then the DC may
return all the NVM subsystem ports that it holds, even those that can only be reached
on NVM subsystem ports that did not receive the Get Log Page command. In other words,
the host may not even be able to reach those subsystems.

Defaults to enabled.

[Service Discovery] section
The following options are available in the [Service Discovery] section:

zeroconf=
Enable zeroconf provisioning using DNS-SD/mDNS. Takes a string argument enabled or
disabled.

When enabled, the default, stafd makes a request with the Avahi daemon to locate
Discovery Controllers using DNS-SD/mDNS.

Discovery Controllers that support zeroconf advertise themselves over mDNS with the
service type _nvme-disc._tcp.

Defaults to true.

[Discovery controller connection management] section
The following options are available in the [Discovery controller connection management]
section:

persistent-connections=
Takes a boolean argument. Whether connections to Discovery Controllers (DC) are
persistent. When true, connections initiated by stafd will persists even when stafd is
stopped. When false, stafd will disconnect from all DCs it is connected to on exit.

Defaults to false.

zeroconf-connections-persistence=
Takes a unit-less value in seconds, or a time span value such as "72hours" or "5days".
A value of 0 means no persistence. In other words, configuration acquired through
zeroconf (mDNS service discovery) will be removed immediately when mDNS no longer
reports the presence of a Discovery Controller (DC) and connectivity to that DC is
lost. A value of -1 means that configuration acquired through zeroconf will persist
forever.

This is used for the case where a DC that was discovered through mDNS service
discovery no longer advertises itself through mDNS and can no longer be connected to.
For example, the DC had some catastrophic failure (e.g. power surge) and needs to be
replaced. In that case, the connection to that DC can never be restored and a
replacement DC will be needed. The replacement DC will likely have a different NQN (or
IP address). In that scenario, the host won't be able to determine that the old DC is
not coming back. It won't know either that a newly discovered DC is really the
replacement for the old one. For that reason, the host needs a way to "age"
zeroconf-acquired configuration and remove it automatically after a certain amount of
time. This is what this parameter is for.

Defaults to 72hours.

[Controllers] section
The following options are available in the [Controllers] section:

controller=
Controllers are specified with the controller option. This option may be specified
more than once to specify more than one controller. The format is one line per
Controller composed of a series of fields separated by semi-colons as follows:

controller=transport=[trtype];traddr=[traddr];trsvcid=[trsvcid];host-traddr=[traddr],host-iface=[iface];nqn=[nqn]

Fields
transport=
This is a mandatory field that specifies the network fabric being used for a
NVMe-over-Fabrics network. Current trtype values understood are:

Table 1. Transport type
┌───────┬────────────────────────────────┐
│trtype │ Definition │
├───────┼────────────────────────────────┤
│rdma │ The network fabric is an rdma │
│ │ network (RoCE, iWARP, │
│ │ Infiniband, basic rdma, etc) │
├───────┼────────────────────────────────┤
│fc │ The network fabric is a Fibre │
│ │ Channel network. │
├───────┼────────────────────────────────┤
│tcp │ The network fabric is a TCP/IP │
│ │ network. │
├───────┼────────────────────────────────┤
│loop │ Connect to a NVMe over Fabrics │
│ │ target on the local host │
└───────┴────────────────────────────────┘

traddr=
This is a mandatory field that specifies the network address of the
Controller. For transports using IP addressing (e.g. rdma) this should be an
IP-based address (ex. IPv4, IPv6). It could also be a resolvable host name
(e.g. localhost).

trsvcid=
This is an optional field that specifies the transport service id. For
transports using IP addressing (e.g. rdma, tcp) this field is the port number.

Depending on the transport type, this field will default to either 8009 or
4420 as follows.

UDP port 4420 and TCP port 4420 have been assigned by IANA for use by NVMe
over Fabrics. NVMe/RoCEv2 controllers use UDP port 4420 by default. NVMe/iWARP
controllers use TCP port 4420 by default.

TCP port 4420 has been assigned for use by NVMe over Fabrics and TCP port 8009
has been assigned by IANA for use by NVMe over Fabrics discovery. TCP port
8009 is the default TCP port for NVMe/TCP discovery controllers. There is no
default TCP port for NVMe/TCP I/O controllers, the Transport Service
Identifier (TRSVCID) field in the Discovery Log Entry indicates the TCP port
to use.

The TCP ports that may be used for NVMe/TCP I/O controllers include TCP port
4420, and the Dynamic and/or Private TCP ports (i.e., ports in the TCP port
number range from 49152 to 65535). NVMe/TCP I/O controllers should not use TCP
port 8009. TCP port 4420 shall not be used for both NVMe/iWARP and NVMe/TCP at
the same IP address on the same network.

Ref: IANA Service names port numbers[1]

nqn=
This field specifies the Controller's NVMe Qualified Name.

This field is mandatory for I/O Controllers, but is optional for Discovery
Controllers (DC). For the latter, the NQN will default to the well-known DC
NQN: nqn.2014-08.org.nvmexpress.discovery if left undefined.

host-traddr=
This is an optional field that specifies the network address used on the host
to connect to the Controller. For TCP, this sets the source address on the
socket.

host-iface=
This is an optional field that specifies the network interface used on the
host to connect to the Controller (e.g. IP eth1, enp2s0, enx78e7d1ea46da).
This forces the connection to be made on a specific interface instead of
letting the system decide.

dhchap-ctrl-secret=
This is an optional field that specifies the NVMe In-band authentication
controller secret (i.e. key) for bi-directional authentication; needs to be in
ASCII format as specified in NVMe 2.0 section 8.13.5.8 'Secret
representation'. Bi-directional authentication will be attempted when present.

hdr-digest=
See definition in [Global] section. This is an optional field used to override
the value specified in the [Global] section.

data-digest=
See definition in [Global] section. This is an optional field used to override
the value specified in the [Global] section.

nr-io-queues=
See definition in [Global] section. This is an optional field used to override
the value specified in the [Global] section.

nr-write-queues=
See definition in [Global] section. This is an optional field used to override
the value specified in the [Global] section.

nr-poll-queues=
See definition in [Global] section. This is an optional field used to override
the value specified in the [Global] section.

queue-size=
See definition in [Global] section. This is an optional field used to override
the value specified in the [Global] section.

kato=
See definition in [Global] section. This is an optional field used to override
the value specified in the [Global] section.

reconnect-delay=
See definition in [Global] section. This is an optional field used to override
the value specified in the [Global] section.

ctrl-loss-tmo=
See definition in [Global] section. This is an optional field used to override
the value specified in the [Global] section.

disable-sqflow=
See definition in [Global] section. This is an optional field used to override
the value specified in the [Global] section.
Examples:

controller = transport=tcp;traddr=localhost;trsvcid=8009
controller = transport=tcp;traddr=2001:db8::370:7334;host-iface=enp0s8
controller = transport=fc;traddr=nn-0x204600a098cbcac6:pn-0x204700a098cbcac6

exclude=
Controllers that should be excluded can be specified with the exclude= option. Using
mDNS to automatically discover and connect to controllers, can result in unintentional
connections being made. This keyword allows configuring the controllers that should
not be connected to.

The syntax is the same as for "controller", except that only transport, traddr,
trsvcid, nqn, and host-iface apply. Multiple exclude= keywords may appear in the
config file to specify more than 1 excluded controller.

Note 1: A minimal match approach is used to eliminate unwanted controllers. That is,
you do not need to specify all the parameters to identify a controller. Just
specifying the host-iface, for example, can be used to exclude all controllers on an
interface.

Note 2: exclude= takes precedence over controller. A controller specified by the
controller keyword, can be eliminated by the exclude= keyword.

Examples:

exclude = transport=tcp;traddr=fe80::2c6e:dee7:857:26bb # Eliminate a specific address
exclude = host-iface=enp0s8 # Eliminate everything on this interface

NOTES

        1. IANA Service names port numbers
           https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml?search=nvme

NAME

SYNOPSIS

DESCRIPTION

CONFIGURATION FILE FORMAT

OPTIONS

SEE ALSO

NOTES