Provided by: nfstest_3.2-2_all 
NAME
nfstest_rdma - NFS-over-RDMA functional tests
SYNOPSIS
nfstest_rdma --server <server> [options]
DESCRIPTION
Verify correct functionality of NFS-over-RDMA
Remote Direct Memory Access (RDMA) provides fast data transfers between servers and
storage. NFS over RDMA is best used when a large amount of data needs to be transferred
with higher performance than regular NFS. NFS over RDMA is usually used over InfiniBand
which provides higher performance and a lower latency.
Although NFS over RDMA is mostly used over InfiniBand, Ethernet could be used as the link
protocol as well. RDMA over Converged Ethernet (RoCE) which allows RDMA over Ethernet by
encapsulating the InfiniBand transport packet over Ethernet. RoCE provides a couple of
variants: RoCEv1 and RoCEv2. One is RoCEv1 which is an Ethernet link layer protocol and
provides RDMA functionality between two hosts in the same Ethernet broadcast domain.
While the second is RoCEv2 or RRoCE (Remote RoCE) which is an internet layer protocol so
these packets can be routed. RoCEv2 runs over UDP/IPv4 or UDP/IPv6. There is also another
variant called iWARP which runs over the TCP protocol. Testing is currently supported for
all of these variants except for iWARP.
NFS over RDMA has a couple of extra layers in the packet: InfiniBand layer and RPC-over-
RDMA or RPCoRDMA layer. The InfiniBand layer contains the OpCode which specifies the type
of RDMA operation to perform and the PSN which is the packet sequence number. The RPCoRDMA
layer contains the XID and the RDMA chunk lists. The RDMA read chunk list is used to
transfer DDP (Direct Data Placement) data from the NFS client to the server, e.g., NFS
write call. On the other hand, the RDMA write chunk list is used to transfer DDP data from
the NFS server back to the client, e.g., NFS read reply. Only certain NFS operations can
be transferred using DDP and only the opaque part of the operation is transferred using
either RDMA reads or writes while the rest of the NFS packet is transferred via the
receive buffer using the RDMA SEND operation. Finally, the RDMA reply chunk is used to
transfer replies having a variable length reply which could be larger than the receive
buffer and could not be transferred using the write chunk list because it does not contain
a single large opaque item.
Tests are divided into three groups: basic, read and write. The basic tests deal mostly
with verifying NFS packets using the reply chunk and some other basic RDMA functionality.
The read tests deal with verifying NFS read which in turn verify the RDMA write
functionality. Finally, the write tests deal with verifying NFS write which in turn verify
the RDMA read functionality. Also, if the NFS read or write is small enough the client
could not use the RDMA write or read functionality, but instead could use the receive
buffer and transfer the data using the RDMA SEND operations.
Tests verify the RPCoRDMA layer is sent when necessary and that the RDMA chunk lists are
sent with the correct information which includes the number of chunks, number of segments
in each chunk and the correct information for each segment. Tests verify each segment
information is correct and their corresponding RDMA read or write information which
includes correct handle, virtual offset, DMA length and the XDR position for the case of
RDMA reads. In addition, the correct number of RDMA I/O fragments is also verified and
their corresponding lengths and packet sequence numbers.
OPTIONS
--version
show program's version number and exit
-h, --help
show this help message and exit
-f FILE, --file=FILE
File where options are specified besides the system wide file /etc/nfstest, user
wide file $HOME/.nfstest or in the current directory .nfstest file
NFS specific options:
-s SERVER, --server=SERVER
Server name or IP address
-e EXPORT, --export=EXPORT
Exported file system to mount [default: '/']
--nfsversion=NFSVERSION
NFS version, e.g., 3, 4, 4.1, etc. [default: 4.1]
-m MTPOINT, --mtpoint=MTPOINT
Mount point [default: '/mnt/t']
-p PORT, --port=PORT
NFS server port [default: 20049]
--proto=PROTO
NFS protocol name [default: 'rdma']
--sec=SEC
Security flavor [default: 'sys']
--nconnect=NCONNECT
Multiple TCP connections option [default: '1']
-o MTOPTS, --mtopts=MTOPTS
Mount options [default: 'hard']
--datadir=DATADIR
Data directory where files are created, directory is created on the mount point
[default: '']
Logging options:
-v VERBOSE, --verbose=VERBOSE
Verbose level for debug messages [default: 'opts|info|dbg1|dbg2|dbg3']
--tverbose=TVERBOSE
Verbose level for test messages [default: '1']
--createlog
Create log file
--rexeclog
Create rexec log files
--warnings
Display warnings
--tag=TAG
Informational tag, it is displayed as an INFO message [default: '']
--notty
Do not use terminal colors on output
--isatty
Use terminal colors on output -- useful when running with nohup
Packet trace options:
--createtraces
Create a packet trace for each test
--tbsize=TBSIZE
Capture buffer size for tcpdump [default: 192k]
--trcdelay=TRCDELAY
Seconds to delay before stopping packet trace [default: 2.0]
--keeptraces
Do not remove any trace files [default: remove trace files if no errors]
--rmtraces
Remove trace files [default: remove trace files if no errors]
-i INTERFACE, --interface=INTERFACE
Device interface [default: automatically selected]
File options:
--nfiles=NFILES
Number of files to create [default: 0]
--filesize=FILESIZE
File size to use for test files [default: 64k]
--rsize=RSIZE
Read size to use when reading files [default: 4k]
--wsize=WSIZE
Write size to use when writing files [default: 4k]
--iodelay=IODELAY
Seconds to delay I/O operations [default: 0.1]
--offset-delta=OFFSET_DELTA
Read/Write offset delta [default: 4k]
Path options:
--sudo=SUDO
Full path of binary for sudo [default: '/usr/bin/sudo']
--kill=KILL
Full path of binary for kill [default: '/usr/bin/kill']
--nfsstat=NFSSTAT
Full path of binary for nfsstat [default: '/usr/sbin/nfsstat']
--tcpdump=TCPDUMP
Full path of binary for tcpdump [default: '/usr/sbin/tcpdump']
--iptables=IPTABLES
Full path of binary for iptables [default: '/usr/sbin/iptables']
--messages=MESSAGES
Full path of log messages file [default: '/var/log/messages']
--trcevents=TRCEVENTS
Full path of tracing events directory [default: '/sys/kernel/debug/tracing/events']
--trcpipe=TRCPIPE
Full path of trace pipe file [default: '/sys/kernel/debug/tracing/trace_pipe']
--tmpdir=TMPDIR
Temporary directory [default: '/tmp']
Debug options:
--nocleanup
Do not cleanup created files
--notimestamps
Do not display timestamps in debug messages
--bugmsgs=BUGMSGS
File containing test messages to mark as bugs if they failed
--nomount
Do not mount server and run the tests on local disk space
--basename=BASENAME
Base name for all files and logs [default: automatically generated]
--nfsdebug=NFSDEBUG
Set NFS kernel debug flags and save log messages [default: '']
--rpcdebug=RPCDEBUG
Set RPC kernel debug flags and save log messages [default: '']
--tracepoints=TRACEPOINTS
List of trace points modules to enable [default: '']
--nfsstats
Get NFS stats [default: 'False']
--pktdisp
Display main packets related to the given test
--nfserrors
Fail every NFS error found in the packet trace
--client-ipaddr=CLIENT_IPADDR
IP address of localhost
Reporting options:
--xunit-report
Generate xUnit compatible test report
--xunit-report-file=XUNIT_REPORT_FILE
Path to xout report file
Test options:
--runtest=RUNTEST
Comma separated list of tests to run, if list starts with a '^' then all tests are
run except the ones listed [default: 'all']
--small-filesize=SMALL_FILESIZE
File size to use for small files [default: 4k]
--large-filesize=LARGE_FILESIZE
File size to use for large files [default: 1m]
--strict
Mark warnings for missing fragments as failures [default: False]
TESTS
basic
Run all NFS-over-RDMA basic functionality tests: basic01, basic02,
basic03, basic04, basic05
basic01
Verify basic NFS-over-RDMA functionality
basic02
Verify NFS-over-RDMA reply chunk on EXCHANGE_ID/SETCLIENTID
basic03
Verify NFS-over-RDMA reply chunk on READDIR
basic04
Verify NFS-over-RDMA reply chunk on READLINK
basic05
Verify NFS-over-RDMA reply chunk on GETATTR(FATTR4_ACL)
read
Run all NFS-over-RDMA functionality tests where file is opened for
reading: read01, read02, read03, read04
read01
Verify NFS-over-RDMA functionality on a file opened for reading (very small file)
read02
Verify NFS-over-RDMA functionality on a file opened for reading (small file)
read03
Verify NFS-over-RDMA functionality on a file opened for reading (medium file)
read04
Verify NFS-over-RDMA functionality on a file opened for reading (large file)
write
Run all NFS-over-RDMA functionality tests where file is opened for
writing: write01, write02, write03, write04
write01
Verify NFS-over-RDMA functionality on a file opened for writing (very small file)
write02
Verify NFS-over-RDMA functionality on a file opened for writing (small file)
write03
Verify NFS-over-RDMA functionality on a file opened for writing (medium file)
write04
Verify NFS-over-RDMA functionality on a file opened for writing (large file)
all
Run all tests: basic01, basic02, basic03, basic04, basic05, read01,
read02, read03, read04, write01, write02, write03, write04
EXAMPLES
The only required option is --server
$ nfstest_rdma --server 192.168.0.11
NOTES
The user id in the local host must have access to run commands as root using the 'sudo'
command without the need for a password.
SEE ALSO
formatstr(3), nfstest.test_util(3), nfstest_alloc(1), nfstest_cache(1),
nfstest_delegation(1), nfstest_dio(1), nfstest_fcmp(1), nfstest_file(1),
nfstest_interop(1), nfstest_io(1), nfstest_lock(1), nfstest_pkt(1), nfstest_pnfs(1),
nfstest_posix(1), nfstest_sparse(1), nfstest_ssc(1), nfstest_xattr(1), nfstest_xid(1),
packet.application.rpc_const(3), packet.application.rpcordma_const(3),
packet.nfs.nfs3_const(3), packet.nfs.nfs4_const(3), packet.transport.ib(3),
packet.transport.rdmap(3)
BUGS
No known bugs.
AUTHOR
Jorge Mora (mora@netapp.com)