Provided by: libfabric-dev_1.5.3-1_amd64 
NAME
fi_tagged - Tagged data transfer operations
fi_trecv / fi_trecvv / fi_trecvmsg
Post a buffer to receive an incoming message
fi_tsend / fi_tsendv / fi_tsendmsg / fi_tinject / fi_tsenddata
Initiate an operation to send a message
SYNOPSIS
#include <rdma/fi_tagged.h>
ssize_t fi_trecv(struct fid_ep *ep, void *buf, size_t len, void *desc,
fi_addr_t src_addr, uint64_t tag, uint64_t ignore, void *context);
ssize_t fi_trecvv(struct fid_ep *ep, const struct iovec *iov, void *desc,
size_t count, fi_addr_t src_addr, uint64_t tag, uint64_t ignore,
void *context);
ssize_t fi_trecvmsg(struct fid_ep *ep, const struct fi_msg_tagged *msg,
uint64_t flags);
ssize_t fi_tsend(struct fid_ep *ep, const void *buf, size_t len,
void *desc, fi_addr_t dest_addr, uint64_t tag, void *context);
ssize_t fi_tsendv(struct fid_ep *ep, const struct iovec *iov,
void *desc, size_t count, fi_addr_t dest_addr, uint64_t tag,
void *context);
ssize_t fi_tsendmsg(struct fid_ep *ep, const struct fi_msg_tagged *msg,
uint64_t flags);
ssize_t fi_tinject(struct fid_ep *ep, const void *buf, size_t len,
fi_addr_t dest_addr, uint64_t tag);
ssize_t fi_tsenddata(struct fid_ep *ep, const void *buf, size_t len,
void *desc, uint64_t data, fi_addr_t dest_addr, uint64_t tag,
void *context);
ssize_t fi_tinjectdata(struct fid_ep *ep, const void *buf, size_t len,
uint64_t data, fi_addr_t dest_addr, uint64_t tag);
ARGUMENTS
fid : Fabric endpoint on which to initiate tagged communication operation.
buf : Data buffer to send or receive.
len : Length of data buffer to send or receive.
iov : Vectored data buffer.
count : Count of vectored data entries.
tag : Tag associated with the message.
ignore : Mask of bits to ignore applied to the tag for receive operations.
desc : Memory descriptor associated with the data buffer
data : Remote CQ data to transfer with the sent data.
dest_addr : Destination address for connectionless transfers. Ignored for connected
endpoints.
src_addr : Source address to receive from for connectionless transfers. Applies only to
connectionless endpoints with the FI_DIRECTED_RECV capability enabled, otherwise this
field is ignored. If set to FI_ADDR_UNSPEC, any source address may match.
msg : Message descriptor for send and receive operations.
flags : Additional flags to apply for the send or receive operation.
context : User specified pointer to associate with the operation.
DESCRIPTION
Tagged messages are data transfers which carry a key or tag with the message buffer. The
tag is used at the receiving endpoint to match the incoming message with a corresponding
receive buffer. Message tags match when the receive buffer tag is the same as the send
buffer tag with the ignored bits masked out. This can be stated as:
send_tag & ~ignore == recv_tag & ~ignore
In general, message tags are checked against receive buffers in the order in which
messages have been posted to the endpoint. See the ordering discussion below for more
details.
The send functions -- fi_tsend, fi_tsendv, fi_tsendmsg, fi_tinject, and fi_tsenddata --
are used to transmit a tagged message from one endpoint to another endpoint. The main
difference between send functions are the number and type of parameters that they accept
as input. Otherwise, they perform the same general function.
The receive functions -- fi_trecv, fi_trecvv, fi_recvmsg -- post a data buffer to an
endpoint to receive inbound tagged messages. Similar to the send operations, receive
operations operate asynchronously. Users should not touch the posted data buffer(s) until
the receive operation has completed. Posted receive buffers are matched with inbound send
messages based on the tags associated with the send and receive buffers.
An endpoint must be enabled before an application can post send or receive operations to
it. For connected endpoints, receive buffers may be posted prior to connect or accept
being called on the endpoint. This ensures that buffers are available to receive incoming
data immediately after the connection has been established.
Completed message operations are reported to the user through one or more event collectors
associated with the endpoint. Users provide context which are associated with each
operation, and is returned to the user as part of the event completion. See fi_cq for
completion event details.
fi_tsend
The call fi_tsend transfers the data contained in the user-specified data buffer to a
remote endpoint, with message boundaries being maintained. The local endpoint must be
connected to a remote endpoint or destination before fi_tsend is called. Unless the
endpoint has been configured differently, the data buffer passed into fi_tsend must not be
touched by the application until the fi_tsend call completes asynchronously.
fi_tsendv
The fi_tsendv call adds support for a scatter-gather list to fi_tsend. The fi_sendv
transfers the set of data buffers referenced by the iov parameter to a remote endpoint as
a single message.
fi_tsendmsg
The fi_tsendmsg call supports data transfers over both connected and unconnected
endpoints, with the ability to control the send operation per call through the use of
flags. The fi_tsendmsg function takes a struct fi_msg_tagged as input.
struct fi_msg_tagged {
const struct iovec *msg_iov; /* scatter-gather array */
void *desc; /* data descriptor */
size_t iov_count;/* # elements in msg_iov */
fi_addr_t addr; /* optional endpoint address */
uint64_t tag; /* tag associated with message */
uint64_t ignore; /* mask applied to tag for receives */
void *context; /* user-defined context */
uint64_t data; /* optional immediate data */
};
fi_tinject
The tagged inject call is an optimized version of fi_tsend. The fi_tinject function
behaves as if the FI_INJECT transfer flag were set, and FI_COMPLETION were not. That is,
the data buffer is available for reuse immediately on returning from from fi_tinject, and
no completion event will be generated for this send. The completion event will be
suppressed even if the endpoint has not been configured with FI_SELECTIVE_COMPLETION. See
the flags discussion below for more details. The requested message size that can be used
with fi_tinject is limited by inject_size.
fi_tsenddata
The tagged send data call is similar to fi_tsend, but allows for the sending of remote CQ
data (see FI_REMOTE_CQ_DATA flag) as part of the transfer.
fi_tinjectdata
The tagged inject data call is similar to fi_tinject, but allows for the sending of remote
CQ data (see FI_REMOTE_CQ_DATA flag) as part of the transfer.
fi_trecv
The fi_trecv call posts a data buffer to the receive queue of the corresponding endpoint.
Posted receives are searched in the order in which they were posted in order to match
sends. Message boundaries are maintained. The order in which the receives complete is
dependent on the endpoint type and protocol.
fi_trecvv
The fi_trecvv call adds support for a scatter-gather list to fi_trecv. The fi_trecvv
posts the set of data buffers referenced by the iov parameter to a receive incoming data.
fi_trecvmsg
The fi_trecvmsg call supports posting buffers over both connected and unconnected
endpoints, with the ability to control the receive operation per call through the use of
flags. The fi_trecvmsg function takes a struct fi_msg_tagged as input.
FLAGS
The fi_trecvmsg and fi_tsendmsg calls allow the user to specify flags which can change the
default message handling of the endpoint. Flags specified with fi_trecvmsg / fi_tsendmsg
override most flags previously configured with the endpoint, except where noted (see
fi_endpoint). The following list of flags are usable with fi_trecvmsg and/or fi_tsendmsg.
FI_REMOTE_CQ_DATA : Applies to fi_tsendmsg and fi_tsenddata. Indicates that remote CQ
data is available and should be sent as part of the request. See fi_getinfo for
additional details on FI_REMOTE_CQ_DATA.
FI_COMPLETION : Indicates that a completion entry should be generated for the specified
operation. The endpoint must be bound to a completion queue with FI_SELECTIVE_COMPLETION
that corresponds to the specified operation, or this flag is ignored.
FI_MORE : Indicates that the user has additional requests that will immediately be posted
after the current call returns. Use of this flag may improve performance by enabling the
provider to optimize its access to the fabric hardware.
FI_INJECT : Applies to fi_tsendmsg. Indicates that the outbound data buffer should be
returned to user immediately after the send call returns, even if the operation is handled
asynchronously. This may require that the underlying provider implementation copy the
data into a local buffer and transfer out of that buffer. This flag can only be used with
messages smaller than inject_size.
FI_INJECT_COMPLETE : Applies to fi_tsendmsg. Indicates that a completion should be
generated when the source buffer(s) may be reused.
FI_TRANSMIT_COMPLETE : Applies to fi_tsendmsg. Indicates that a completion should not be
generated until the operation has been successfully transmitted and is no longer being
tracked by the provider.
FI_FENCE : Applies to transmits. Indicates that the requested operation, also known as
the fenced operation, be deferred until all previous operations targeting the same target
endpoint have completed.
The following flags may be used with fi_trecvmsg.
FI_PEEK : The peek flag may be used to see if a specified message has arrived. A peek
request is often useful on endpoints that have provider allocated buffering enabled (see
fi_rx_attr total_buffered_recv). Unlike standard receive operations, a receive operation
with the FI_PEEK flag set does not remain queued with the provider after the peek
completes successfully. The peek operation operates asynchronously, and the results of
the peek operation are available in the completion queue associated with the endpoint. If
no message is found matching the tags specified in the peek request, then a completion
queue error entry with err field set to FI_ENOMSG will be available.
If a peek request locates a matching message, the operation will complete successfully.
The returned completion data will indicate the meta-data associated with the message, such
as the message length, completion flags, available CQ data, tag, and source address. The
data available is subject to the completion entry format (e.g. struct
fi_cq_tagged_entry).
An application may supply a buffer if it desires to receive data as a part of the peek
operation. In order to receive data as a part of the peek operation, the buf and len
fields must be available in the CQ format. In particular, FI_CQ_FORMAT_CONTEXT and
FI_CQ_FORMAT_MSG cannot be used if peek operations desire to obtain a copy of the data.
The returned data is limited to the size of the input buffer(s) or the message size, if
smaller. A provider indicates if data is available by setting the buf field of the CQ
entry to the user's first input buffer. If buf is NULL, no data was available to return.
A provider may return NULL even if the peek operation completes successfully. Note that
the CQ entry len field will reference the size of the message, not necessarily the size of
the returned data.
FI_CLAIM : If this flag is used in conjunction with FI_PEEK, it indicates if the peek
request completes successfully -- indicating that a matching message was located -- the
message is claimed by caller. Claimed messages can only be retrieved using a subsequent,
paired receive operation with the FI_CLAIM flag set. A receive operation with the
FI_CLAIM flag set, but FI_PEEK not set is used to retrieve a previously claimed message.
In order to use the FI_CLAIM flag, an application must supply a struct fi_context
structure as the context for the receive operation. The same fi_context structure used
for an FI_PEEK + FI_CLAIM operation must be used by the paired FI_CLAIM request.
FI_DISCARD : This flag must be used in conjunction with either FI_PEEK or FI_CLAIM. If
this flag is used in conjunction with FI_PEEK, it indicates if the peek request completes
successfully -- indicating that a matching message was located -- the message is discarded
by the provider, as the data is not needed by the application. This flag may also be used
in conjunction with FI_CLAIM in order to retrieve and discard a message previously claimed
using an FI_PEEK + FI_CLAIM request.
If this flag is set, the input buffer(s) and length parameters are ignored.
RETURN VALUE
The tagged send and receive calls return 0 on success. On error, a negative value
corresponding to fabric errno is returned. Fabric errno values are defined in
fi_errno.h.
ERRORS
-FI_EAGAIN : See fi_msg(3) for a detailed description of handling FI_EAGAIN.
-FI_EINVAL : Indicates that an invalid argument was supplied by the user.
-FI_EOTHER : Indicates that an unspecified error occurred.
SEE ALSO
fi_getinfo(3), fi_endpoint(3), fi_domain(3), fi_cq(3)
AUTHORS
OpenFabrics.