Ubuntu Manpage: fi_cm - Connection management operations

Provided by: libfabric-dev_1.17.0-3ubuntu1_amd64

NAME

       fi_cm - Connection management operations

       fi_connect / fi_listen / fi_accept / fi_reject / fi_shutdown
              Manage endpoint connection state.

       fi_setname / fi_getname / fi_getpeer
              Set local, or return local or peer endpoint address.

       fi_join / fi_close / fi_mc_addr
              Join, leave, or retrieve a multicast address.

SYNOPSIS

              #include <rdma/fi_cm.h>

              int fi_connect(struct fid_ep *ep, const void *addr,
                  const void *param, size_t paramlen);

              int fi_listen(struct fid_pep *pep);

              int fi_accept(struct fid_ep *ep, const void *param, size_t paramlen);

              int fi_reject(struct fid_pep *pep, fid_t handle,
                  const void *param, size_t paramlen);

              int fi_shutdown(struct fid_ep *ep, uint64_t flags);

              int fi_setname(fid_t fid, void *addr, size_t addrlen);

              int fi_getname(fid_t fid, void *addr, size_t *addrlen);

              int fi_getpeer(struct fid_ep *ep, void *addr, size_t *addrlen);

              int fi_join(struct fid_ep *ep, const void *addr, uint64_t flags,
                  struct fid_mc **mc, void *context);

              int fi_close(struct fid *mc);

              fi_addr_t fi_mc_addr(struct fid_mc *mc);

ARGUMENTS

       ep / pep
              Fabric endpoint on which to change connection state.

       fid Active or passive endpoint to get/set address.

       addr   Buffer  to  address.   On  a  set call, the endpoint will be assigned the specified
              address.  On a get, the local address will be copied into the  buffer,  up  to  the
              space  provided.  For connect, this parameter indicates the peer address to connect
              to.  The address must be in the  same  format  as  that  specified  using  fi_info:
              addr_format when the endpoint was created.

       addrlen
              On input, specifies size of addr buffer.  On output, stores number of bytes written
              to addr buffer.

       param  User-specified data exchanged as part of the connection exchange.

       paramlen
              Size of param buffer.

       info   Fabric information associated with a connection request.

       mc     Multicast group associated with an endpoint.

       flags  Additional flags for controlling connection operation.

       context
              User context associated with the request.

DESCRIPTION

Connection management functions are used to connect an connection-oriented endpoint to a
peer endpoint.

fi_listen
The fi_listen call indicates that the specified endpoint should be transitioned into a
passive connection state, allowing it to accept incoming connection requests. Connection
requests against a listening endpoint are reported asynchronously to the user through a
bound CM event queue using the FI_CONNREQ event type. The number of outstanding
connection requests that can be queued at an endpoint is limited by the listening
endpoint’s backlog parameter. The backlog is initialized based on administrative
configuration values, but may be adjusted through the fi_control call.

fi_connect
The fi_connect call initiates a connection request on a connection-oriented endpoint to
the destination address. fi_connect may only be called on an endpoint once in its
lifetime.

fi_accept / fi_reject
The fi_accept and fi_reject calls are used on the passive (listening) side of a connection
to accept or reject a connection request, respectively. To accept a connection, the
listening application first waits for a connection request event (FI_CONNREQ). After
receiving such an event, the application allocates a new endpoint to accept the
connection. This endpoint must be allocated using an fi_info structure referencing the
handle from this FI_CONNREQ event. fi_accept is then invoked with the newly allocated
endpoint. If the listening application wishes to reject a connection request, it calls
fi_reject with the listening endpoint and a reference to the connection request.

A successfully accepted connection request will result in the active (connecting) endpoint
seeing an FI_CONNECTED event on its associated event queue. A rejected or failed
connection request will generate an error event. The error entry will provide additional
details describing the reason for the failed attempt.

An FI_CONNECTED event will also be generated on the passive side for the accepting
endpoint once the connection has been properly established. The fid of the FI_CONNECTED
event will be that of the endpoint passed to fi_accept as opposed to the listening passive
endpoint. Outbound data transfers cannot be initiated on a connection-oriented endpoint
until an FI_CONNECTED event has been generated. However, receive buffers may be
associated with an endpoint anytime.

fi_shutdown
The fi_shutdown call is used to gracefully disconnect an endpoint from its peer. The
flags parameter is reserved and must be 0.

Outstanding operations posted to the endpoint when fi_shutdown is called will be canceled
or discarded. Notification of canceled operations will be reported by the provider to the
corresponding completion queue(s). Discarded operations will silently be dropped, with no
completions generated. The choice of canceling, versus discarding operations, is provider
dependent. However, all canceled completions will be written before fi_shutdown returns.

When called, fi_shutdown does not affect completions already written to a completion
queue. Any queued completions associated with asynchronous operations posted to the
endpoint may still be retrieved from the corresponding completion queue(s) after an
endpoint has been shutdown.

An FI_SHUTDOWN event will be generated for an endpoint when the remote peer issues a
disconnect using fi_shutdown or abruptly closes the endpoint. Note that in the abrupt
close case, an FI_SHUTDOWN event will only be generated if the peer system is reachable
and a service or kernel agent on the peer system is able to notify the local endpoint that
the connection has been aborted.

fi_close
Fi_close is used to disassociate an endpoint from a multicast group and close all
resources associated with the group. Fi_close must be called on all multicast groups that
an endpoint joins.

fi_setname
The fi_setname call may be used to modify or assign the address of the local endpoint. It
is conceptually similar to the socket bind operation. An endpoint may be assigned an
address on its creation, through the fi_info structure. The fi_setname call allows an
endpoint to be created without being associated with a specific service (e.g., port
number) and/or node (e.g., network) address, with the addressing assigned dynamically.
The format of the specified addressing data must match that specified through the fi_info
structure when the endpoint was created.

If no service address is specified and a service address has not yet been assigned to the
endpoint, then the provider will allocate a service address and assign it to the endpoint.
If a node or service address is specified, then, upon successful completion of fi_setname,
the endpoint will be assigned the given addressing. If an address cannot be assigned, or
the endpoint address cannot be modified, an appropriate fabric error number is returned.

fi_getname / fi_getpeer
The fi_getname and fi_getpeer calls may be used to retrieve the local or peer endpoint
address, respectively. On input, the addrlen parameter should indicate the size of the
addr buffer. If the actual address is larger than what can fit into the buffer, it will
be truncated and -FI_ETOOSMALL will be returned. On output, addrlen is set to the size of
the buffer needed to store the address, which may be larger than the input value.

fi_getname is not guaranteed to return a valid source address until after the specified
endpoint has been enabled or has had an address assigned. An endpoint may be enabled
explicitly through fi_enable, or implicitly, such as through fi_connect or fi_listen. An
address may be assigned using fi_setname. fi_getpeer is not guaranteed to return a valid
peer address until an endpoint has been completely connected – an FI_CONNECTED event has
been generated.

fi_join
This call attaches an endpoint to a multicast group. By default, the endpoint will join
the group based on the data transfer capabilities of the endpoint. For example, if the
endpoint has been configured to both send and receive data, then the endpoint will be able
to initiate and receive transfers to and from the multicast group. The fi_join flags may
be used to restrict access to the multicast group, subject to endpoint capability
limitations.

Multicast join operations complete asynchronously. An endpoint must be bound to an event
queue prior to calling fi_join. The result of the join operation will be reported to the
EQ as an FI_JOIN_COMPLETE event. Applications cannot issue multicast transfers until
receiving notification that the join operation has completed. Note that an endpoint may
begin receiving messages from the multicast group as soon as the join completes, which can
occur prior to the FI_JOIN_COMPLETE event being generated.

Applications must call fi_close on the multicast group to disconnect the endpoint from the
group. After a join operation has completed, the fi_mc_addr call may be used to retrieve
the address associated with the multicast group.

fi_mc_addr
Returns the fi_addr_t address associated with a multicast group. This address must be
used when transmitting data to a multicast group and paired with the FI_MULTICAST
operation flag.

FLAGS

       Except in functions noted below, flags are reserved and must be 0.

       FI_SEND
              Applies  to  fi_join.   This  flag  indicates  that  the  endpoint  should join the
              multicast group as a send  only  member.   The  endpoint  must  be  configured  for
              transmit operations to use this flag, or an error will occur.

       FI_RECV
              Applies  to  fi_join.   This  flag  indicates  that  the  endpoint  should join the
              multicast group with receive permissions only.  The endpoint must be configured for
              receive operations to use this flag, or an error will occur.

RETURN VALUE

       Returns  0  on  success.   On  error,  a  negative  value corresponding to fabric errno is
       returned.  Fabric errno values are defined in rdma/fi_errno.h.

ERRORS

NOTES

       For connection-oriented endpoints, the buffer referenced by param will be sent as part  of
       the  connection  request  or  response,  subject  to  the  constraints  of  the underlying
       connection protocol.  Applications may use fi_getopt with the FI_OPT_CM_DATA_SIZE endpoint
       option  to  determine  the  size  of  application  data that may be exchanged as part of a
       connection request or response.  The  fi_connect,  fi_accept,  and  fi_reject  calls  will
       silently truncate any application data which cannot fit into underlying protocol messages.
       User data exchanged as part of  the  connection  process  is  available  as  part  of  the
       fi_eq_cm_entry  structure,  for  FI_CONNREQ  and  FI_CONNECTED  events,  or  as additional
       err_data to fi_eq_err_entry, in the case of a rejected connection.

AUTHORS

       OpenFabrics.