Provided by: libfabric-dev_1.17.0-3build2_amd64 bug

NAME

       fi_rma - Remote memory access operations

       fi_read / fi_readv / fi_readmsg
              Initiates a read from remote memory

       fi_write  /  fi_writev  / fi_writemsg fi_inject_write / fi_writedata : Initiate a write to
       remote memory

SYNOPSIS

              #include <rdma/fi_rma.h>

              ssize_t fi_read(struct fid_ep *ep, void *buf, size_t len, void *desc,
                  fi_addr_t src_addr, uint64_t addr, uint64_t key, void *context);

              ssize_t fi_readv(struct fid_ep *ep, const struct iovec *iov, void **desc,
                  size_t count, fi_addr_t src_addr, uint64_t addr, uint64_t key,
                  void *context);

              ssize_t fi_readmsg(struct fid_ep *ep, const struct fi_msg_rma *msg,
                  uint64_t flags);

              ssize_t fi_write(struct fid_ep *ep, const void *buf, size_t len,
                  void *desc, fi_addr_t dest_addr, uint64_t addr, uint64_t key,
                  void *context);

              ssize_t fi_writev(struct fid_ep *ep, const struct iovec *iov, void **desc,
                  size_t count, fi_addr_t dest_addr, uint64_t addr, uint64_t key,
                  void *context);

              ssize_t fi_writemsg(struct fid_ep *ep, const struct fi_msg_rma *msg,
                  uint64_t flags);

              ssize_t fi_inject_write(struct fid_ep *ep, const void *buf, size_t len,
                  fi_addr_t dest_addr, uint64_t addr, uint64_t key);

              ssize_t fi_writedata(struct fid_ep *ep, const void *buf, size_t len,
                  void *desc, uint64_t data, fi_addr_t dest_addr, uint64_t addr,
                  uint64_t key, void *context);

              ssize_t fi_inject_writedata(struct fid_ep *ep, const void *buf, size_t len,
                  uint64_t data, fi_addr_t dest_addr, uint64_t addr, uint64_t key);

ARGUMENTS

       ep     Fabric endpoint on which to initiate read or write operation.

       buf    Local data buffer to read into (read target) or write from (write source)

       len    Length of data to read or write, specified in bytes.  Valid transfers  are  from  0
              bytes up to the endpoint’s max_msg_size.

       iov    Vectored data buffer.

       count  Count of vectored data entries.

       addr   Address of remote memory to access.  This will be the virtual address of the remote
              region in the case of FI_MR_BASIC, or the offset from the starting address  in  the
              case of FI_MR_SCALABLE.

       key    Protection key associated with the remote memory.

       desc   Descriptor associated with the local data buffer See fi_mr(3).

       data   Remote CQ data to transfer with the operation.

       dest_addr
              Destination  address  for  connectionless  write  transfers.  Ignored for connected
              endpoints.

       src_addr
              Source address to read from for connectionless transfers.   Ignored  for  connected
              endpoints.

       msg    Message descriptor for read and write operations.

       flags  Additional flags to apply for the read or write operation.

       context
              User  specified pointer to associate with the operation.  This parameter is ignored
              if the operation will not generate a  successful  completion,  unless  an  op  flag
              specifies the context parameter be used for required input.

DESCRIPTION

       RMA  (remote  memory access) operations are used to transfer data directly between a local
       data buffer and a remote data buffer.  RMA transfers occur on a  byte  level  granularity,
       and no message boundaries are maintained.

       The  write functions – fi_write, fi_writev, fi_writemsg, fi_inject_write, and fi_writedata
       – are used to transmit data into a remote memory  buffer.   The  main  difference  between
       write  functions  are  the  number  and  type  of  parameters  that  they accept as input.
       Otherwise, they perform the same general function.

       The read functions – fi_read, fi_readv, and fi_readmsg – are used to transfer data from  a
       remote  memory  region  into  local data buffer(s).  Similar to the write operations, read
       operations operate asynchronously.  Users should not touch the posted data buffer(s) until
       the read operation has completed.

       Completed  RMA  operations  are reported to the user through one or more completion queues
       associated with the endpoint.  Users  provide  context  which  are  associated  with  each
       operation,  and  is  returned  to  the  user  as  part  of  the completion.  See fi_cq for
       completion event details.

       By default, the remote endpoint does not generate an event  or  notify  the  user  when  a
       memory  region  has  been  accessed by an RMA read or write operation.  However, immediate
       data may be associated with an RMA write operation.  RMA writes with immediate  data  will
       generate  a  completion  entry  at  the remote endpoint, so that the immediate data may be
       delivered.

   fi_write
       The call fi_write transfers the data contained in the  user-specified  data  buffer  to  a
       remote memory region.

   fi_writev
       The  fi_writev  call  adds  support  for a scatter-gather list to fi_write.  The fi_writev
       transfers the set of data buffers referenced by the iov parameter  to  the  remote  memory
       region.

   fi_writemsg
       The  fi_writemsg  call  supports  data  transfers  over  both connected and connectionless
       endpoints, with the ability to control the write operation per call  through  the  use  of
       flags.  The fi_writemsg function takes a struct fi_msg_rma as input.

              struct fi_msg_rma {
                  const struct iovec *msg_iov;     /* local scatter-gather array */
                  void               **desc;       /* operation descriptor */
                  size_t             iov_count;    /* # elements in msg_iov */
                  fi_addr_t          addr;        /* optional endpoint address */
                  const struct fi_rma_iov *rma_iov;/* remote SGL */
                  size_t             rma_iov_count;/* # elements in rma_iov */
                  void               *context;     /* user-defined context */
                  uint64_t           data;         /* optional immediate data */
              };

              struct fi_rma_iov {
                  uint64_t           addr;         /* target RMA address */
                  size_t             len;          /* size of target buffer */
                  uint64_t           key;          /* access key */
              };

   fi_inject_write
       The write inject call is an optimized version of fi_write.  It provides similar completion
       semantics as fi_inject fi_msg(3).

   fi_writedata
       The write data call is similar to fi_write, but allows for the sending of remote  CQ  data
       (see FI_REMOTE_CQ_DATA flag) as part of the transfer.

   fi_inject_writedata
       The  inject  write  data call is similar to fi_inject_write, but allows for the sending of
       remote CQ data (see FI_REMOTE_CQ_DATA flag) as part of the transfer.

   fi_read
       The fi_read call requests that the remote endpoint transfer data from  the  remote  memory
       region into the local data buffer.

   fi_readv
       The  fi_readv  call  adds  support  for  a  scatter-gather  list to fi_read.  The fi_readv
       transfers data from the remote memory region into the set of data  buffers  referenced  by
       the iov parameter.

   fi_readmsg
       The  fi_readmsg  call  supports  data  transfers  over  both  connected and connectionless
       endpoints, with the ability to control the read operation per  call  through  the  use  of
       flags.  The fi_readmsg function takes a struct fi_msg_rma as input.

FLAGS

       The  fi_readmsg and fi_writemsg calls allow the user to specify flags which can change the
       default data transfer operation.  Flags specified with fi_readmsg /  fi_writemsg  override
       most   flags   previously   configured   with   the  endpoint,  except  where  noted  (see
       fi_endpoint.3).   The  following  list  of  flags  are  usable  with   fi_readmsg   and/or
       fi_writemsg.

       FI_REMOTE_CQ_DATA
              Applies  to  fi_writemsg  and  fi_writedata.   Indicates  that  remote  CQ  data is
              available and should be sent as part of the request.  See fi_getinfo for additional
              details on FI_REMOTE_CQ_DATA.

       FI_COMPLETION
              Indicates  that a completion entry should be generated for the specified operation.
              The endpoint must be bound to a completion queue with FI_SELECTIVE_COMPLETION  that
              corresponds to the specified operation, or this flag is ignored.

       FI_MORE
              Indicates  that  the  user  has additional requests that will immediately be posted
              after the current call returns.  Use  of  this  flag  may  improve  performance  by
              enabling the provider to optimize its access to the fabric hardware.

       FI_INJECT
              Applies to fi_writemsg.  Indicates that the outbound data buffer should be returned
              to user immediately after the write call returns, even if the operation is  handled
              asynchronously.   This may require that the underlying provider implementation copy
              the data into a local buffer and transfer out of that buffer.  This flag  can  only
              be used with messages smaller than inject_size.

       FI_INJECT_COMPLETE
              Applies  to  fi_writemsg.  Indicates that a completion should be generated when the
              source buffer(s) may be reused.

       FI_TRANSMIT_COMPLETE
              Applies to fi_writemsg.  Indicates that a completion should not be generated  until
              the  operation  has been successfully transmitted and is no longer being tracked by
              the provider.

       FI_DELIVERY_COMPLETE
              Applies to fi_writemsg.  Indicates that a completion should be generated  when  the
              operation has been processed by the destination.

       FI_COMMIT_COMPLETE
              Applies  to fi_writemsg when targeting persistent memory regions.  Indicates that a
              completion should be generated only after the result of the operation has been made
              durable.

       FI_FENCE
              Applies  to  transmits.   Indicates that the requested operation, also known as the
              fenced operation, and any operation posted  after  the  fenced  operation  will  be
              deferred  until  all  previous  operations  targeting  the  same peer endpoint have
              completed.  Operations posted after the fencing will see and/or replace the results
              of any operations initiated prior to the fenced operation.

       The  ordering of operations starting at the posting of the fenced operation (inclusive) to
       the posting of a subsequent fenced operation (exclusive) is controlled by  the  endpoint’s
       ordering semantics.

RETURN VALUE

       Returns  0  on  success.   On  error,  a  negative  value corresponding to fabric errno is
       returned.  Fabric errno values are defined in rdma/fi_errno.h.

ERRORS

       -FI_EAGAIN
              See fi_msg(3) for a detailed description of handling FI_EAGAIN.

SEE ALSO

       fi_getinfo(3), fi_endpoint(3), fi_domain(3), fi_cq(3)

AUTHORS

       OpenFabrics.