Ubuntu Manpages

ibv_reg_mr, ibv_reg_mr_iova, ibv_reg_dmabuf_mr, ibv_reg_mr_ex,

#include <infiniband/verbs.h>

struct ibv_mr *ibv_reg_mr(struct ibv_pd *pd, void *addr,
                          size_t length, int access);

struct ibv_mr *ibv_reg_mr_iova(struct ibv_pd *pd, void *addr,
                               size_t length, uint64_t hca_va,
                               int access);

struct ibv_mr *ibv_reg_dmabuf_mr(struct ibv_pd *pd, uint64_t offset,
                                 size_t length, uint64_t iova,
                                 int fd, int access);

struct ibv_mr *ibv_reg_mr_ex(struct ibv_pd *pd, struct ibv_mr_init_attr *mr_init_attr");

int ibv_dereg_mr(struct ibv_mr *mr);

ibv_reg_mr() registers a memory region (MR) associated with the protection domain pd. The MR's starting address is addr and its size is length. The argument access describes the desired memory protection attributes; it is either 0 or the bitwise OR of one or more of the following flags:

Enable Local Write Access
Enable Remote Write Access
Enable Remote Read Access
Enable Remote Atomic Operation Access (if supported)
Enable Remote Flush Operation with global visibility placement type (if supported)
Enable Remote Flush Operation with persistence placement type (if supported)
Enable Memory Window Binding
Use byte offset from beginning of MR to access this MR, instead of a pointer address
Create an on-demand paging MR
Huge pages are guaranteed to be used for this MR, applicable with IBV_ACCESS_ON_DEMAND in explicit mode only
This setting allows the NIC to relax the order that data is transferred between the network and the target memory region. Relaxed ordering allows network initiated writes (such as incoming message send or RDMA write operations) to reach memory in an arbitrary order. This can improve the performance of some applications. However, relaxed ordering has the following impact: RDMA write-after-write message order is no longer guaranteed. (Send messages will still match posted receive buffers in order.) Back-to-back network writes that target the same memory region leave the region in an unknown state. Relaxed ordering does not change completion semantics, such as data visibility. That is, a completion still ensures that all data is visible, including data from prior transfers. Relaxed ordered operations will also not bypass atomic operations.

If IBV_ACCESS_REMOTE_WRITE or IBV_ACCESS_REMOTE_ATOMIC is set, then IBV_ACCESS_LOCAL_WRITE must be set too.

Local read access is always enabled for the MR.

To create an implicit ODP MR, IBV_ACCESS_ON_DEMAND should be set, addr should be 0 and length should be SIZE_MAX.

If IBV_ACCESS_HUGETLB is set, then application awares that for this MR all pages are huge and must promise it will never do anything to break huge pages.

ibv_reg_mr_iova() ibv_reg_mr_iova is the same as the normal reg_mr, except that the user is allowed to specify the virtual base address of the MR when accessed through a lkey or rkey. The offset in the memory region is computed as 'addr + (iova - hca_va)'. Specifying 0 for hca_va has the same effect as IBV_ACCESS_ZERO_BASED.

ibv_reg_dmabuf_mr() registers a dma-buf based memory region (MR) associated with the protection domain pd. The MR starts at offset of the dma-buf and its size is length. The dma-buf is identified by the file descriptor fd. The argument iova specifies the virtual base address of the MR when accessed through a lkey or rkey. It must have the same page offset as offset. The argument access describes the desired memory protection attributes; it is similar to the ibv_reg_mr case except that only the following flags are supported: IBV_ACCESS_LOCAL_WRITE, IBV_ACCESS_REMOTE_WRITE, IBV_ACCESS_REMOTE_READ, IBV_ACCESS_REMOTE_ATOMIC, IBV_ACCESS_RELAXED_ORDERING.

ibv_reg_mr_ex() ibv_reg_mr_ex is a an API that enables all the variants of the other ibv_reg_mr_xxx() that desecibed in that man page. It has the mr_init_attr->comp_mask field to let application mark which fields are applicable. In addition, it includes the mr_init_attr->dmah which can be used to include an ibv_dmah object that will be used for that MR. The other fields on the input pointer have the same meaning as of the fields that described in that man page for the other verbs.

ibv_dereg_mr() deregisters the MR mr.

ibv_reg_mr() / ibv_reg_mr_iova() / ibv_reg_dmabuf_mr() / ibv_reg_mr_ex() returns a pointer to the registered MR, or NULL if the request fails. The local key (L_Key) field lkey is used as the lkey field of struct ibv_sge when posting buffers with ibv_post_* verbs, and the the remote key (R_Key) field rkey is used by remote processes to perform Atomic and RDMA operations. The remote process places this rkey as the rkey field of struct ibv_send_wr passed to the ibv_post_send function.

ibv_dereg_mr() returns 0 on success, or the value of errno on failure (which indicates the failure reason).

ibv_dereg_mr() fails if any memory window is still bound to this MR. ibv_dereg_mr_ex() One among mr_init_attr->fd and mr_init_attr->addr is required, both can't come together.

ibv_alloc_pd(3), ibv_post_send(3), ibv_post_recv(3), ibv_post_srq_recv(3)