Ubuntu Manpage: libpmem2 - persistent memory support library

name
synopsis
description
granularity
caveats
environment
debugging
example
acknowledgements
see also

Provided by: libpmem2-dev_1.13.1-1.1ubuntu2_amd64

NAME

       libpmem2 - persistent memory support library

              NOTE:  Support for Windows and FreeBSD deprecated since PMDK 1.13.0 release and will be removed in
              the PMDK 1.14.0 release.

SYNOPSIS

              #include <libpmem2.h>
              cc ... -lpmem2

DESCRIPTION

       libpmem2 provides low-level persistent memory (pmem) support for applications using direct access storage
       (DAX),  which  is  storage  that  supports  load/store  access without paging blocks from a block storage
       device.  Some types of non-volatile memory DIMMs (NVDIMMs) provide this type of byte  addressable  access
       to  storage.   A  persistent  memory  aware  file system is typically used to expose the direct access to
       applications.  Memory mapping a file from this type of file system results in the  load/store,  non-paged
       access to pmem.

       This  library  is  for applications that use persistent memory directly, without the help of any library-
       supplied transactions or memory allocation.  Higher-level  libraries  that  currently  build  on  libpmem
       (previous variation of libpmem2) are available and are recommended for most applications, see:

       • libpmemobj(7),  a  general  use  persistent  memory  API, providing memory allocation and transactional
         operations on variable-sized objects.

       • libpmemblk(7), providing pmem-resident arrays of fixed-sized blocks with atomic updates.

       • libpmemlog(7), providing a pmem-resident log file.

       The libpmem2 library provides a comprehensive set of functions for robust use of Persistent  Memory.   It
       relies  on  three core concepts: struct pmem2_src source, struct pmem2_config config and struct pmem2_map
       map:

       • source - an object describing the data source for mapping.  The data source can be a file descriptor, a
         file handle, or an anonymous mapping.  APIs dedicated for creating source are: pmem2_source_from_fd(3),
         pmem2_source_from_handle(3), pmem2_source_from_anon(3).

       • config - an object containing parameters that are  used  to  create  a  mapping  from  a  source.   The
         configuration structure must always be provided to create a mapping, but the only required parameter to
         set in the config is granularity.  The granularity should by  set  using  dedicated  libpmem2  function
         pmem2_config_set_required_store_granularity(3)  which defines a maximum permitted granularity requested
         by the user.  For more information about the granularity concept read GRANULARITY section below.

       In addition to the granularity setting, libpmem2 provides multiple optional functions to configure target
       mapping,   e.g.,   pmem2_config_set_length(3)   to  set  length  which  will  be  used  for  mapping,  or
       pmem2_config_set_offset(3) which will be used to map the contents from  the  specified  location  of  the
       source,  pmem2_config_set_sharing(3) which defines the behavior and visibility of writes to the mapping’s
       pages.

       • map - an object created by pmem2_map_new(3) using source and config as an input  parameters.   The  map
         structure can be then used to directly operate on the created mapping through the use of its associated
         set of functions: pmem2_map_get_address(3), pmem2_map_get_size(3), pmem2_map_get_store_granularity(3) -
         for getting address, size and effective mapping granularity.

       In  addition  to  the basic functionality of managing the virtual address mapping, libpmem2 also provides
       optimized functions for modifying the mapped data.   This  includes  data  flushing  as  well  as  memory
       copying.

       To  get  proper  function  for  data  flushing  use:  pmem2_get_flush_fn(3),  pmem2_get_persist_fn(3)  or
       pmem2_get_drain_fn(3).  To get proper function  for  copying  to  persistent  memory,  use  map  getters:
       pmem2_get_memcpy_fn(3), pmem2_get_memset_fn(3), pmem2_get_memmove_fn(3).

       The libpmem2 API also provides support for the badblock and unsafe shutdown state handling.

       To  read  or  clear  badblocks,  the  following  functions  are  provided: pmem2_badblock_context_new(3),
       pmem2_badblock_context_delete(3), pmem2_badblock_next(3) and pmem2_badblock_clear(3).

       To   handle   unsafe   shutdown   in   the   application,   the   following   functions   are   provided:
       pmem2_source_device_id(3),  pmem2_source_device_usc(3).   More detailed information about unsafe shutdown
       detection and unsafe shutdown count can be found in the libpmem2_unsafe_shutdown(7) man page.

GRANULARITY

The libpmem2 library introduces the concept of granularity through which you may easily distinguish
between different levels of storage performance capabilities available to the application as related to
power-fail protected domain. The way data reaches this protected domain differs based on the platform
and storage device capabilities.

Traditional block storage devices (SSD, HDD) must use system API calls such as msync(), fsync() on Linux,
or FlushFileBuffers(),FlushViewOfFile() on Windows to write data reliably. Invoking these functions
flushes the data to the medium with page granularity. In the libpmem2 library, this type of flushing
behavior is called PMEM2_GRANULARITY_PAGE.

In systems with persistent memory support, a power-fail protected domain may cover different sets of
resources: either the memory controller or the memory controller and CPU caches. For this reason,
libpmem2 distinguishes two types of granularity for persistent memory: PMEM2_GRANULARITY_CACHE_LINE and
PMEM2_GRANULARITY_BYTE.

If the power-fail protected domain covers only the memory controller, the CPU appropriate cache lines
must be flushed for the data to be considered persistent. This granularity type is called
PMEM2_GRANULARITY_CACHE_LINE. Depending on the architecture, there are different types of machine
instructions for flushing cache lines (e.g., CLWB, CLFLUSHOPT, CLFLUSH for Intel x86_64 architecture).
Usually, to ensure the ordering of stores, such instructions must be followed by a barrier (e.g.,
SFENCE).

The third type of granularity PMEM2_GRANULARITY_BYTE applies to platforms where power-fail protected
domain covers both the memory controller and CPU caches. In such cases, cache flush instructions are no
longer needed, and the platform itself guarantees the persistence of data. But barriers might still be
required for ordering.

The library declares these granularity level in pmem2_granularity enum, which the application must set in
pmem2_config to the appropriate level for a mapping to succeed. The software should set this config
parameter to a value that most accurately represents the target hardware characteristics and the storage
patterns of the application. For example, a database storage engine that operates on large logical pages
that reside either on SSDs or PMEM should set this value to PMEM2_GRANULARITY_PAGE. The library will
create mappings where the new map granularity is lower or equal to the requested one. For example, a
mapping with PMEM2_GRANULARITY_CACHE_LINE can be created for the required granularity
PMEM2_GRANULARITY_PAGE, but not vice versa.

CAVEATS

       libpmem2 relies on the library destructor being called from  the  main  thread.   For  this  reason,  all
       functions  that  might  trigger  destruction  (e.g. dlclose(3))  should  be  called  in  the main thread.
       Otherwise some of the resources associated with that thread might not be cleaned up properly.

ENVIRONMENT

libpmem2 can change its default behavior based on the following environment variables. These are
primarily intended for testing and are generally not required.

• PMEM2_FORCE_GRANULARITY=val

Setting this environment variable to val forces libpmem2 to use persist method specific for forced
granularity and skip granularity autodetecting mechanism. The concept of the granularity is described in
GRANULARITY section above. This variable is intended for use during library testing.

The val argument accepts following text values:

• BYTE - force byte granularity.

• CACHE_LINE - force cache line granularity.

• PAGE - force page granularity.

Granularity values listed above are case-insensitive.

NOTE: The value of PMEM2_FORCE_GRANULARITY is not queried (and cached) at library initialization
time, but read during each pmem2_map_new(3) call.

This means that PMEM2_FORCE_GRANULARITY may still be set or modified by the program until the first
attempt to map a file.

• PMEM_NO_CLWB=1

Setting this environment variable to 1 forces libpmem2 to never issue the CLWB instruction on Intel
hardware, falling back to other cache flush instructions on that hardware instead (CLFLUSHOPT or
CLFLUSH). Without this setting, libpmem2 will always use the CLWB instruction for flushing processor
caches on platforms that support this instruction. This variable is intended for use during library
testing, but may be required for some rare cases when using CLWB has a negative impact on performance.

• PMEM_NO_CLFLUSHOPT=1

Setting this environment variable to 1 forces libpmem2 to never issue the CLFLUSHOPT instruction on Intel
hardware, falling back to the CLFLUSH instructions instead. Without this environment variable, libpmem2
will always use the CLFLUSHOPT instruction for flushing processor caches on platforms that support the
instruction, but where CLWB is not available. This variable is intended for use during library testing.

• PMEM_NO_MOVNT=1

Setting this environment variable to 1 forces libpmem2 to never use the non-temporal move instructions on
Intel hardware. Without this environment variable, libpmem2 will use the non-temporal instructions for
copying larger ranges to persistent memory on platforms that support these instructions. This variable
is intended for use during library testing.

• PMEM_MOVNT_THRESHOLD=val

This environment variable allows overriding the minimum length of the pmem2_memmove_fn operations, for
which libpmem2 uses non-temporal move instructions. Setting this environment variable to 0 forces
libpmem2 to always use the non-temporal move instructions if available. It has no effect if
PMEM_NO_MOVNT is set to 1. This variable is intended for use during library testing.

DEBUGGING

Two versions of libpmem2 are typically available on a development system. The normal version, accessed
when a program is linked using the -lpmem2 option, is optimized for performance. That version skips
checks that impact performance and never logs any trace information or performs any run-time assertions.

A second version of libpmem2, accessed when a program uses the libraries under /usr/lib/pmdk_debug,
contains run-time assertions and trace points. The typical way to access the debug version is to set the
environment variable LD_LIBRARY_PATH to /usr/lib/pmdk_debug or /usr/lib64/pmdk_debug, as appropriate.
Debugging output is controlled using the following environment variables. These variables have no effect
on the non-debug version of the library.

NOTE: On Debian/Ubuntu systems, this extra debug version of the library is shipped in the
respective -debug Debian package and placed in the /usr/lib/$ARCH/pmdk_dbg/ directory.

• PMEM2_LOG_LEVEL

The value of PMEM2_LOG_LEVEL enables trace points in the debug version of the library, as follows:

• 0 - This is the default level when PMEM2_LOG_LEVEL is not set. No log messages are emitted at this
level.

• 1 - Additional details on any errors detected are logged, in addition to returning the errno-based
errors as usual. The same information may be retrieved using pmem2_errormsg().

• 2 - A trace of basic operations is logged.

• 3 - Enables a very verbose amount of function call tracing in the library.

• 4 - Enables voluminous and fairly obscure tracing information that is likely only useful to the
libpmem2 developers.

Unless PMEM2_LOG_FILE is set, debugging output is written to stderr.

• PMEM2_LOG_FILE

Specifies the name of a file where all logging information should be written. If the last character in
the name is “-”, the PID of the current process will be appended to the file name when the log file is
created. If PMEM2_LOG_FILE is not set, output is written to stderr.

EXAMPLE

       The following example uses libpmem2 to flush changes made to raw, memory-mapped persistent memory.

              WARNING: There is nothing transactional about the persist  from  pmem2_get_persist_fn(3)  call  in
              this  example.   Interrupting  the  program  may  result  in  a  partial  write  to  pmem.   Use a
              transactional library such as libpmemobj(7) to avoid torn updates.

       The basic  example  can  be  found  in  the  repository  under  path  src/examples/libpmem2/basic/basic.c
       (https://github.com/pmem/pmdk/blob/master/src/examples/libpmem2/basic/basic.c).    It   is  described  in
       detail here (https://pmem.io/pmdk/libpmem2/).

ACKNOWLEDGEMENTS

       libpmem2 builds on the persistent memory programming  model  recommended  by  the  SNIA  NVM  Programming
       Technical Work Group: <https://snia.org/nvmp>