Ubuntu Manpage: libpmem2 - persistent memory support library

Provided by: libpmem2-dev_1.12.0-2_amd64

NAME

       libpmem2 - persistent memory support library

SYNOPSIS

              #include <libpmem2.h>
              cc ... -lpmem2

DESCRIPTION

       libpmem2 provides low-level persistent memory (pmem) support for applications using direct
       access storage (DAX), which is storage that  supports  load/store  access  without  paging
       blocks  from  a  block  storage device.  Some types of non-volatile memory DIMMs (NVDIMMs)
       provide this type of byte addressable access to storage.  A persistent memory  aware  file
       system  is  typically  used to expose the direct access to applications.  Memory mapping a
       file from this type of file system results in the load/store, non-paged access to pmem.

       This library is for applications that use persistent memory directly, without the help  of
       any  library-supplied  transactions  or  memory  allocation.   Higher-level libraries that
       currently build on  libpmem  (previous  variation  of  libpmem2)  are  available  and  are
       recommended for most applications, see:

       • libpmemobj(7),  a  general  use  persistent  memory API, providing memory allocation and
         transactional operations on variable-sized objects.

       • libpmemblk(7), providing pmem-resident arrays of fixed-sized blocks with atomic updates.

       • libpmemlog(7), providing a pmem-resident log file.

       The libpmem2 library  provides  a  comprehensive  set  of  functions  for  robust  use  of
       Persistent  Memory.   It  relies  on  three core concepts: struct pmem2_src source, struct
       pmem2_config config and struct pmem2_map map:

       • source - an object describing the data source for mapping.  The data  source  can  be  a
         file  descriptor,  a  file handle, or an anonymous mapping.  APIs dedicated for creating
         source        are:         pmem2_source_from_fd(3),         pmem2_source_from_handle(3),
         pmem2_source_from_anon(3).

       • config  -  an  object  containing  parameters  that  are used to create a mapping from a
         source.  The configuration structure must always be provided to create  a  mapping,  but
         the only required parameter to set in the config is granularity.  The granularity should
         by set using dedicated libpmem2 function  pmem2_config_set_required_store_granularity(3)
         which  defines  a  maximum  permitted  granularity  requested  by  the  user.   For more
         information about the granularity concept read GRANULARITY section below.

       In addition to the granularity setting, libpmem2 provides multiple optional  functions  to
       configure  target  mapping,  e.g.,  pmem2_config_set_length(3) to set length which will be
       used for mapping, or pmem2_config_set_offset(3) which will be used  to  map  the  contents
       from  the  specified location of the source, pmem2_config_set_sharing(3) which defines the
       behavior and visibility of writes to the mapping's pages.

       • map - an object created  by  pmem2_map_new(3)  using  source  and  config  as  an  input
         parameters.   The  map  structure  can  be  then used to directly operate on the created
         mapping through the use of its associated set  of  functions:  pmem2_map_get_address(3),
         pmem2_map_get_size(3),  pmem2_map_get_store_granularity(3)  -  for getting address, size
         and effective mapping granularity.

       In addition to the basic functionality of managing the virtual address  mapping,  libpmem2
       also  provides  optimized  functions  for  modifying  the mapped data.  This includes data
       flushing as well as memory copying.

       To   get   proper    function    for    data    flushing    use:    pmem2_get_flush_fn(3),
       pmem2_get_persist_fn(3)  or  pmem2_get_drain_fn(3).  To get proper function for copying to
       persistent  memory,  use  map  getters:  pmem2_get_memcpy_fn(3),   pmem2_get_memset_fn(3),
       pmem2_get_memmove_fn(3).

       The  libpmem2  API  also  provides  support  for  the  badblock  and unsafe shutdown state
       handling.

       To   read   or    clear    badblocks,    the    following    functions    are    provided:
       pmem2_badblock_context_new(3),   pmem2_badblock_context_delete(3),  pmem2_badblock_next(3)
       and pmem2_badblock_clear(3).

       To handle unsafe shutdown in  the  application,  the  following  functions  are  provided:
       pmem2_source_device_id(3),  pmem2_source_device_usc(3).   More  detailed information about
       unsafe  shutdown  detection  and   unsafe   shutdown   count   can   be   found   in   the
       libpmem2_unsafe_shutdown(7) man page.

GRANULARITY

The libpmem2 library introduces the concept of granularity through which you may easily
distinguish between different levels of storage performance capabilities available to the
application as related to power-fail protected domain. The way data reaches this
protected domain differs based on the platform and storage device capabilities.

Traditional block storage devices (SSD, HDD) must use system API calls such as msync(),
fsync() on Linux, or FlushFileBuffers(),FlushViewOfFile() on Windows to write data
reliably. Invoking these functions flushes the data to the medium with page granularity.
In the libpmem2 library, this type of flushing behavior is called PMEM2_GRANULARITY_PAGE.

In systems with persistent memory support, a power-fail protected domain may cover
different sets of resources: either the memory controller or the memory controller and CPU
caches. For this reason, libpmem2 distinguishes two types of granularity for persistent
memory: PMEM2_GRANULARITY_CACHE_LINE and PMEM2_GRANULARITY_BYTE.

If the power-fail protected domain covers only the memory controller, the CPU appropriate
cache lines must be flushed for the data to be considered persistent. This granularity
type is called PMEM2_GRANULARITY_CACHE_LINE. Depending on the architecture, there are
different types of machine instructions for flushing cache lines (e.g., CLWB, CLFLUSHOPT,
CLFLUSH for Intel x86_64 architecture). Usually, to ensure the ordering of stores, such
instructions must be followed by a barrier (e.g., SFENCE).

The third type of granularity PMEM2_GRANULARITY_BYTE applies to platforms where power-fail
protected domain covers both the memory controller and CPU caches. In such cases, cache
flush instructions are no longer needed, and the platform itself guarantees the
persistence of data. But barriers might still be required for ordering.

The library declares these granularity level in pmem2_granularity enum, which the
application must set in pmem2_config to the appropriate level for a mapping to succeed.
The software should set this config parameter to a value that most accurately represents
the target hardware characteristics and the storage patterns of the application. For
example, a database storage engine that operates on large logical pages that reside either
on SSDs or PMEM should set this value to PMEM2_GRANULARITY_PAGE. The library will create
mappings where the new map granularity is lower or equal to the requested one. For
example, a mapping with PMEM2_GRANULARITY_CACHE_LINE can be created for the required
granularity PMEM2_GRANULARITY_PAGE, but not vice versa.

CAVEATS

       libpmem2 relies on the library destructor being called from the  main  thread.   For  this
       reason, all functions that might trigger destruction (e.g. dlclose(3)) should be called in
       the main thread.  Otherwise some of the resources associated with that thread might not be
       cleaned up properly.

ENVIRONMENT

libpmem2 can change its default behavior based on the following environment variables.
These are primarily intended for testing and are generally not required.

• PMEM2_FORCE_GRANULARITY=val

Setting this environment variable to val forces libpmem2 to use persist method specific
for forced granularity and skip granularity autodetecting mechanism. The concept of the
granularity is described in GRANULARITY section above. This variable is intended for use
during library testing.

The val argument accepts following text values:

• BYTE - force byte granularity.

• CACHE_LINE - force cache line granularity.

• PAGE - force page granularity.

Granularity values listed above are case-insensitive.

NOTE: The value of PMEM2_FORCE_GRANULARITY is not queried (and cached) at library
initialization time, but read during each pmem2_map_new(3) call.

This means that PMEM2_FORCE_GRANULARITY may still be set or modified by the program until
the first attempt to map a file.

• PMEM_NO_CLWB=1

Setting this environment variable to 1 forces libpmem2 to never issue the CLWB instruction
on Intel hardware, falling back to other cache flush instructions on that hardware instead
(CLFLUSHOPT or CLFLUSH). Without this setting, libpmem2 will always use the CLWB
instruction for flushing processor caches on platforms that support this instruction.
This variable is intended for use during library testing, but may be required for some
rare cases when using CLWB has a negative impact on performance.

• PMEM_NO_CLFLUSHOPT=1

Setting this environment variable to 1 forces libpmem2 to never issue the CLFLUSHOPT
instruction on Intel hardware, falling back to the CLFLUSH instructions instead. Without
this environment variable, libpmem2 will always use the CLFLUSHOPT instruction for
flushing processor caches on platforms that support the instruction, but where CLWB is not
available. This variable is intended for use during library testing.

• PMEM_NO_MOVNT=1

Setting this environment variable to 1 forces libpmem2 to never use the non-temporal move
instructions on Intel hardware. Without this environment variable, libpmem2 will use the
non-temporal instructions for copying larger ranges to persistent memory on platforms that
support these instructions. This variable is intended for use during library testing.

• PMEM_MOVNT_THRESHOLD=val

This environment variable allows overriding the minimum length of the pmem2_memmove_fn
operations, for which libpmem2 uses non-temporal move instructions. Setting this
environment variable to 0 forces libpmem2 to always use the non-temporal move instructions
if available. It has no effect if PMEM_NO_MOVNT is set to 1. This variable is intended
for use during library testing.

DEBUGGING

Two versions of libpmem2 are typically available on a development system. The normal
version, accessed when a program is linked using the -lpmem2 option, is optimized for
performance. That version skips checks that impact performance and never logs any trace
information or performs any run-time assertions.

A second version of libpmem2, accessed when a program uses the libraries under
/usr/lib/pmdk_debug, contains run-time assertions and trace points. The typical way to
access the debug version is to set the environment variable LD_LIBRARY_PATH to
/usr/lib/pmdk_debug or /usr/lib64/pmdk_debug, as appropriate. Debugging output is
controlled using the following environment variables. These variables have no effect on
the non-debug version of the library.

NOTE: On Debian/Ubuntu systems, this extra debug version of the library is shipped
in the respective -debug Debian package and placed in the /usr/lib/$ARCH/pmdk_dbg/
directory.

• PMEM2_LOG_LEVEL

The value of PMEM2_LOG_LEVEL enables trace points in the debug version of the library, as
follows:

• 0 - This is the default level when PMEM2_LOG_LEVEL is not set. No log messages are
emitted at this level.

• 1 - Additional details on any errors detected are logged, in addition to returning the
errno-based errors as usual. The same information may be retrieved using
pmem2_errormsg().

• 2 - A trace of basic operations is logged.

• 3 - Enables a very verbose amount of function call tracing in the library.

• 4 - Enables voluminous and fairly obscure tracing information that is likely only useful
to the libpmem2 developers.

Unless PMEM2_LOG_FILE is set, debugging output is written to stderr.

• PMEM2_LOG_FILE

Specifies the name of a file where all logging information should be written. If the last
character in the name is “-”, the PID of the current process will be appended to the file
name when the log file is created. If PMEM2_LOG_FILE is not set, output is written to
stderr.

EXAMPLE

       The following example uses libpmem2 to flush changes made to raw, memory-mapped persistent
       memory.

              WARNING:   There   is   nothing    transactional    about    the    persist    from
              pmem2_get_persist_fn(3)  call in this example.  Interrupting the program may result
              in a partial write to pmem.  Use a transactional library such as  libpmemobj(7)  to
              avoid torn updates.

       The above example is described in detail here (https://pmem.io/pmdk/libpmem2/).

ACKNOWLEDGEMENTS

       libpmem2  builds  on  the  persistent memory programming model recommended by the SNIA NVM
       Programming Technical Work Group: <https://snia.org/nvmp>