Ubuntu Manpage: libpmemcto - close-to-open persistence (EXPERIMENTAL)

Provided by: libpmemcto-dev_1.4.1-0ubuntu1~18.04.1_amd64

NAME

       libpmemcto - close-to-open persistence (EXPERIMENTAL)

SYNOPSIS

              #include <libpmemcto.h>
              cc ... -lpmemcto -lpmem

   Library API versioning:
              const char *pmemcto_check_version(
                  unsigned major_required,
                  unsigned minor_required);

   Managing library behavior:
              void pmemcto_set_funcs(
                  void *(*malloc_func)(size_t size),
                  void (*free_func)(void *ptr),
                  void *(*realloc_func)(void *ptr, size_t size),
                  char *(*strdup_func)(const char *s),
                  void (*print_func)(const char *s));

   Error handling:
              const char *pmemcto_errormsg(void);

   Other library functions:
       A description of other libpmemcto functions can be found on the following manual pages:

       pmemcto_aligned_alloc(3),   pmemcto_malloc(3),   pmemcto_malloc_usable_size(3),  pmemcto_open(3),  pmemc‐
       to_set_root_pointer(3), pmemcto_stats_print(3), pmemcto_strdup(3), pmemcto_wcsdup(3)

DESCRIPTION

libpmemcto is a persistent memory allocator with no overhead imposed by run-time flushing or transaction‐
al updates:

• It runs at traditional volatile memory allocator speeds - there is no flushing or consistency check at
run-time.

• An overhead imposed only when program exits normally and have to flush the file.

• The program flushes the pool contents when it exits, and then rebuilds the pool on the next run.

• If the program crashes before flushing the file (or if flushing fails), the pool is in an inconsistent
state causing subsequent pool opening to fail.

libpmemcto provides common malloc-like interfaces to persistent memory pools built on memory-mapped
files. libpmemcto uses the mmap(2) system call to create a pool of persistent memory. The library is
intended for applications using Direct Access storage (DAX), which is memory-addressable persistent stor‐
age that supports load/store access without being paged via the system page cache. A Persistent Memo‐
ry-aware file system is typically used to provide this type of access. Memory-mapping a file from a Per‐
sistent Memory-aware file system provides the raw memory pools, and this library supplies the more famil‐
iar malloc-like interfaces on top of those pools. libpmemcto is one of a collection of persistent memory
libraries available, the others are:

• libpmemobj(7), a general use persistent memory API, providing memory allocation and transactional oper‐
ations on variable-sized objects.

• libpmemblk(7), providing pmem-resident arrays of fixed-sized blocks with atomic updates.

• libpmemlog(7), providing a pmem-resident log file.

• libpmem(7), low-level persistent memory support.

Under normal usage, libpmemcto will never print messages or intentionally cause the process to exit. Ex‐
ceptions to this are prints caused by calls to pmemcto_stats_print(3), or by enabling debugging as de‐
scribed under DEBUGGING AND ERROR HANDLING below. The library uses pthreads(7) to be fully MT-safe, but
never creates or destroys threads itself. The library does not make use of any signals, networking, and
never calls select() or poll().

The system memory allocation routines like malloc() and free() are used by libpmemcto for managing a
small amount of run-time state, but applications are allowed to override these calls if necessary (see
pmemcto_set_funcs()).

This library builds on the low-level pmem support provided by libpmem(7).

To use close-to-open persistence supplied by libpmemcto, a memory pool is first created using the pmemc‐
to_create() function described in pmemcto_open(3). The other libpmemcto functions operate on the result‐
ing block memory pool using the opaque handle, of type PMEMctopool*, that is returned by pmemcto_create()
or pmemcto_open(). Internally, libpmemcto will use either pmem_persist(3) or msync(2) when it needs to
flush changes, depending on whether the memory pool appears to be persistent memory or a regular file
(see the pmem_is_pmem(3) function in libpmem(7) for more information). There is no need for applications
to flush changes directly when using the close-to-open persistence memory API provided by libpmemcto.

CAVEATS

       libpmemcto  relies  on  the  library  destructor being called from the main thread.  For this reason, all
       functions that might trigger destruction (e.g.  dlclose(3)) should be called in the main thread.   Other‐
       wise some of the resources associated with that thread might not be cleaned up properly.

LIBRARY API VERSIONING

       This  section  describes how the library API is versioned, allowing applications to work with an evolving
       API.

       The pmemcto_check_version() function is used to determine whether the installed libpmemcto  supports  the
       version of the library API required by an application.  The easiest way to do this is for the application
       to supply the compile-time version information, supplied by defines in <ibpmemcto.h>, like this:

              reason = pmemcto_check_version(PMEMCTO_MAJOR_VERSION,
                                             PMEMCTO_MINOR_VERSION);
              if (reason != NULL) {
                  /* version check failed, reason string tells you why */
              }

       Any  mismatch  in the major version number is considered a failure, but a library with a newer minor ver‐
       sion number will pass this check since increasing minor versions imply backwards compatibility.

       An application can also check specifically for the existence of an interface by checking for the  version
       where  that  interface was introduced.  These versions are documented in this man page as follows: unless
       otherwise specified, all interfaces described here are available in version 1.0 of the  library.   Inter‐
       faces added after version 1.0 will contain the text introduced in version x.y in the section of this man‐
       ual describing the feature.

       When  the  version  check  performed  by pmemcto_check_version() is successful, the return value is NULL.
       Otherwise the return value is a static string describing the reason for failing the version  check.   The
       string returned by pmemcto_check_version() must not be modified or freed.

MANAGING LIBRARY BEHAVIOR

       The pmemcto_set_funcs() function allows an application to override memory allocation calls used internal‐
       ly  by libpmemcto.  Passing in NULL for any of the handlers will cause the libpmemcto default function to
       be used.  The library does not make heavy use of the system malloc functions, but it  does  allocate  ap‐
       proximately 4-8 kilobytes for each memory pool in use.

DEBUGGING AND ERROR HANDLING

       The  pmemcto_errormsg()  function  returns a pointer to a static buffer containing the last error message
       logged for the current thread.  If errno was set, the error message may include a description of the cor‐
       responding error code, as returned by strerror(3).  The error message buffer is thread-local; errors  en‐
       countered in one thread do not affect its value in other threads.  The buffer is never cleared by any li‐
       brary  function;  its content is significant only when the return value of the immediately preceding call
       to a libpmemcto function indicated an error, or if errno was set.  The application  must  not  modify  or
       free the error message string, but it may be modified by subsequent calls to other library functions.

       Two versions of libpmemcto are typically available on a development system.  The normal version, accessed
       when  a  program  is linked using the -lpmemcto option, is optimized for performance.  That version skips
       checks that impact performance and never logs any trace information or performs any run-time  assertions.
       If  an  error  is  detected  in a call to libpmemcto, the error message describing the failure may be re‐
       trieved with pmemcto_errormsg() as described above.

       A second version of libpmemcto, accessed when a program uses  the  libraries  under  /usr/lib/pmdk_debug,
       contains run-time assertions and trace points.  The typical way to access the debug version is to set the
       LD_LIBRARY_PATH  environment  variable  to  /usr/lib/pmdk_debug or /usr/lib64/pmdk_debug, as appropriate.
       Debugging output is controlled using the following environment variables.  These variables have no effect
       on the non-debug version of the library.

       • PMEMCTO_LOG_LEVEL

       The value of PMEMCTO_LOG_LEVEL enables trace points in the debug version of the library, as follows:

       • 0 - This is the default level when PMEMCTO_LOG_LEVEL is not set.  No log messages are emitted  at  this
         level.

       • 1  - Additional details on any errors detected are logged, in addition to returning the errno-based er‐
         rors as usual.  The same information may be retrieved using pmemcto_errormsg().

       • 2 - A trace of basic operations is logged.

       • 3 - Enables a very verbose amount of function call tracing in the library.

       • 4 - Enables voluminous and fairly obscure tracing information that is likely only useful to  the  libp‐
         memcto developers.

       Unless PMEMCTO_LOG_FILE is set, debugging output is written to stderr.

       • PMEMCTO_LOG_FILE

       Specifies  the  name of a file where all logging information should be written.  If the last character in
       the name is “-”, the PID of the current process will be appended to the file name when the  log  file  is
       created.  If PMEMCTO_LOG_FILE is not set, the logging output is written to stderr.

       See also libpmem(7) for information on other environment variables that may affect libpmemcto behavior.

EXAMPLE

       The  following example creates a memory pool, allocates some memory to contain the string “hello, world”,
       and then frees that memory.

              #include <stdio.h>
              #include <fcntl.h>
              #include <errno.h>
              #include <stdlib.h>
              #include <unistd.h>
              #include <string.h>
              #include <libpmemcto.h>

              /* size of the pmemcto pool -- 1 GB */
              #define POOL_SIZE ((size_t)(1 << 30))

              /* name of our layout in the pool */
              #define LAYOUT_NAME "example_layout"

              struct root {
                  char *str;
                  char *data;
              };

              int
              main(int argc, char *argv[])
              {
                  const char path[] = "/pmem-fs/myfile";
                  PMEMctopool *pcp;

                  /* create the pmemcto pool or open it if already exists */
                  pcp = pmemcto_create(path, LAYOUT_NAME, POOL_SIZE, 0666);

                  if (pcp == NULL)
                      pcp = pmemcto_open(path, LAYOUT_NAME);

                  if (pcp == NULL) {
                      perror(path);
                      exit(1);
                  }

                  /* get the root object pointer */
                  struct root *rootp = pmemcto_get_root_pointer(pcp);

                  if (rootp == NULL) {
                      /* allocate root object */
                      rootp = pmemcto_malloc(pcp, sizeof(*rootp));

                      if (rootp == NULL) {
                          perror(pmemcto_errormsg());
                          exit(1);
                      }

                      /* save the root object pointer */
                      pmemcto_set_root_pointer(pcp, rootp);

                      rootp->str = pmemcto_strdup(pcp, "Hello World!");
                      rootp->data = NULL;
                  }

                  /* ... */

                  pmemcto_close(pcp);
              }

       See <http://pmem.io/pmdk/libpmemcto> for more examples using the libpmemcto API.

BUGS

       Unlike libpmemobj(3), data replication is not supported in libpmemcto.  Thus, it is not allowed to speci‐
       fy replica sections in pool set files.

NOTES

       Unlike the normal malloc(), which asks the system for additional memory when it runs out, libpmemcto  al‐
       locates the size it is told to and never attempts to grow or shrink that memory pool.

AVAILABILITY

       libpmemcto is part of the PMDK since version 1.4 and is available from <https://github.com/pmem/pmdk>

ACKNOWLEDGEMENTS

       libpmemcto depends on jemalloc, written by Jason Evans, to do the heavy lifting of managing dynamic memo‐
       ry allocation.  See: <http://www.canonware.com/jemalloc>

       libpmemcto  builds  on  the  persistent  memory programming model recommended by the SNIA NVM Programming
       Technical Work Group: <http://snia.org/nvmp>