focal (7) lvmcache.7.gz

Provided by: lvm2_2.03.07-1ubuntu1_amd64 bug

NAME

       lvmcache — LVM caching

DESCRIPTION

       lvm(8)  includes  two  kinds  of  caching that can be used to improve the performance of a Logical Volume
       (LV). Typically, a smaller, faster device is used to improve i/o performance of a larger, slower  LV.  To
       do  this, a separate LV is created from the faster device, and then the original LV is converted to start
       using the fast LV.

       The two kinds of caching are:

       • A read and write hot-spot cache, using the dm-cache kernel module.  This  cache  is  slow  moving,  and
         adjusts  the  cache  content  over  time  so  that the most used parts of the LV are kept on the faster
         device.  Both reads and writes use the cache. LVM refers to this using the LV type cache.

       • A streaming write cache, using the dm-writecache kernel module.  This cache is intended to be used with
         SSD  or PMEM devices to speed up all writes to an LV.  Reads do not use this cache.  LVM refers to this
         using the LV type writecache.

USAGE

       Both kinds of caching use similar lvm commands:

       1. Identify main LV that needs caching

       A main LV exists on slower devices.

         $ lvcreate -n main -L Size vg /dev/slow

       2. Identify fast LV to use as the cache

       A fast LV exists on faster devices.  This LV will be used to hold the cache.

         $ lvcreate -n fast -L Size vg /dev/fast

         $ lvs -a
         LV   Attr       Type   Devices
         fast -wi------- linear /dev/fast
         main -wi------- linear /dev/slow

       3. Start caching the main LV

       To start caching the main LV using the fast LV, convert the main LV to  the  desired  caching  type,  and
       specify the fast LV to use:

       using dm-cache:

         $ lvconvert --type cache --cachevol fast vg/main

       using dm-writecache:

         $ lvconvert --type writecache --cachevol fast vg/main

       using dm-cache (with cachepool):

         $ lvconvert --type cache --cachepool fast vg/main

       4. Display LVs

       Once  the  fast  LV  has  been  attached  to the main LV, lvm reports the main LV type as either cache or
       writecache depending on the type used.  While attached, the fast LV is hidden, and renamed with  a  _cvol
       or  _cpool  suffix.   It  is  displayed  by  lvs -a.  The _corig or _wcorig LV represents the original LV
       without the cache.

       using dm-cache:

         $ lvs -a
         LV           Pool        Type   Devices
         main         [fast_cvol] cache  main_corig(0)
         [fast_cvol]              linear /dev/fast
         [main_corig]             linear /dev/slow

       using dm-writecache:

         $ lvs -a
         LV            Pool        Type       Devices
         main          [fast_cvol] writecache main_wcorig(0)
         [fast_cvol]               linear     /dev/fast
         [main_wcorig]             linear     /dev/slow

       using dm-cache (with cachepool):

         $ lvs -a
         LV                 Pool         Type       Devices
         main               [fast_cpool] cache      main_corig(0)
         [fast_cpool]                    cache-pool fast_pool_cdata(0)
         [fast_cpool_cdata]              linear     /dev/fast
         [fast_cpool_cmeta]              linear     /dev/fast
         [main_corig]                    linear     /dev/slow

       5. Use the main LV

       Use the LV until the cache is no longer wanted, or needs to be changed.

       6. Stop caching

       To stop caching the main LV, separate the fast LV from the main LV.  This changes the type of the main LV
       back to what it was before the cache was attached.

         $ lvconvert --splitcache vg/main

         $ lvs -a
         LV   VG Attr       Type   Devices
         fast vg -wi------- linear /dev/fast
         main vg -wi------- linear /dev/slow

OPTIONS

   option args

       --cachevol LV

       Pass this option a standard LV.  With a cachevol, cache data and metadata are contained within the single
       LV.  This is used with dm-writecache or dm-cache.

       --cachepool CachePoolLV|LV

       Pass this option a cache pool object.  With a cache pool, lvm places cache data  and  cache  metadata  on
       different  LVs.   The  two LVs together are called a cache pool.  This permits specific placement of data
       and metadata.  A cache pool is represented as a special type of LV that cannot be used directly.   (If  a
       standard  LV  is  passed  to  this option, lvm will first convert it to a cache pool by combining it with
       another LV to use for metadata.)  This can be used with dm-cache.

   dm-cache block size

       A cache pool will have a logical block size of 4096 bytes if it is created on a  device  with  a  logical
       block size of 4096 bytes.

       If  a  main  LV  has  logical  block size 512 (with an existing xfs file system using that size), then it
       cannot use a cache pool with a 4096 logical block size.  If the cache pool is attached, the main LV  will
       likely fail to mount.

       To  avoid this problem, use a mkfs option to specify a 4096 block size for the file system, or attach the
       cache pool before running mkfs.

   dm-writecache block size

       The dm-writecache block size can be 4096 bytes (the default), or 512 bytes.  The default 4096 has  better
       performance  and  should be used except when 512 is necessary for compatibility.  The dm-writecache block
       size is specified with --cachesettings block_size=4096|512 when caching is started.

       When a file system like xfs already exists on the main LV prior to caching, and the file system is  using
       a  block  size of 512, then the writecache block size should be set to 512.  (The file system will likely
       fail to mount if writecache block size of 4096 is used in this case.)

       Check the xfs sector size while the fs is mounted:

       $ xfs_info /dev/vg/main
       Look for sectsz=512 or sectsz=4096

       The writecache block size should be chosen to match the xfs sectsz value.

       It is also possible to specify a sector size of 4096 to mkfs.xfs when creating the file system.  In  this
       case the writecache block size of 4096 can be used.

   dm-writecache settings

       Tunable parameters can be passed to the dm-writecache kernel module using the --cachesettings option when
       caching is started, e.g.

       $ lvconvert --type writecache --cachevol fast \
            --cachesettings 'high_watermark=N writeback_jobs=N' vg/main

       Tunable options are:

       • high_watermark = <count>

         Start writeback when the number of used blocks reach this watermark

       • low_watermark = <count>

         Stop writeback when the number of used blocks drops below this watermark

       • writeback_jobs = <count>

         Limit the number of blocks that are in flight during writeback.  Setting this value  reduces  writeback
         throughput, but it may improve latency of read requests.

       • autocommit_blocks = <count>

         When  the  application  writes  this amount of blocks without issuing the FLUSH request, the blocks are
         automatically commited.

       • autocommit_time = <milliseconds>

         The data is automatically commited if this time passes and no FLUSH request is received.

       • fua = 0|1

         Use the FUA flag when writing data from persistent memory back to the  underlying  device.   Applicable
         only to persistent memory.

       • nofua = 0|1

         Don't  use  the FUA flag when writing back data and send the FLUSH request afterwards.  Some underlying
         devices perform better with fua, some with nofua.  Testing is necessary to determine which.  Applicable
         only to persistent memory.

   dm-cache with separate data and metadata LVs

       When  using  dm-cache,  the  cache  metadata and cache data can be stored on separate LVs.  To do this, a
       "cache pool" is created, which is a special LV that references two sub LVs, one  for  data  and  one  for
       metadata.

       To create a cache pool from two separate LVs:

       $ lvcreate -n fast -L DataSize vg /dev/fast1
       $ lvcreate -n fastmeta -L MetadataSize vg /dev/fast2
       $ lvconvert --type cache-pool --poolmetadata fastmeta vg/fast

       Then use the cache pool LV to start caching the main LV:

       $ lvconvert --type cache --cachepool fast vg/main

       A  variation  of  the  same  procedure automatically creates a cache pool when caching is started.  To do
       this, use a standard LV as the --cachepool (this will hold cache data), and use another  standard  LV  as
       the  --poolmetadata  (this  will  hold  cache  metadata).   LVM  will create a cache pool LV from the two
       specified LVs, and use the cache pool to start caching the main LV.

       $ lvcreate -n fast -L DataSize vg /dev/fast1
       $ lvcreate -n fastmeta -L MetadataSize vg /dev/fast2
       $ lvconvert --type cache --cachepool fast --poolmetadata fastmeta vg/main

   dm-cache cache modes

       The default dm-cache cache mode is "writethrough".  Writethrough ensures that any data  written  will  be
       stored  both  in  the cache and on the origin LV.  The loss of a device associated with the cache in this
       case would not mean the loss of any data.

       A second cache mode is "writeback".  Writeback delays writing data blocks from  the  cache  back  to  the
       origin LV.  This mode will increase performance, but the loss of a cache device can result in lost data.

       With  the --cachemode option, the cache mode can be set when caching is started, or changed on an LV that
       is already cached.  The current cache mode can be displayed with the cache_mode reporting option:

       lvs -o+cache_mode VG/LV

       lvm.conf(5) allocation/cache_mode
       defines the default cache mode.

       $ lvconvert --type cache --cachevol fast \
            --cachemode writethrough vg/main

   dm-cache chunk size

       The size of data blocks managed by dm-cache can be specified with the --chunksize option when caching  is
       started.  The default unit is KiB.  The value must be a multiple of 32KiB between 32KiB and 1GiB.

       Using  a  chunk  size that is too large can result in wasteful use of the cache, in which small reads and
       writes cause large sections of an LV to be stored in the cache.  However, choosing a chunk size  that  is
       too  small  can  result in more overhead trying to manage the numerous chunks that become mapped into the
       cache.  Overhead can include both excessive CPU time searching for chunks, and excessive memory  tracking
       chunks.

       Command to display the chunk size:
       lvs -o+chunksize VG/LV

       lvm.conf(5) cache_pool_chunk_size
       controls the default chunk size.

       The default value is shown by:
       lvmconfig --type default allocation/cache_pool_chunk_size

   dm-cache cache policy

       The  dm-cache  subsystem  has additional per-LV parameters: the cache policy to use, and possibly tunable
       parameters for the cache policy.  Three policies are currently available: "smq" is  the  default  policy,
       "mq"  is  an  older  implementation,  and  "cleaner" is used to force the cache to write back (flush) all
       cached writes to the origin LV.

       The older "mq" policy has a number of tunable parameters. The defaults are chosen to be suitable for  the
       majority of systems, but in special circumstances, changing the settings can improve performance.

       With the --cachepolicy and --cachesettings options, the cache policy and settings can be set when caching
       is started, or changed on an existing cached LV (both options can be used together).  The  current  cache
       policy and settings can be displayed with the cache_policy and cache_settings reporting options:

       lvs -o+cache_policy,cache_settings VG/LV

       Change the cache policy and settings of an existing LV.

       $ lvchange --cachepolicy mq --cachesettings \
            'migration_threshold=2048 random_threshold=4' vg/main

       lvm.conf(5) allocation/cache_policy
       defines the default cache policy.

       lvm.conf(5) allocation/cache_settings
       defines the default cache settings.

   dm-cache spare metadata LV

       See  lvmthin(7)  for  a  description of the "pool metadata spare" LV.  The same concept is used for cache
       pools.

   dm-cache metadata formats

       There are  two  disk  formats  for  dm-cache  metadata.   The  metadata  format  can  be  specified  with
       --cachemetadataformat  when  caching is started, and cannot be changed.  Format 2 has better performance;
       it is more compact, and stores dirty bits in a separate btree, which improves the speed of shutting  down
       the cache.  With auto, lvm selects the best option provided by the current dm-cache kernel module.

   mirrored cache device

       The  fast LV holding the cache can be created as a raid1 mirror so that it can tolerate a device failure.
       (When using dm-cache with separate data and metadata LVs, each of the sub-LVs can use raid1.)

       $ lvcreate -n main -L Size vg /dev/slow
       $ lvcreate --type raid1 -m 1 -n fast -L Size vg /dev/fast1 /dev/fast2
       $ lvconvert --type cache --cachevol fast vg/main

   dm-cache command shortcut

       A single command can be used to create a cache pool and attach that new cache pool to a main LV:

       $ lvcreate --type cache --name Name --size Size VG/LV [PV]

       In this command, the specified LV already exists, and is the main LV to be cached.  The command creates a
       new  cache pool with the given name and size, using the optionally specified PV (typically an ssd).  Then
       it attaches the new cache pool to the existing main LV to begin caching.

       (Note: ensure that the specified main LV is a standard LV.  If a cache pool LV is  mistakenly  specified,
       then the command does something different.)

       (Note:  the  type  option  is interpreted differently by this command than by normal lvcreate commands in
       which --type specifies the type of the newly created LV.  In this case, an LV  with  type  cache-pool  is
       being created, and the existing main LV is being converted to type cache.)

SEE ALSO

       lvm.conf(5),  lvchange(8), lvcreate(8), lvdisplay(8), lvextend(8), lvremove(8), lvrename(8), lvresize(8),
       lvs(8), vgchange(8), vgmerge(8), vgreduce(8), vgsplit(8)