Provided by: multipath-tools_0.9.9-1ubuntu3_amd64 bug

NAME

       /etc/multipath.conf, /etc/multipath/conf.d/*.conf - multipath daemon configuration file.

DESCRIPTION

       /etc/multipath.conf  is  the  configuration  file  for the multipath daemon. It is used to
       overwrite the built-in configuration table of multipathd.  Any line whose first non-white-
       space character is a '#' is considered a comment line. Empty lines are ignored.

       Currently  used  multipathd  configuration  can  be  displayed  with  the  multipath -t or
       multipathd show config command.

       Additional configuration can be made in drop-in files under /etc/multipath/conf.d.   Files
       ending  in  .conf  in  this  directory  are  read  in  alphabetical  order,  after reading
       /etc/multipath.conf.  They use the same syntax as /etc/multipath.conf itself, and  support
       all  sections and keywords. If a keyword occurs in the same section in multiple files, the
       last occurrence will take precedence over all others.

SYNTAX

       The configuration file contains entries of the form:

              <section> {
                     <attribute> <value>
                     ...
                     <subsection> {
                            <attribute> <value>
                            ...
                     }
              }

       Each section contains one or more attributes or subsections. The recognized  keywords  for
       attributes or subsections depend on the section in which they occur.

       <attribute>  and  <value>  must  be  on a single line.  <attribute> is one of the keywords
       listed in this man page.  <value> is either a simple word (containing  no  whitespace  and
       none of the characters '"', '#', and '!') or one string enclosed in double quotes ("...").
       Outside a quoted string, text starting with '#', and '!' is  regarded  as  a  comment  and
       ignored  until  the  end  of  the  line.  Inside  a  quoted string, '#' and '!' are normal
       characters, and whitespace is preserved.  To represent a double quote character  inside  a
       double  quoted  string,  use  two consecutive double quotes ('""'). Thus '2.5" SSD' can be
       written as "2.5"" SSD".

       Opening braces ('{') must follow the (sub)section name on the same  line.  Closing  braces
       ('}') that mark the end of a (sub)section must be the only non-whitespace character on the
       line. Whitespace is ignored except inside double quotes, thus the indentation shown in the
       above example is helpful for human readers but not mandatory.

       Note  on  regular expressions: The /etc/multipath.conf syntax allows many attribute values
       to be specified as POSIX  Extended  Regular  Expressions  (see  regex(7)).  These  regular
       expressions  are  case-sensitive  and  not  anchored,  thus  the  expression "bar" matches
       "barbie", "rhabarber", and "wunderbar", but not  "Barbie".  To  avoid  unwanted  substring
       matches,  standard  regular expression syntax using the special characters "^" and "$" can
       be used.

       The following section keywords are recognized:

       defaults         This section  defines  default  values  for  attributes  which  are  used
                        whenever  no  values  are  given  in  the appropriate device or multipath
                        sections.

       blacklist        This section defines which devices should be excluded from the  multipath
                        topology discovery.

       blacklist_exceptions
                        This  section  defines  which devices should be included in the multipath
                        topology discovery, despite being listed in the blacklist section.

       multipaths       This section defines the multipath topologies.  They  are  indexed  by  a
                        World  Wide  Identifier(WWID).  For  details  on  the WWID generation see
                        section WWID generation  below.  Attributes  set  in  this  section  take
                        precedence over all others.

       devices          This section defines the device-specific settings. Devices are identified
                        by vendor, product, and revision.

       overrides        This section defines values  for  attributes  that  should  override  the
                        device-specific settings for all devices.

defaults section

       The defaults section recognizes the following keywords:

       verbosity        Default  verbosity.  Higher  values  increase  the verbosity level. Valid
                        levels are between 0 and 6.

                        The default is: 2

       polling_interval Interval between two path checks in  seconds.  For  properly  functioning
                        paths,   the   interval   between   checks  will  gradually  increase  to
                        max_polling_interval.  This value will be overridden by  the  WatchdogSec
                        setting in the multipathd.service definition if systemd is used.

                        The default is: 5

       max_polling_interval
                        Maximal interval between two path checks in seconds.

                        The default is: 4 * polling_interval

       reassign_maps    Enable  reassigning  of  device-mapper  maps. With this option multipathd
                        will remap existing device-mapper  maps  to  always  point  to  multipath
                        device, not the underlying block devices. Possible values are yes and no.

                        The default is: no

       multipath_dir    (Deprecated) This option is not supported anymore, and will be ignored.

       path_selector    The  default  path  selector  algorithm  to  use; they are offered by the
                        kernel multipath target:

                        round-robin 0
                                    Loop through every path in the path group, sending  the  same
                                    amount  of  I/O  to  each.  Some  aspects  of behavior can be
                                    controlled with the attributes: rr_min_io,  rr_min_io_rq  and
                                    rr_weight.

                        queue-length 0
                                    (Since  2.6.31  kernel) Choose the path for the next bunch of
                                    I/O based on the amount of outstanding I/O to the path.

                        service-time 0
                                    (Since 2.6.31 kernel) Choose the path for the next  bunch  of
                                    I/O  based  on  the amount of outstanding I/O to the path and
                                    its relative throughput.

                        historical-service-time 0
                                    (Since 5.8 kernel) Choose the path for the next bunch of  I/O
                                    based  on  the estimation of future service time based on the
                                    history of previous I/O submitted to each path.

                        The default is: service-time 0

       path_grouping_policy
                        The default path grouping policy  to  apply  to  unspecified  multipaths.
                        Possible values are:

                        failover    One path per priority group.

                        multibus    All paths in one priority group.

                        group_by_serial
                                    One priority group per serial number.

                        group_by_prio
                                    One   priority  group  per  priority  value.  Priorities  are
                                    determined by callout programs specified as  a  global,  per-
                                    controller or per-multipath option in the configuration file.

                        group_by_node_name
                                    One  priority  group  per target node name. Target node names
                                    are fetched in /sys/class/fc_transport/target*/node_name.

                        group_by_tpg
                                    One priority group per ALUA target port group.  In  order  to
                                    use  this policy, all paths in the multipath device must have
                                    prio set to alua.

                        The default is: failover

       detect_pgpolicy  If set to yes and all path devices are configured with either the alua or
                        sysfs  prioritizer,  the  multipath  device  will  automatically  use the
                        group_by_prio    path_grouping_policy.    If    set    to     no,     the
                        path_grouping_policy will be selected as usual.

                        The default is: yes

       detect_pgpolicy_use_tpg
                        If  both this and detect_pgpolicy are set to yes and all path devices are
                        configured with either the  alua  or  sysfs  prioritizer,  the  multipath
                        device  will  automatically use the group_by_tpg path_grouping_policy. If
                        set to no, the  path_grouping_policy  will  be  selected  by  the  method
                        described for detect_pgpolicy above.

                        The default is: no

       pg_timeout       (Deprecated) This option is not supported anymore, and will be ignored.

       uid_attrs        Setting  this option activates merging uevents by WWID, which may improve
                        uevent processing efficiency. Moreover, it's  an  alternative  method  to
                        configure  the  udev  properties  to  use  for  determining  unique  path
                        identifiers (WWIDs).

                        The value of this option is  a  space  separated  list  of  records  like
                        "type:ATTR",  where  type  is matched against the beginning of the device
                        node name (e.g. sd:ATTR matches sda), and ATTR is the name  of  the  udev
                        property to use for matching devices.

                        If  this  option  is  configured  and  matches  the device node name of a
                        device, it overrides any other configured  methods  for  determining  the
                        WWID for this device.

                        This  option  cannot  be  changed  during  runtime  with  the  multipathd
                        reconfigure command.

                        The default is: <unset>.  To  enable  uevent  merging,  set  it  e.g.  to
                        "sd:ID_SERIAL dasd:ID_UID nvme:ID_WWN".

       uid_attribute    The  udev  attribute  providing  a  unique  path  identifier  (WWID).  If
                        uid_attribute is set to the empty  string,  WWID  determination  is  done
                        using  the  sysfs  method  rather  than  using  udev  (not recommended in
                        production; see WWID generation below).

                        The default is: ID_SERIAL, for SCSI devices

                        The default is: ID_UID, for DASD devices

                        The default is: ID_WWN, for NVMe devices

       getuid_callout   (Deprecated) This option is not supported anymore, and will be ignored.

       prio             The name of the path  priority  routine.  The  specified  routine  should
                        return  a  numeric  value  specifying the relative priority of this path.
                        Higher number have a higher priority.  "none" is a valid value. Currently
                        the following path priority routines are implemented:

                        const       Return a constant priority of 1.

                        sysfs       Use  the  sysfs attributes access_state and preferred_path to
                                    generate the path  priority.  This  prioritizer  accepts  the
                                    optional prio_arg exclusive_pref_bit.

                        emc         (Hardware-dependent) Generate the path priority for DGC class
                                    arrays as CLARiiON CX/AX and EMC VNX families  with  Failover
                                    Mode 1 (Passive Not Ready(PNR)).

                        alua        (Hardware-dependent)  Generate the path priority based on the
                                    SCSI-3 ALUA settings. This prioritizer accepts  the  optional
                                    prio_arg exclusive_pref_bit.

                        ontap       (Hardware-dependent)  Generate  the  path priority for NetApp
                                    ONTAP FAS/AFF Series and rebranded arrays, with ONTAP  native
                                    mode(not ALUA).

                        rdac        (Hardware-dependent)   Generate   the   path   priority   for
                                    LSI/Engenio/NetApp  RDAC  class  as  NetApp  SANtricity  E/EF
                                    Series and rebranded arrays, with "Linux DM-MP (Kernel 3.9 or
                                    earlier)" option.

                        hp_sw       (Hardware-dependent)   Generate   the   path   priority   for
                                    HP/COMPAQ/DEC  HSG80  and  MSA/HSV arrays with Active/Standby
                                    mode exclusively.

                        hds         (Hardware-dependent) Generate the path priority  for  Hitachi
                                    AMS families of arrays other than AMS 2000.

                        random      Generate a random priority between 1 and 10.

                        weightedpath
                                    Generate  the  path  priority based on the regular expression
                                    and the priority provided  as  argument.  Requires  prio_args
                                    keyword.

                        path_latency
                                    Generate  the  path  priority  based  on a latency algorithm.
                                    Requires prio_args keyword.

                        ana         (Hardware-dependent) Generate the path priority based on  the
                                    NVMe ANA settings.

                        datacore    (Hardware-dependent)  Generate  the  path  priority  for some
                                    DataCore storage arrays. Requires prio_args keyword.

                        iet         (iSCSI only) Generate path priority for iSCSI  targets  based
                                    on IP address. Requires prio_args keyword.

                        The  default  depends  on  the detect_prio setting: If detect_prio is yes
                        (default), the default priority algorithm is  sysfs  (except  for  NetAPP
                        E/EF  Series,  where  it  is  alua).  If  detect_prio  is no, the default
                        priority algorithm is const.

       prio_args        Arguments to pass to to the prio function. This only applies  to  certain
                        prioritizers:

                        weighted    Needs a value of the form "<hbtl|devname|serial|wwn> <regex1>
                                    <prio1> <regex2> <prio2> ..."

                                    hbtl    Regex can be of SCSI  H:B:T:L  format.  For  example:
                                            1:0:.:. , *:0:0:.

                                    devname Regex  can be of device name format. For example: sda
                                            , sd.e

                                    serial  Regex can be of serial number  format.  For  example:
                                            .*J1FR.*324  .  The  serial  can be looked up through
                                            sysfs or by  running  multipathd  show  paths  format
                                            "%z". For example: 0395J1FR904324

                                    wwn     Regex       can       be       of       the      form
                                            "host_wwnn:host_wwpn:target_wwnn:target_wwpn"   these
                                            values  can  be looked up through sysfs or by running
                                            multipathd  show  paths  format  "%N:%R:%n:%r".   For
                                            example:  0x200100e08ba0aea0:0x210100e08ba0aea0:.*:.*
                                            , .*:.*:iqn.2009-10.com.redhat.msp.lab.ask-06:.*

                        path_latency
                                    Needs a value of the form "io_num=<20> base_num=<10>"

                                    io_num  The number of read  IOs  sent  to  the  current  path
                                            continuously,  used  to  calculate  the  average path
                                            latency.  Valid Values: Integer, [2, 200].

                                    base_num
                                            The base number value of logarithmic scale,  used  to
                                            partition  different  priority  ranks.  Valid Values:
                                            Integer, [2, 10]. And Max average  latency  value  is
                                            100s, min average latency value is 1us.  For example:
                                            If base_num=10, the paths will be grouped in priority
                                            groups  with  path latency <=1us, (1us, 10us], (10us,
                                            100us], (100us, 1ms],  (1ms,  10ms],  (10ms,  100ms],
                                            (100ms, 1s], (1s, 10s], (10s, 100s], >100s.

                        alua        If  exclusive_pref_bit  is set, paths with the preferred path
                                    bit set will always be in their own path group.

                        sysfs       If exclusive_pref_bit is set, paths with the  preferred  path
                                    bit set will always be in their own path group.

                        datacore

                                    preferredsds
                                            (Mandatory) The preferred "SDS name".

                                    timeout (Optional) The timeout for the INQUIRY, in ms.

                        iet

                                    preferredip=...
                                            (Mandatory)   Th  preferred  IP  address,  in  dotted
                                            decimal notation, for iSCSI targets.

                        The default is: <unset>

       features         Specify any device-mapper features to be used. Syntax is num  list  where
                        num is the number, between 0 and 8, of features in list.  Possible values
                        for the feature list are:

                        queue_if_no_path
                                    (Deprecated, superseded by no_path_retry)  Queue  I/O  if  no
                                    path  is  active.   Identical to the no_path_retry with queue
                                    value. If both this feature and no_path_retry  are  set,  the
                                    latter value takes precedence. See KNOWN ISSUES.

                        pg_init_retries <times>
                                    (Since  kernel  2.6.24)  Number of times to retry pg_init, it
                                    must be between 1 and 50.

                        pg_init_delay_msecs <msecs>
                                    (Since kernel 2.6.38) Number of msecs before  pg_init  retry,
                                    it must be between 0 and 60000.

                        queue_mode <mode>
                                    (Since  kernel  4.8)  Select  the queueing mode per multipath
                                    device.  <mode> can be bio, rq or mq,  which  corresponds  to
                                    bio-based,   request-based,   and  block-multiqueue  (blk-mq)
                                    request-based, respectively.  Before kernel 4.20 The  default
                                    depends  on  the kernel parameter dm_mod.use_blk_mq. It is mq
                                    if the latter is set, and rq otherwise. Since kernel 4.20, rq
                                    and  mq both correspond to block-multiqueue. Once a multipath
                                    device has been created, its queue_mode  cannot  be  changed.
                                    nvme:tcp  paths  are only supported in multipath devices with
                                    queue_mode set to bio. multipath will automatically set  this
                                    when creating a device with nvme:tcp paths.

                        The default is: 0

       path_checker     The  default  method  used to determine the path's state. The synchronous
                        checkers (all except tur and directio) will  cause  multipathd  to  pause
                        most  activity,  waiting  up  to  checker_timeout seconds for the path to
                        respond. The asynchronous checkers (tur  and  directio)  will  not  pause
                        multipathd.  Instead,  multipathd  will  check  for  a  response once per
                        second, until checker_timeout seconds have elapsed. Possible values are:

                        readsector0 (Deprecated) Read  the  first  sector  of  the  device.  This
                                    checker  is  being  deprecated,  please  use  tur or directio
                                    instead.

                        tur         Issue a TEST UNIT READY command to the device.

                        emc_clariion
                                    (Hardware-dependent) Query the  DGC/EMC  specific  EVPD  page
                                    0xC0  to  determine the path state for CLARiiON CX/AX and EMC
                                    VNX and Unity arrays families.

                        hp_sw       (Hardware-dependent) Check the path state  for  HP/COMPAQ/DEC
                                    HSG80   and   MSA/HSV   arrays   with   Active/Standby   mode
                                    exclusively.

                        rdac        (Hardware-dependent)    Check    the    path    state     for
                                    LSI/Engenio/NetApp  RDAC  class  as  NetApp  SANtricity  E/EF
                                    Series, and rebranded arrays.

                        directio    Read the first sector with direct  I/O.  This  checker  could
                                    cause  spurious  path  failures  under  high load. Increasing
                                    checker_timeout can help with this.

                        cciss_tur   (Hardware-dependent) Check the path state for HP/COMPAQ Smart
                                    Array(CCISS) controllers.

                        none        Do not check the device, fallback to use the values retrieved
                                    from sysfs

                        The default is: tur

       alias_prefix     The user_friendly_names prefix.

                        The default is: mpath

       failback         Tell multipathd how to manage path group failback.  To  select  immediate
                        or  a  value,  it's  mandatory  that the device has support for a working
                        prioritizer.

                        immediate   Immediately failback to the highest priority  pathgroup  that
                                    contains active paths.

                        manual      Do not perform automatic failback.

                        followover  Used  to  deal  with  multiple  computers  accessing the same
                                    Active/Passive  storage  devices.  Only   perform   automatic
                                    failback  when  the first path of a pathgroup becomes active.
                                    This keeps a cluster node  from  automatically  failing  back
                                    when another node requested the failover.

                        values > 0  Deferred failback (time to defer in seconds).

                        The default is: manual

       rr_min_io        Number of I/O requests to route to a path before switching to the next in
                        the same path group. This is only for Block I/O(BIO) based multipath  and
                        only apply to round-robin path_selector.

                        The default is: 1000

       rr_min_io_rq     Number of I/O requests to route to a path before switching to the next in
                        the same path group. This is only for Request based  multipath  and  only
                        apply to round-robin path_selector.

                        The default is: 1

       max_fds          Specify  the  maximum  number  of  file descriptors that can be opened by
                        multipath and multipathd. This is equivalent to ulimit -n. A value of max
                        will  set  this to the system limit from /proc/sys/fs/nr_open. If this is
                        not set, the maximum number of open file descriptors is  taken  from  the
                        calling  process.  It  is usually 1024. To be safe, this should be set to
                        the maximum number of paths plus 32, if that number is greater than 1024.

                        The default is: max

       rr_weight        If set to priorities the multipath configurator will assign path  weights
                        as  "path  prio * rr_min_io". Possible values are priorities or uniform .
                        Only apply to round-robin path_selector.

                        The default is: uniform

       no_path_retry    Specify what to do when all paths are down. Possible values are:

                        value > 0   Number of retries until disable I/O queueing.

                        fail        For immediate failure (no I/O queueing).

                        queue       For never stop I/O queueing, similar to queue_if_no_path. See
                                    KNOWN ISSUES.

                        The default is: fail

       queue_without_daemon
                        If set to no , when multipathd stops, queueing will be turned off for all
                        devices.  This is useful for devices that set no_path_retry. If a machine
                        is shut down while all paths to a device are down, it is possible to hang
                        waiting for I/O to return from  the  device  after  multipathd  has  been
                        stopped.  Without  multipathd  running,  access  to  the  paths cannot be
                        restored, and the kernel cannot be told to  stop  queueing  I/O.  Setting
                        queue_without_daemon to no , avoids this problem.

                        The default is: no

       checker_timeout  Specify  the  timeout  to  use  for  path  checkers  and prioritizers, in
                        seconds.  Only prioritizers that issue scsi commands use checker_timeout.
                        If  a  path does not respond to the checker command after checker_timeout
                        seconds have elapsed, it is considered down.

                        The default is: in /sys/block/<dev>/device/timeout

       allow_usb_devices
                        If set to no , all USB devices will be skipped during path discovery.  If
                        you intend to use multipath on USB attached devices, set this to yes.

                        The default is: no

       flush_on_last_del
                        If set to always , multipathd will disable queueing when the last path to
                        a device has been deleted. If set to never , multipathd will not  disable
                        queueing when the last path to a device has been deleted. Since multipath
                        cannot safely remove a device while queueing is enabled, setting this  to
                        never  means  that  multipathd  will  not  automatically remove an unused
                        multipath device whose paths are all deleted if it is  currently  set  to
                        queue_if_no_path.  If  set  to  unused  ,  multipathd  will  only disable
                        queueing when the last path is  removed  if  nothing  currently  has  the
                        multipath  device  or  any  of  the kpartx partition devices on top of it
                        open.

                        The default is: unused

       user_friendly_names
                        If set to yes , using the bindings file /etc/multipath/bindings to assign
                        a  persistent and unique alias to the multipath, in the form of mpath<n>.
                        If set to no use the WWID as the alias. In either case this  be  will  be
                        overridden by any specific aliases in the multipaths section.

                        The default is: no

       fast_io_fail_tmo Specify  the  number  of seconds the SCSI layer will wait after a problem
                        has been detected on a FC remote port before failing I/O  to  devices  on
                        that remote port.  This should be smaller than dev_loss_tmo. Setting this
                        to off will disable the timeout.

                        The default is: 5

       dev_loss_tmo     Specify the number of seconds the SCSI layer will wait after a connection
                        loss  has  been  detected  on  a  remote port before removing it from the
                        system. This can be set to "infinity", which effectively means 136  years
                        (2^32-1  seconds).   This  parameter is only applied to Fibre Channel and
                        SAS devices.

                        The  value  of  dev_loss_tmo  is  restricted  by  other   settings.    If
                        fast_io_fail_tmo  is  set  to a positive value, multipathd will make sure
                        that  the  value  of  dev_loss_tmo  is  larger   than   no_path_retry   *
                        polling_interval.   If fast_io_fail_tmo is not set, the kernel limits the
                        dev_loss_tmo value to 600 seconds.  In this case, the user  has  to  make
                        sure  that no_path_retry is smaller than dev_loss_tmo / polling_interval.
                        In particular, no_path_retry must  not  be  set  to  "queue".  See  KNOWN
                        ISSUES.

                        When path devices reappear after a connection loss, it is much easier for
                        the kernel to simply reactivate an  inactive  device  than  to  re-add  a
                        previously  deleted  one. It is therefore recommended to set dev_loss_tmo
                        to a large value within the restrictions mentioned above.

                        Fibre Channel and SAS devices have hardware-dependent defaults, which are
                        left  unchanged  if  dev_loss_tmo  is  not  specified.  For a few storage
                        arrays, the multipath-tools built-in settings override the  default.  Run
                        multipath -T to see the settings for your device.

                        The default is: <hardware dependent>

       eh_deadline      Specify  the  maximum  number  of seconds the SCSI layer will spend doing
                        error handling when scsi devices fail. After this timeout the scsi  layer
                        will  perform  a  full  HBA reset. Setting this may be necessary in cases
                        where the rport is never lost, so fast_io_fail_tmo and dev_loss_tmo  will
                        never  trigger,  but  (frequently  due to load) scsi commands still hang.
                        Note: when the scsi error handler performs  the  HBA  reset,  all  target
                        paths  on  that  HBA  will be affected. eh_deadline should only be set in
                        cases where all targets on the affected HBAs are multipathed.

                        The default is: <unset>

       max_retries      Specify the maximum number of times the SCSI layer will retry IO commands
                        for  some types of SCSI errors before returning failure. Setting this can
                        be helpful for cases where IO commands hang and timeout. By default,  the
                        SCSI  layer  will  retry  IOs  5  times.  Reducing  this value will allow
                        multipath to retry the IO down another path sooner. Valid  values  are  0
                        through 5.

                        The default is: <unset>

       bindings_file    (Deprecated) This option is not supported anymore, and will be ignored.

                        The compiled-in value is: /etc/multipath/bindings

       wwids_file       (Deprecated) This option is not supported anymore, and will be ignored.

                        The compiled-in value is: /etc/multipath/wwids

       prkeys_file      (Deprecated) This option is not supported anymore, and will be ignored.

                        The compiled-in value is: /etc/multipath/prkeys

       log_checker_err  If  set to once , multipathd logs the first path checker error at logging
                        level 2. Any later errors are logged at  level  3  until  the  device  is
                        restored.  If  set  to  always  , multipathd always logs the path checker
                        error at logging level 2.

                        The default is: always

       reservation_key  This is the service action reservation key used by mpathpersist. It  must
                        be  set  for  all multipath devices using persistent reservations, and it
                        must be the same as the RESERVATION KEY field of the  PERSISTENT  RESERVE
                        OUT  parameter  list  which  contains  an  8-byte  value  provided by the
                        application client to the device server to identify the I_T nexus. If the
                        --param-aptpl  option is used when registering the key with mpathpersist,
                        :aptpl must be appended to the end of the reservation key.

                        Alternatively, this can be set to file, which will store the  RESERVATION
                        KEY  registered  by mpathpersist in the prkeys_file. multipathd will then
                        use this key to register additional  paths  as  they  appear.   When  the
                        registration  is  removed,  the  RESERVATION  KEY  is  removed  from  the
                        prkeys_file. The prkeys file will automatically keep track of whether the
                        key was registered with --param-aptpl.

                        The default is: <unset>

       all_tg_pt        Set the 'all targets ports' flag when registering keys with mpathpersist.
                        Some arrays automatically set and clear registration keys on  all  target
                        ports  from  a  host,  instead of per target port per host. The ALL_TG_PT
                        flag must be set  to  successfully  use  mpathpersist  on  these  arrays.
                        Setting  this  option  is identical to calling mpathpersist with --param-
                        alltgpt

                        The default is: no

       retain_attached_hw_handler
                        (Obsolete for kernels >= 4.3) If set  to  yes  and  the  SCSI  layer  has
                        already  attached  a  hardware_handler  to the device, multipath will not
                        force   the   device   to   use   the   hardware_handler   specified   by
                        /etc/multipath.conf.  If  the  SCSI  layer  has  not  attached a hardware
                        handler, multipath will continue to use its configured hardware handler.

                        The default is: yes

                        Important  Note:  Linux  kernel  4.3  or  newer  always  behaves  as   if
                        "retain_attached_hw_handler yes" was set.

       detect_prio      If  set  to  yes  ,  multipath  will try to detect if the device supports
                        SCSI-3  ALUA.  If  so,  the  device  will  automatically  use  the  sysfs
                        prioritizer   if   the   required   sysfs   attributes  access_state  and
                        preferred_path are supported, or the alua prioritizer if not. If  set  to
                        no , the prioritizer will be selected as usual.

                        The default is: yes

       detect_checker   if  set  to  yes  ,  multipath  will try to detect if the device supports
                        SCSI-3 ALUA. If so, the device will automatically use the tur checker. If
                        set to no , the checker will be selected as usual.

                        The default is: yes

       force_sync       If set to yes , multipathd will call the path checkers in sync mode only.
                        This means that only one checker will run at a time.  This is  useful  in
                        the  case  where  many  multipathd  checkers  running  in parallel causes
                        significant CPU pressure.

                        The default is: no

       strict_timing    If set to yes , multipathd will start  a  new  path  checker  loop  after
                        exactly  one  second,  so  that  each  path  check  will occur at exactly
                        polling_interval seconds. On busy systems path checks might  take  longer
                        than one second; here the missing ticks will be accounted for on the next
                        round.  A warning will  be  printed  if  path  checks  take  longer  than
                        polling_interval seconds.

                        The default is: no

       deferred_remove  If set to yes , multipathd will do a deferred remove instead of a regular
                        remove when the last path device has been deleted.  This  means  that  if
                        the multipath device is still in use, it will be freed when the last user
                        closes it.  If path is added to the multipath device before the last user
                        closes it, the deferred remove will be canceled.

                        The default is: no

       partition_delimiter
                        This  parameter  controls  how  multipath  chooses the names of partition
                        devices of multipath maps if a multipath map is renamed (e.g.  if  a  map
                        alias  is  added  or changed). If this parameter is set to a string other
                        than "/UNSET/" (even the empty string),  multipath  inserts  that  string
                        between  device  name  and  partition  number  to construct the partition
                        device name.  Otherwise (i.e. if this parameter is unset or has the value
                        "/UNSET/"),  the behavior depends on the map name: if it ends in a digit,
                        a "p" is inserted between  name  and  partition  number;  otherwise,  the
                        partition  number  is  simply appended.  Distributions may use a non-null
                        default value for this option; in this case, the  user  must  set  it  to
                        "/UNSET/"  to  obtain  the original <unset> behavior. Use multipath -T to
                        check the current settings.

                        The default is: <unset>

       config_dir       (Deprecated) This option is not supported anymore, and will be ignored.

                        The compiled-in value is: /etc/multipath/conf.d

       san_path_err_threshold
                        If set to a value greater than 0, multipathd will watch paths  and  check
                        how  many  times  a  path  has been failed due to errors.If the number of
                        failures on a particular path is greater than the san_path_err_threshold,
                        then  the  path will not reinstate till san_path_err_recovery_time. These
                        path failures should occur within a san_path_err_forget_rate  checks,  if
                        not  we  will  consider  the path is good enough to reinstate. See "Shaky
                        paths detection" below.

                        The default is: no

       san_path_err_forget_rate
                        If set to a value greater than 0, multipathd will check whether the  path
                        failures has exceeded  the san_path_err_threshold within this many checks
                        i.e san_path_err_forget_rate . If so we will not reinstate the path  till
                        san_path_err_recovery_time. See "Shaky paths detection" below.

                        The default is: no

       san_path_err_recovery_time
                        If  set  to  a  value greater than 0, multipathd will make sure that when
                        path   failures   has   exceeded   the   san_path_err_threshold    within
                        san_path_err_forget_rate then the path will be placed in failed state for
                        san_path_err_recovery_time duration.Once  san_path_err_recovery_time  has
                        timeout   we will reinstate the failed path .  san_path_err_recovery_time
                        value should be in secs.  See "Shaky paths detection" below.

                        The default is: no

       marginal_path_double_failed_time
                        One of the four parameters of supporting path check based  on  accounting
                        IO  error  such  as  intermittent  error. When a path failed event occurs
                        twice in marginal_path_double_failed_time seconds due to an IO error  and
                        all the other three parameters are set, multipathd will fail the path and
                        enqueue this path into a queue of which members  are  sent  a  couple  of
                        continuous direct reading asynchronous IOs at a fixed sample rate of 10HZ
                        to start IO error accounting process. See "Shaky paths detection" below.

                        The default is: no

       marginal_path_err_sample_time
                        One of the four parameters of supporting path check based  on  accounting
                        IO error such as intermittent error. If it is set to a value no less than
                        120,    when    a     path     fail     event     occurs     twice     in
                        marginal_path_double_failed_time  second  due  to an IO error, multipathd
                        will fail the path and enqueue this path into a queue  of  which  members
                        are  sent  a  couple  of  continuous direct reading asynchronous IOs at a
                        fixed sample rate of 10HZ to start the IO accounting process for the path
                        will  last for marginal_path_err_sample_time.  If the rate of IO error on
                        a particular path is greater than  the  marginal_path_err_rate_threshold,
                        then  the  path will not reinstate for marginal_path_err_recheck_gap_time
                        seconds   unless   there    is    only    one    active    path.    After
                        marginal_path_err_recheck_gap_time expires, the path will be requeued for
                        rechecking.  If  checking  result  is  good  enough,  the  path  will  be
                        reinstated. See "Shaky paths detection" below.

                        The default is: no

       marginal_path_err_rate_threshold
                        The  error  rate  threshold  as  a  permillage  (1/1000). One of the four
                        parameters of supporting path check based on accounting IO error such  as
                        intermittent  error.  Refer to marginal_path_err_sample_time. If the rate
                        of IO errors on a particular path is greater than  this  parameter,  then
                        the   path  will  not  reinstate  for  marginal_path_err_recheck_gap_time
                        seconds unless there is only one active path. See "Shaky paths detection"
                        below.

                        The default is: no

       marginal_path_err_recheck_gap_time
                        One  of  the four parameters of supporting path check based on accounting
                        IO     error     such     as     intermittent     error.     Refer     to
                        marginal_path_err_sample_time.  If  this  parameter  is set to a positive
                        value, the failed path of   which  the  IO  error  rate  is  larger  than
                        marginal_path_err_rate_threshold   will  be  kept  in  failed  state  for
                        marginal_path_err_recheck_gap_time             seconds.              When
                        marginal_path_err_recheck_gap_time  seconds  expires,  the  path  will be
                        requeued for checking. If checking result is good enough, the  path  will
                        be  reinstated,  or else it will keep failed. See "Shaky paths detection"
                        below.

                        The default is: no

       delay_watch_checks
                        (Deprecated)   This    option    is    deprecated,    and    mapped    to
                        san_path_err_forget_rate.   If  this is set to a value greater than 0 and
                        no san_path_err options are set, san_path_err_forget_rate will be set  to
                        the value of delay_watch_checks and san_path_err_threshold will be set to
                        1.  See the san_path_err_forget_rate and san_path_err_threshold  options,
                        and "Shaky paths detection" below for more information.

                        The default is: no

       delay_wait_checks
                        (Deprecated)    This    option    is    deprecated,    and    mapped   to
                        san_path_err_recovery_time.  If this is set to a value greater than 0 and
                        no  san_path_err  options are set, san_path_err_recovery_time will be set
                        to the value of delay_wait_checks times max_polling_interval.  This  will
                        give  approximately  the  same  wait time as delay_wait_checks previously
                        did.   Also,  san_path_err_threshold  will  be  set   to   1.   See   the
                        san_path_err_recovery_time and san_path_err_threshold options, and "Shaky
                        paths detection" below for more information.

                        The default is: no

       marginal_pathgroups
                        If set to off, the delay_*_checks,  marginal_path_*,  and  san_path_err_*
                        options will keep marginal, or "shaky", paths from being reinstated until
                        they have been monitored for some time. This can cause  situations  where
                        all non-marginal paths are down, and no paths are usable until multipathd
                        detects this and reinstates a marginal path. If the multipath  device  is
                        not configured to queue IO in this case, it can cause IO errors to occur,
                        even though there are marginal paths available.  However, if this  option
                        is  set to on, when one of the marginal path detecting methods determines
                        that a path is marginal, it will be reinstated and placed in  a  separate
                        pathgroup  that  will  only be used after all the non-marginal pathgroups
                        have been tried  first.  This  prevents  the  possibility  of  IO  errors
                        occurring  while marginal paths are still usable. After the path has been
                        monitored for the configured time, and is declared healthy,  it  will  be
                        returned  to  its  normal  pathgroup.   If  this  option  is set to fpin,
                        multipathd will receive fpin notifications, set path states to "marginal"
                        accordingly,  and regroup paths as described for on. This option can't be
                        used in combination with other options for "Shaky  path  detection"  (see
                        below).   Note:   If  this  is  set  to  fpin,  the  marginal_path_*  and
                        san_path_err_* options are implicitly set to no. Also, this option cannot
                        be  switched  either  to  or  from  fpin  on  a  multipathd  reconfigure.
                        multipathd must be restarted for the change to take effect.   See  "Shaky
                        paths detection" below for more information.

                        The default is: off

       find_multipaths  This  option  controls  whether  multipath  and  multipathd try to create
                        multipath maps over non-blacklisted devices they encounter. This  matters
                        a)  when  a  device  is  encountered  by  multipath  -u  during udev rule
                        processing (a device is blocked from further processing by higher  layers
                        -  such  as LVM - if and only if it´s considered a valid multipath device
                        path), and b) when multipathd detects a new device. The following  values
                        are possible:

                        strict    Both  multipath  and  multipathd  treat  only  such  devices as
                                  multipath devices which have  been  part  of  a  multipath  map
                                  previously,  and  which are therefore listed in the wwids_file.
                                  Users can manually set up multipath maps using  the  multipathd
                                  add map command. Once set up manually, the map is remembered in
                                  the wwids file and will be set up automatically in the future.

                        off       Multipath behaves like strict. Multipathd behaves like  greedy.
                                  no or 0 is accepted as an alias for off.

                        on        Both  multipathd  and  multipath  treat  a  device as multipath
                                  device if the conditions for strict are met, or if at least two
                                  non-blacklisted  paths  with  the same WWID have been detected.
                                  yes or 1 is accepted as an alias for on.

                        greedy    Both  multipathd  and  multipath  treat  every  non-blacklisted
                                  device as multipath device path.

                        smart     This differs from find_multipaths yes only in the way it treats
                                  new devices for which only one path has been detected yet. When
                                  such a device is first encountered in udev rules, it is treated
                                  as a multipath  device.  multipathd  waits  whether  additional
                                  paths with the same WWID appears. If that happens, it sets up a
                                  multipath map. If it doesn´t happen until a timeout expires, or
                                  if  setting up the map fails, a new uevent is triggered for the
                                  device; at second encounter in the udev rules, the device  will
                                  be  treated  as  non-multipath  and  passed on to upper layers.
                                  Note: this may cause delays during device  detection  if  there
                                  are single-path devices which aren´t blacklisted.

                        The default is: strict

       find_multipaths_timeout
                        Timeout,  in  seconds,  to  wait for additional paths after detecting the
                        first one, if find_multipaths "smart" (see above) is set. If the value is
                        positive,  this  timeout is used for all unknown, non-blacklisted devices
                        encountered. If the value is negative (recommended), it's only applied to
                        "known"  devices that have an entry in multipath's hardware table, either
                        in the built-in table or in a device section; other  ("unknown")  devices
                        will  use a timeout of only 1 second to avoid booting delays. The value 0
                        means "use the built-in default". If find_multipath  has  a  value  other
                        than smart, this option has no effect.

                        The default is: -10 (10s for known and 1s for unknown hardware)

       uxsock_timeout   CLI  receive  timeout  in  milliseconds.  For larger systems CLI commands
                        might timeout before the multipathd lock is released and the CLI  command
                        can  be  processed.  This  will  result in errors like "timeout receiving
                        packet" to  be  returned  from  CLI  commands.   In  these  cases  it  is
                        recommended to increase the CLI timeout to avoid those issues.

                        The default is: 4000

       retrigger_tries  Sets the number of times multipathd will try to retrigger a uevent to get
                        the WWID.

                        The default is: 3

       retrigger_delay  Sets the amount of time, in seconds, to wait between retriggers.

                        The default is: 10

       missing_uev_wait_timeout
                        Controls how many seconds multipathd will wait,  after  a  new  multipath
                        device  is  created,  to receive a change event from udev for the device,
                        before automatically enabling device  reloads.  Usually  multipathd  will
                        delay  reloads  on  a  device  until it receives a change uevent from the
                        initial table load.

                        The default is: 30

       skip_kpartx      If set to yes , kpartx will not automatically create  partitions  on  the
                        device.

                        The default is: no

       disable_changed_wwids
                        (Deprecated) This option is not supported anymore, and will be ignored.

       remove_retries   This  sets  how  may times multipath will retry removing a device that is
                        in-use.  Between each attempt, multipath will sleep 1 second.

                        The default is: 0

       max_sectors_kb   Sets the max_sectors_kb device parameter on some  path  devices  and  the
                        multipath  device  to  the specified value. max_sectors_kb is the largest
                        I/O size, in units of 1024 bytes, that the kernel allows for a single I/O
                        request.  For  hardware devices like SCSI disks, this value is limited by
                        the capabilities of the hardware.  It is crucial  that  the  value  of  a
                        multipath  map  is never higher than the minimum value of of all its path
                        devices. This is ensured by the kernel when a multipath  map  is  loaded,
                        but manipulating the values of a map or either of its paths while the map
                        is live can cause race conditions and I/O errors. Therefore this value is
                        only  enforced  by  multipathd  when a multipath map is first created, or
                        when a path device is added to a map. In both cases, race conditions  are
                        avoided by the kernel.

                        Setting max_sectors_kb does not guarantee that all path devices will have
                        this value set. It is not an error if the  value  of  a  path  device  is
                        higher than that of the containing multipath map. It is also not an error
                        if  the  actual  limit  of  a  map   is   lower   than   the   value   in
                        /etc/multipath.conf.  This  can  happen  if the hardware limits of one or
                        more path devices are lower than the configured value.

                        Normally the kernel and its device drivers take care of the  maximum  I/O
                        size, and administrators do not need to bother about max_sectors_kb.  But
                        some hardware devices may report incorrect  I/O  size  limits,  or  other
                        components  in  the  environment (e.g. the fabric) may impose constraints
                        that the kernel cannot detect. In such cases setting max_sectors_kb makes
                        sense.  It  should be set when maps are first created, and not be changed
                        thereafter.  If the setting must be changed for a live map, set the value
                        in  /etc/multipath.conf,  run  multipathd reconfigure, and use multipathd
                        del path <path> and multipathd add path <path> to delete and  re-add  the
                        same path device.

                        The default is: <undefined>

       ghost_delay      Sets  the  number  of  seconds  that multipath will wait after creating a
                        device with only ghost paths before marking it ready for use in  systemd.
                        This  gives the active paths time to appear before the multipath runs the
                        hardware handler to switch the ghost paths to active ones.  Setting  this
                        to  0  or  no  makes  multipath immediately mark a device with only ghost
                        paths as ready.

                        The default is: no

       auto_resize      Controls when multipathd will automatically resize  a  multipath  device.
                        If  set  to  never,  multipath devices must always be manually resized by
                        either running multipathd resize map <name>.  If set to  grow_only,  when
                        multipathd  detects that all of a multipath device's paths have increased
                        in size, it will automatically grow the multipath device to the new size.
                        If  set  to  grow_shrink,  multipathd  will also automatically shrink the
                        device once it detects all of its paths have decreased in size.

                        The default is: never

       enable_foreign   Enables or disables foreign  libraries  (see  section  FOREIGN  MULTIPATH
                        SUPPORT  below). The value is a regular expression; foreign libraries are
                        loaded if their name (e.g. "nvme") matches the expression. By default, no
                        foreign  libraries  are enabled. Set this to "nvme" to enable NVMe native
                        multipath support, or ".*" to enable all foreign libraries.

                        The default is: "NONE"

       recheck_wwid     If set to yes, when a failed path is restored, its wwid is rechecked.  If
                        the  wwid  has  changed,  the  path is removed from the current multipath
                        device, and re-added as a new path. Multipathd will also recheck a path's
                        wwid  if it is manually re-added. This option only works for SCSI devices
                        that are configured to use the default uid_attribute, ID_SERIAL, or sysfs
                        for getting their wwid.

                        The default is: no

blacklist and blacklist_exceptions sections

       The  blacklist section is used to exclude specific devices from the multipath topology. It
       is most commonly used to exclude local disks or non-disk devices (such  as  LUNs  for  the
       storage array controller) from being handled by multipath-tools.

       In  the  blacklist  and  blacklist_exceptions  sections,  starting  a quoted value with an
       exclamation mark "!" will invert the matching of the rest of the regular  expression.  For
       instance,  "!^sd[a-z]"  will  match  all  values  that  do  not  start with "sd[a-z]". The
       exclamation mark can be escaped "\!" to match a literal  !  at  the  start  of  a  regular
       expression. Note: The exclamation mark must be inside quotes, otherwise it will be treated
       as starting a comment.

       The blacklist_exceptions section is used to revert the actions of the  blacklist  section.
       This  allows  one  to  selectively  include  ("whitelist") devices which would normally be
       excluded via the blacklist section. A common usage is to blacklist  "everything"  using  a
       catch-all  regular  expression, and create specific blacklist_exceptions entries for those
       devices that should be handled by multipath-tools.

       The following keywords are recognized in both sections.  The  defaults  are  empty  unless
       explicitly stated.

       devnode          Regular expression matching the device nodes to be excluded/included.

                        The  default  blacklist  consists  of  the  regular  expression "!^(sd[a-
                        z]|dasd[a-z]|nvme[0-9])". This causes all device types other  than  scsi,
                        dasd, and nvme to be excluded from multipath handling by default.

       wwid             Regular  expression  for  the  World  Wide  Identifier  of a device to be
                        excluded/included.

       device           Subsection for the device description.  This  subsection  recognizes  the
                        vendor  and  product  keywords.  Both are regular expressions. For a full
                        description of these keywords please see the devices section description.

       property         Regular expression for an udev property. All devices that  have  matching
                        udev  properties will be excluded/included.  The handling of the property
                        keyword is special, because devices must have at  least  one  whitelisted
                        udev  property; otherwise they're treated as blacklisted, and the message
                        "blacklisted, udev property missing" is displayed in the logs.

                        Note: The behavior of this option has changed  in  multipath-tools  0.8.2
                        compared  to  previous  versions.   Blacklisting by missing properties is
                        only  applied  to  devices  which  do  have  the  property  specified  by
                        uid_attribute  (e.g.  ID_SERIAL) set. Previously, it was applied to every
                        device, possibly causing devices to be blacklisted because  of  temporary
                        I/O error conditions.

                        The  default  blacklist exception is: (SCSI_IDENT_|ID_WWN), causing well-
                        behaved SCSI devices and devices that provide a WWN (World  Wide  Number)
                        to be included, and all others to be excluded.

       protocol         Regular expression for the protocol of a device to be excluded/included.

                        The  protocol  strings  that multipath recognizes are scsi:fcp, scsi:spi,
                        scsi:ssa, scsi:sbp, scsi:srp, scsi:iscsi, scsi:sas,  scsi:adt,  scsi:ata,
                        scsi:unspec,   nvme:pcie,   nvme:rdma,   nvme:fc,   nvme:tcp,  nvme:loop,
                        nvme:apple-nvme, nvme:unspec, ccw, cciss, and undef.  The protocol that a
                        path  is  using can be viewed by running multipathd show paths format "%d
                        %P"

       For every device, these 5 blacklist criteria are evaluated in the  order  "property,  dev‐
       node,  device,  protocol, wwid". If a device turns out to be blacklisted by any criterion,
       it's excluded from handling  by  multipathd,  and  the  later  criteria  aren't  evaluated
       anymore. For each criterion, the whitelist takes precedence over the blacklist if a device
       matches both.

       Note:  Besides  the  blacklist  and  whitelist,  other  configuration  options   such   as
       find_multipaths  have  an impact on whether or not a given device is handled by multipath-
       tools.

multipaths section

       The multipaths section allows setting attributes of multipath maps.  The  attributes  that
       are  set  via  the  multipaths  section  (see  list  below) take precedence over all other
       configuration settings, including those from the overrides section.

       The only recognized attribute for the multipaths section is the multipath  subsection.  If
       there  are  multiple  multipath  subsections  matching a given WWID, the contents of these
       sections are merged, and settings from later entries take precedence.

       The multipath subsection recognizes the following attributes:

       wwid             (Mandatory) World Wide Identifier. Detected multipath  maps  are  matched
                        against  this  attribute.   Note  that,  unlike the wwid attribute in the
                        blacklist section, this is not a regular expression or a substring; WWIDs
                        must match exactly inside the multipaths section.

       alias            Symbolic  name  for  the  multipath  map. This takes precedence over a an
                        entry for the same WWID in the bindings_file.

       The following attributes are optional; if not set the default values are  taken  from  the
       overrides, devices, or defaults section:

              path_grouping_policy
              path_selector
              prio
              prio_args
              failback
              rr_weight
              no_path_retry
              rr_min_io
              rr_min_io_rq
              flush_on_last_del
              features
              reservation_key
              user_friendly_names
              deferred_remove
              san_path_err_threshold
              san_path_err_forget_rate
              san_path_err_recovery_time
              marginal_path_err_sample_time
              marginal_path_err_rate_threshold
              marginal_path_err_recheck_gap_time
              marginal_path_double_failed_time
              delay_watch_checks
              delay_wait_checks
              skip_kpartx
              max_sectors_kb
              ghost_delay

devices section

       Important:
           The  built-in  hardware  device  table of multipath-tools is created by members of the
           Linux community in the hope that it will be useful.  The existence of an entry  for  a
           given  storage  product  in  the hardware table does not imply that the product vendor
           supports, or has tested, the product with multipath-tools in any way.  Always  consult
           the vendor's official documentation for support-related information.

       multipath-tools  have  a  built-in device table with reasonable defaults for more than 100
       known multipath-capable storage devices. The devices section can be used to override these
       settings. If there are multiple matches for a given device, the attributes of all matching
       entries are applied to it.  If an  attribute  is  specified  in  several  matching  device
       subsections,  later  entries  take precedence. Thus, entries in files under config_dir (in
       reverse  alphabetical  order)  have  the  highest  precedence,  followed  by  entries   in
       /etc/multipath.conf;  the  built-in  hardware  table  has  the lowest precedence. Inside a
       configuration file, later entries have higher precedence than earlier ones.

       The only recognized attribute for the devices section is the  device  subsection.  Devices
       detected  in  the system are matched against the device entries using the vendor, product,
       and revision fields, which are all POSIX Extended regular expressions (see regex(7)).

       The vendor, product, and revision fields that multipath or multipathd detect  for  devices
       in a system depend on the device type. For SCSI devices, they correspond to the respective
       fields of the SCSI INQUIRY page. In general, the command 'multipathd show paths format "%d
       %s"' command can be used to see the detected properties for all devices in the system.

       The device subsection recognizes the following attributes:

       vendor           (Mandatory) Regular expression to match the vendor name.

       product          (Mandatory) Regular expression to match the product name.

       revision         Regular  expression  to match the product revision. If not specified, any
                        revision matches.

       product_blacklist
                        Products with the given vendor matching this string are blacklisted. This
                        is  equivalent to a device entry in the blacklist section with the vendor
                        attribute set to this entry's vendor, and the product  attribute  set  to
                        the value of product_blacklist.

       alias_prefix     The  user_friendly_names  prefix  to use for this device type, instead of
                        the default "mpath".

       vpd_vendor       The  vendor  specific  vpd  page  information,   using   the   vpd   page
                        abbreviation.   The  vpd page abbreviation can be found by running sg_vpd
                        -e. multipathd will  use  this  information  to  gather  device  specific
                        information that can be displayed with the %g wildcard for the multipathd
                        show maps format and multipathd show  paths  format  commands.  Currently
                        only the hp3par vpd page is supported.

       hardware_handler The hardware handler to use for this device type.  The following hardware
                        handler are implemented:

                        1 emc       (Hardware-dependent) Hardware handler for DGC class arrays as
                                    CLARiiON  CX/AX  and  EMC  VNX  families with Failover Mode 1
                                    (Passive Not Ready(PNR)).

                        1 rdac      (Hardware-dependent) Hardware handler for  LSI/Engenio/NetApp
                                    RDAC  class  as  NetApp  SANtricity E/EF Series and rebranded
                                    arrays, with "Linux DM-MP (Kernel 3.9 or earlier)" option.

                        1 hp_sw     (Hardware-dependent) Hardware handler for HP/COMPAQ/DEC HSG80
                                    and MSA/HSV arrays with Active/Standby mode exclusively.

                        1 alua      (Hardware-dependent)   Hardware   handler   for  SCSI-3  ALUA
                                    compatible arrays.

                        1 ana       (Hardware-dependent) Hardware handler for NVMe ANA compatible
                                    arrays.

                        The default is: <unset>

                        Important Note: Linux kernels 4.3 and newer automatically attach a device
                        handler to known devices (which includes all  devices  supporting  SCSI-3
                        ALUA)   and   disallow   changing   the   handler   afterwards.   Setting
                        hardware_handler for such devices on these kernels has no effect.

       The following attributes are optional; if not set the default values are  taken  from  the
       defaults section:

              path_grouping_policy
              uid_attribute
              path_selector
              path_checker
              prio
              prio_args
              features
              failback
              rr_weight
              no_path_retry
              rr_min_io
              rr_min_io_rq
              fast_io_fail_tmo
              dev_loss_tmo
              eh_deadline
              flush_on_last_del
              user_friendly_names
              retain_attached_hw_handler
              detect_prio
              detect_checker
              deferred_remove
              san_path_err_threshold
              san_path_err_forget_rate
              san_path_err_recovery_time
              marginal_path_err_sample_time
              marginal_path_err_rate_threshold
              marginal_path_err_recheck_gap_time
              marginal_path_double_failed_time
              delay_watch_checks
              delay_wait_checks
              skip_kpartx
              max_sectors_kb
              ghost_delay
              all_tg_pt

overrides section

       The  overrides section recognizes the following optional attributes; if not set the values
       are taken from the devices or defaults sections:

              path_grouping_policy
              uid_attribute
              path_selector
              path_checker
              alias_prefix
              features
              prio
              prio_args
              failback
              rr_weight
              no_path_retry
              rr_min_io
              rr_min_io_rq
              flush_on_last_del
              fast_io_fail_tmo
              dev_loss_tmo
              eh_deadline
              user_friendly_names
              retain_attached_hw_handler
              detect_prio
              detect_checker
              deferred_remove
              san_path_err_threshold
              san_path_err_forget_rate
              san_path_err_recovery_time
              marginal_path_err_sample_time
              marginal_path_err_rate_threshold
              marginal_path_err_recheck_gap_time
              marginal_path_double_failed_time
              delay_watch_checks
              delay_wait_checks
              skip_kpartx
              max_sectors_kb
              ghost_delay
              all_tg_pt

       The overrides section also recognizes the optional protocol subsection,  and  can  contain
       multiple  protocol  subsections.  Path devices are matched against the protocol subsection
       using the mandatory type attribute.  Attributes in a  matching  protocol  subsection  take
       precedence  over  attributes  in  the rest of the overrides section. If there are multiple
       matching protocol subsections, later entries take precedence.

       protocol subsection
              The protocol subsection recognizes the following mandatory attribute:

              type   The protocol string of the path device. The possible  values  are  scsi:fcp,
                     scsi:spi,  scsi:ssa,  scsi:sbp,  scsi:srp,  scsi:iscsi,  scsi:sas, scsi:adt,
                     scsi:ata, scsi:unspec, nvme:pcie, nvme:rdma, nvme:fc,  nvme:tcp,  nvme:loop,
                     nvme:apple-nvme,  nvme:unspec,  ccw, cciss, and undef. This is not a regular
                     expression. the path device protocol string must match exactly. The protocol
                     that  a  path is using can be viewed by running multipathd show paths format
                     "%d %P"

              The following attributes are optional; if not set, the  default  values  are  taken
              from the overrides, devices, or defaults section:

                     fast_io_fail_tmo
                     dev_loss_tmo
                     eh_deadline

WWID generation

       Multipath  uses  a World Wide Identification (WWID) to determine which paths belong to the
       same device. Each path presenting the same WWID is assumed to point to the same device.

       The WWID is generated by four methods (in the order of preference):

       uid_attrs        The WWID is derived from udev attributes  by  matching  the  device  node
                        name; cf uid_attrs above.

       uid_attribute    Use the value of the specified udev attribute; cf uid_attribute above.

       sysfs            Try  to determine the WWID from sysfs attributes.  For SCSI devices, this
                        means reading the Vital Product Data (VPD) page  "Device  Identification"
                        (0x83).

       The  default  settings (using udev and uid_attribute configured from the built-in hardware
       table) should work fine in most scenarios. Users who want to enable  uevent  merging  must
       set uid_attrs.

Shaky paths detection

       A  common  problem  in  SAN  setups  is  the  occurrence of intermittent errors: a path is
       unreachable, then reachable again for a short time, disappears again, and so  forth.  This
       happens  typically  on  unstable  interconnects.  It  is  undesirable to switch pathgroups
       unnecessarily on such frequent, unreliable events.  multipathd  supports  three  different
       methods for detecting this situation and dealing with it. All methods share the same basic
       mode of operation: If a path is found to be "shaky" or "flipping", and appears  to  be  in
       healthy  status, it is not reinstated (put back to use) immediately. Instead, it is placed
       in the "delayed" state and watched for some time, and only reinstated if the healthy state
       appears  to be stable.  If the marginal_pathgroups option is set, the path will reinstated
       immediately, but placed in a special pathgroup for  marginal  paths.  Marginal  pathgroups
       will  not  be  used  until all other pathgroups have been tried. At the time when the path
       would normally be reinstated, it will be returned to its normal pathgroup.  The  logic  of
       determining "shaky" condition, as well as the logic when to reinstate, differs between the
       three methods.

       "delay_checks" failure tracking
               (Deprecated) This method is deprecated and mapped to  the  "san_path_err"  method.
               See   the   delay_watch_checks   and  delay_wait_checks  options  above  for  more
               information.

       "marginal_path" failure tracking
               If   a   second   failure   event    (good->bad    transition)    occurs    within
               marginal_path_double_failed_time   seconds   after   a   failure,   high-frequency
               monitoring is started for the affected path: I/O is sent  at  a  rate  of  10  per
               second.  This  is  done  for  marginal_path_err_sample_time  seconds.  During this
               period, the  path  is  not  reinstated.  If  the  rate  of  errors  remains  below
               marginal_path_err_rate_threshold   during  the  monitoring  period,  the  path  is
               reinstated.    Otherwise,    it    is     kept     in     failed     state     for
               marginal_path_err_recheck_gap_time,  and  after  that,  it is monitored again. For
               this method, time intervals are measured in seconds.

       "san_path_err" failure tracking
               multipathd counts path failures for each path. Once the number of failures exceeds
               the  value  given  by  san_path_err_threshold,  the  path  is  not  reinstated for
               san_path_err_recovery_time seconds. While counting failures, multipathd  "forgets"
               one  past  failure  every  "san_path_err_forget_rate"  ticks; thus if errors don't
               occur more often then once in the forget rate interval, the failure count  doesn't
               increase  and  the  threshold  is  never  reached. Ticks are the time between path
               checks by multipathd, which is variable and controlled by the polling_interval and
               max_polling_interval parameters.

               This  algorithm is superseded by the "marginal_path" failure tracking, but remains
               supported for backward compatibility.

       "FPIN" failure tracking
               Fibre channel fabrics can notify hosts about fabric-level issues such as integrity
               failures  or  congestion  with  so-called  Fabric Performance Impact Notifications
               (FPINs).On receiving the fpin notifications through ELS multipathd will  move  the
               affected path and port states to marginal.

       See  the  documentation  of  the  individual  options  above  for details.  It is strongly
       discouraged to use more than one of these methods for any given multipath map, because the
       two  concurrent  methods may interact in unpredictable ways. If the "marginal_path" method
       is active, the "san_path_err" parameters are implicitly set to 0.

FOREIGN MULTIPATH SUPPORT

       multipath  and  multipathd  can  load  "foreign"  libraries  to  add  support  for   other
       multipathing  technologies  besides  the  Linux  device mapper.  Currently this support is
       limited to printing detected information about multipath setup. In  topology  output,  the
       names  of  foreign maps are prefixed by the foreign library name in square brackets, as in
       this example:

       # multipath -ll
       uuid.fedcba98-3579-4567-8765-123456789abc [nvme]:nvme4n9 NVMe,Some NVMe controller,FFFFFFFF
       size=167772160 features='n/a' hwhandler='ANA' wp=rw
       |-+- policy='n/a' prio=50 status=optimized
       | `- 4:38:1    nvme4c38n1 0:0     n/a   optimized    live
       `-+- policy='n/a' prio=50 status=optimized
         `- 4:39:1    nvme4c39n1 0:0     n/a   optimized    live

       The "nvme" foreign library provides support for NVMe native multipathing in the kernel. It
       is part of the standard multipath package.

KNOWN ISSUES

       The  usage  of  queue_if_no_path  option  can lead to D state processes being hung and not
       killable in situations where all the paths to the LUN go offline. It is advisable  to  use
       the no_path_retry option instead.

       The  use of queue_if_no_path or no_path_retry might lead to a deadlock if the dev_loss_tmo
       setting results in a device being removed while I/O is still queued. The multipath  daemon
       will  update  the  dev_loss_tmo  setting accordingly to avoid this deadlock. Hence if both
       values  are  specified  the  order  of  precedence  is  no_path_retry,   queue_if_no_path,
       dev_loss_tmo.

SEE ALSO

       udev(8), dmsetup(8), multipath(8), multipathc(8), multipathd(8).

AUTHORS

       multipath-tools  was developed by Christophe Varoqui, <christophe.varoqui@opensvc.com> and
       others.