Provided by: mdadm_3.2.5-5ubuntu4.4_amd64 bug


       mdmon - monitor MD external metadata arrays


       mdmon [--all] [--takeover] [--offroot] CONTAINER


       The  2.6.27  kernel  brings  the  ability  to  support external metadata arrays.  External
       metadata implies that user space handles  all  updates  to  the  metadata.   The  kernel's
       responsibility  is to notify user space when a "metadata event" occurs, like disk failures
       and clean-to-dirty transitions.  The kernel, in important cases, waits for user  space  to
       take action on these notifications.


   Metadata updates:
       To  service metadata update requests a daemon, mdmon, is introduced.  Mdmon is tasked with
       polling the sysfs namespace looking for changes in array_state, sync_action, and per  disk
       state  attributes.  When a change is detected it calls a per metadata type handler to make
       modifications to the metadata.  The following actions are taken:

              array_state - inactive
                     Clear the dirty bit for the volume and let the array be stopped

              array_state - write pending
                     Set the dirty bit for the array and then set array_state to active.   Writes
                     are blocked until userspace writes active.

              array_state - active-idle
                     The  safe mode timer has expired so set array state to clean to block writes
                     to the array

              array_state - clean
                     Clear the dirty bit for the volume

              array_state - read-only
                     This is the initial state that all arrays start at.  mdmon takes one of  the
                     three actions:

                     1/     Transition  the array to read-auto keeping the dirty bit clear if the
                            metadata handler determines that the array does not need resyncing or
                            other modification

                     2/     Transition  the  array to active if the metadata handler determines a
                            resync or some other manipulation is necessary

                     3/     Leave the  array  read-only  if  the  volume  is  marked  to  not  be
                            monitored;  for  example,  the  metadata  version  has  been  set  to
                            "external:-dev/md127" instead of "external:/dev/md127"

              sync_action - resync-to-idle
                     Notify the metadata handler that a resync may have completed.  If  a  resync
                     process  is idled before it completes this event allows the metadata handler
                     to checkpoint resync.

              sync_action - recover-to-idle
                     A spare may have completed rebuilding so tell the metadata handler about the
                     state of each disk.  This is the metadata handler's opportunity to clear any
                     "out-of-sync" bits and clear the volume's degraded status.   If  a  recovery
                     process  is idled before it completes this event allows the metadata handler
                     to checkpoint recovery.

              <disk>/state - faulty
                     A disk failure kicks off a series of events.   First,  notify  the  metadata
                     handler  that  a  disk  has  failed,  and then notify the kernel that it can
                     unblock writes that were dependent  on  this  disk.   After  unblocking  the
                     kernel  this  disk is set to be removed+ from the member array.  Finally the
                     disk is marked failed in all other member arrays in the container.

                     + Note This behavior differs slightly from native MD arrays where removal is
                     reserved  for  a  mdadm  --remove  event.  In the external metadata case the
                     container holds the final reference on a block device and a  mdadm  --remove
                     <container> <victim> call is still required.

       External  metadata  formats,  like DDF, differ from the native MD metadata formats in that
       they define a set of disks and a series of sub-arrays within those disks.  MD metadata  in
       comparison  defines  a  1:1  relationship between a set of block devices and a raid array.
       For example to create 2 arrays at different raid levels on  a  single  set  of  disks,  MD
       metadata  requires  the  disks  be  partitioned  and then each array can be created with a
       subset of those partitions.  The supported external  formats  perform  this  disk  carving

       Container devices simply hold references to all member disks and allow tools like mdmon to
       determine which active arrays belong to which container.  Some array  management  commands
       like  disk  removal  and  disk add are now only valid at the container level.  Attempts to
       perform these actions on member arrays are blocked with error messages like:

              "mdadm: Cannot remove disks from a ┬┤member┬┤ array, perform this  operation  on  the
              parent container"

       Containers   are   identified   in   /proc/mdstat   with   a   metadata   version   string
       "external:<metadata  name>".  Member  devices  are  identified  by   "external:/<container
       device>/<member  index>", or "external:-<container device>/<member index>" if the array is
       to remain readonly.


              The container device to monitor.  It can be a full path like /dev/md/container,  or
              a simple md device name like md127.

              This  instructs mdmon to replace any active mdmon which is currently monitoring the
              array.  This is primarily used late in the boot process to replace any mdmon  which
              was  started from an initramfs before the root filesystem was mounted.  This avoids
              holding a reference on that initramfs indefinitely and ensures  that  the  pid  and
              sock files used to communicate with mdmon are in a standard place.

       --all  This tells mdmon to find any active containers and start monitoring each of them if
              appropriate.  This is normally used with --takeover late in the boot  sequence.   A
              separate mdmon process is started for each container as the --all argument is over-
              written with the name of the container.  To allow for containers with names  longer
              than 5 characters, this argument can be arbitrarily extended, e.g. to --all-active-

              Set  first  character  of  argv[0]  to  @  to  indicate  mdmon  was  launched  from
              initrd/initramfs  and  should  not  be  shutdown  by systemd as part of the regular
              shutdown process. This option is normally only used by  the  system's  initscripts.
              Please see here for more details on how systemd handled argv[0]:


       Note  that  mdmon is automatically started by mdadm when needed and so does not need to be
       considered when working with RAID arrays.  The only times it is run other than by mdadm is
       when the boot scripts need to restart it after mounting the new root filesystem.


       As  mdmon  needs  to be running whenever any filesystem on the monitored device is mounted
       there are special considerations when  the  root  filesystem  is  mounted  from  an  mdmon
       monitored  device.  Note that in general mdmon is needed even if the filesystem is mounted
       read-only as some filesystems can still write to the device in  those  circumstances,  for
       example to replay a journal after an unclean shutdown.

       When the array is assembled by the initramfs code, mdadm will automatically start mdmon as
       required.  This means that mdmon must be installed on the initramfs and there  must  be  a
       writable  filesystem  (typically  tmpfs)  in which mdmon can create a .pid and .sock file.
       The particular filesystem to use is given  to  mdmon  at  compile  time  and  defaults  to

       This filesystem must persist through to shutdown time.

       After the final root filesystem has be instantiated (usually with pivot_root) mdmon should
       be run with --all --takeover so that the mdmon running from the initramfs can be  replaced
       with  one  running  in  the  main  root,  and  so  the memory used by the initramfs can be

       At shutdown time, mdmon should not be killed along with other processes.  Also as it holds
       a file (socket actually) open in /dev (by default) it will not be possible to unmount /dev
       if it is a separate filesystem.


         mdmon --all-active-arrays --takeover
       Any mdmon which is currently running is killed and a new instance is started.  This should
       be  run  during  in  the boot sequence if an initramfs was used, so that any mdmon running
       from the initramfs will not hold the initramfs active.


       mdadm(8), md(4).