Ubuntu Manpage: mdmon - monitor MD external metadata arrays

NAME

       mdmon - monitor MD external metadata arrays

SYNOPSIS

       mdmon [--all] [--takeover] [--foreground] CONTAINER

OVERVIEW

       The 2.6.27 kernel brings the ability to support external metadata arrays.  External metadata implies that
       user space handles all updates to the metadata.  The kernel's responsibility is to notify user space when
       a  "metadata  event" occurs, like disk failures and clean-to-dirty transitions.  The kernel, in important
       cases, waits for user space to take action on these notifications.

DESCRIPTION

Metadata updates:
To service metadata update requests a daemon, mdmon, is introduced. Mdmon is tasked with polling the
sysfs namespace looking for changes in array_state, sync_action, and per disk state attributes. When a
change is detected it calls a per metadata type handler to make modifications to the metadata. The
following actions are taken:

array_state - inactive
Clear the dirty bit for the volume and let the array be stopped

array_state - write pending
Set the dirty bit for the array and then set array_state to active. Writes are blocked
until userspace writes active.

array_state - active-idle
The safe mode timer has expired so set array state to clean to block writes to the array

array_state - clean
Clear the dirty bit for the volume

array_state - read-only
This is the initial state that all arrays start at. mdmon takes one of the three actions:

1/ Transition the array to read-auto keeping the dirty bit clear if the metadata
handler determines that the array does not need resyncing or other modification

2/ Transition the array to active if the metadata handler determines a resync or some
other manipulation is necessary

3/ Leave the array read-only if the volume is marked to not be monitored; for example,
the metadata version has been set to "external:-dev/md127" instead of
"external:/dev/md127"

sync_action - resync-to-idle
Notify the metadata handler that a resync may have completed. If a resync process is idled
before it completes this event allows the metadata handler to checkpoint resync.

sync_action - recover-to-idle
A spare may have completed rebuilding so tell the metadata handler about the state of each
disk. This is the metadata handler's opportunity to clear any "out-of-sync" bits and clear
the volume's degraded status. If a recovery process is idled before it completes this
event allows the metadata handler to checkpoint recovery.

<disk>/state - faulty
A disk failure kicks off a series of events. First, notify the metadata handler that a
disk has failed, and then notify the kernel that it can unblock writes that were dependent
on this disk. After unblocking the kernel this disk is set to be removed+ from the member
array. Finally the disk is marked failed in all other member arrays in the container.

+ Note This behavior differs slightly from native MD arrays where removal is reserved for a
mdadm --remove event. In the external metadata case the container holds the final
reference on a block device and a mdadm --remove <container> <victim> call is still
required.

Containers:
External metadata formats, like DDF, differ from the native MD metadata formats in that they define a set
of disks and a series of sub-arrays within those disks. MD metadata in comparison defines a 1:1
relationship between a set of block devices and a RAID array. For example to create 2 arrays at
different RAID levels on a single set of disks, MD metadata requires the disks be partitioned and then
each array can be created with a subset of those partitions. The supported external formats perform this
disk carving internally.

Container devices simply hold references to all member disks and allow tools like mdmon to determine
which active arrays belong to which container. Some array management commands like disk removal and disk
add are now only valid at the container level. Attempts to perform these actions on member arrays are
blocked with error messages like:

"mdadm: Cannot remove disks from a ´member´ array, perform this operation on the parent container"

Containers are identified in /proc/mdstat with a metadata version string "external:<metadata name>".
Member devices are identified by "external:/<container device>/<member index>", or "external:-<container
device>/<member index>" if the array is to remain readonly.

OPTIONS

CONTAINER
The container device to monitor. It can be a full path like /dev/md/container, or a simple md
device name like md127.

--foreground
Normally, mdmon will fork and continue in the background. Adding this option will skip that step
and run mdmon in the foreground.

--takeover
This instructs mdmon to replace any active mdmon which is currently monitoring the array. This is
primarily used late in the boot process to replace any mdmon which was started from an initramfs
before the root filesystem was mounted. This avoids holding a reference on that initramfs
indefinitely and ensures that the pid and sock files used to communicate with mdmon are in a
standard place.

--all This tells mdmon to find any active containers and start monitoring each of them if appropriate.
This is normally used with --takeover late in the boot sequence. A separate mdmon process is
started for each container as the --all argument is over-written with the name of the container.
To allow for containers with names longer than 5 characters, this argument can be arbitrarily
extended, e.g. to --all-active-arrays.

Note that
mdmon is automatically started by mdadm when needed and so does not need to be considered when
working with RAID arrays. The only times it is run other than by mdadm is when the boot scripts
need to restart it after mounting the new root filesystem.

START UP AND SHUTDOWN

       As mdmon needs to be running whenever any filesystem on the monitored device is mounted there are special
       considerations  when the root filesystem is mounted from an mdmon monitored device.  Note that in general
       mdmon is needed even if the filesystem is mounted read-only as some filesystems can still  write  to  the
       device in those circumstances, for example to replay a journal after an unclean shutdown.

       When  the  array  is  assembled  by the initramfs code, mdadm will automatically start mdmon as required.
       This means that mdmon must be installed on  the  initramfs  and  there  must  be  a  writable  filesystem
       (typically  tmpfs)  in which mdmon can create a .pid and .sock file.  The particular filesystem to use is
       given to mdmon at compile time and defaults to /run/mdadm.

       This filesystem must persist through to shutdown time.

       After the final root filesystem has be instantiated (usually with pivot_root) mdmon should  be  run  with
       --all  --takeover  so  that  the mdmon running from the initramfs can be replaced with one running in the
       main root, and so the memory used by the initramfs can be released.

       At shutdown time, mdmon should not be killed along with other processes.  Also as it holds a file (socket
       actually)  open  in  /dev  (by  default)  it  will  not  be  possible to unmount /dev if it is a separate
       filesystem.

EXAMPLES

         mdmon --all-active-arrays --takeover
       Any mdmon which is currently running is killed and a new instance is started.  This should be run  during
       in the boot sequence if an initramfs was used, so that any mdmon running from the initramfs will not hold
       the initramfs active.