Ubuntu Manpage: mdmon - monitor MD external metadata arrays

NAME

       mdmon - monitor MD external metadata arrays

SYNOPSIS

       mdmon [--all] [--takeover] [--foreground] CONTAINER

OVERVIEW

       The  2.6.27  kernel  brings  the  ability  to  support external metadata arrays.  External
       metadata implies that user space handles  all  updates  to  the  metadata.   The  kernel's
       responsibility  is to notify user space when a "metadata event" occurs, like disk failures
       and clean-to-dirty transitions.  The kernel, in important cases, waits for user  space  to
       take action on these notifications.

DESCRIPTION

Metadata updates:
To service metadata update requests a daemon, mdmon, is introduced. Mdmon is tasked with
polling the sysfs namespace looking for changes in array_state, sync_action, and per disk
state attributes. When a change is detected it calls a per metadata type handler to make
modifications to the metadata. The following actions are taken:

array_state - inactive
Clear the dirty bit for the volume and let the array be stopped

array_state - write pending
Set the dirty bit for the array and then set array_state to active. Writes
are blocked until userspace writes active.

array_state - active-idle
The safe mode timer has expired so set array state to clean to block writes
to the array

array_state - clean
Clear the dirty bit for the volume

array_state - read-only
This is the initial state that all arrays start at. mdmon takes one of the
three actions:

1/ Transition the array to read-auto keeping the dirty bit clear if the
metadata handler determines that the array does not need resyncing or
other modification

2/ Transition the array to active if the metadata handler determines a
resync or some other manipulation is necessary

3/ Leave the array read-only if the volume is marked to not be
monitored; for example, the metadata version has been set to
"external:-dev/md127" instead of "external:/dev/md127"

sync_action - resync-to-idle
Notify the metadata handler that a resync may have completed. If a resync
process is idled before it completes this event allows the metadata handler
to checkpoint resync.

sync_action - recover-to-idle
A spare may have completed rebuilding so tell the metadata handler about the
state of each disk. This is the metadata handler's opportunity to clear any
"out-of-sync" bits and clear the volume's degraded status. If a recovery
process is idled before it completes this event allows the metadata handler
to checkpoint recovery.

<disk>/state - faulty
A disk failure kicks off a series of events. First, notify the metadata
handler that a disk has failed, and then notify the kernel that it can
unblock writes that were dependent on this disk. After unblocking the
kernel this disk is set to be removed+ from the member array. Finally the
disk is marked failed in all other member arrays in the container.

+ Note This behavior differs slightly from native MD arrays where removal is
reserved for a mdadm --remove event. In the external metadata case the
container holds the final reference on a block device and a mdadm --remove
<container> <victim> call is still required.

Containers:
External metadata formats, like DDF, differ from the native MD metadata formats in that
they define a set of disks and a series of sub-arrays within those disks. MD metadata in
comparison defines a 1:1 relationship between a set of block devices and a RAID array.
For example to create 2 arrays at different RAID levels on a single set of disks, MD
metadata requires the disks be partitioned and then each array can be created with a
subset of those partitions. The supported external formats perform this disk carving
internally.

Container devices simply hold references to all member disks and allow tools like mdmon to
determine which active arrays belong to which container. Some array management commands
like disk removal and disk add are now only valid at the container level. Attempts to
perform these actions on member arrays are blocked with error messages like:

"mdadm: Cannot remove disks from a 'member' array, perform this operation on the
parent container"

Containers are identified in /proc/mdstat with a metadata version string
"external:<metadata name>". Member devices are identified by "external:/<container
device>/<member index>", or "external:-<container device>/<member index>" if the array is
to remain readonly.

OPTIONS

CONTAINER
The container device to monitor. It can be a full path like /dev/md/container, or
a simple md device name like md127.

--foreground
Normally, mdmon will fork and continue in the background. Adding this option will
skip that step and run mdmon in the foreground.

--takeover
This instructs mdmon to replace any active mdmon which is currently monitoring the
array. This is primarily used late in the boot process to replace any mdmon which
was started from an initramfs before the root filesystem was mounted. This avoids
holding a reference on that initramfs indefinitely and ensures that the pid and
sock files used to communicate with mdmon are in a standard place.

--all This tells mdmon to find any active containers and start monitoring each of them if
appropriate. This is normally used with --takeover late in the boot sequence. A
separate mdmon process is started for each container as the --all argument is over-
written with the name of the container. To allow for containers with names longer
than 5 characters, this argument can be arbitrarily extended, e.g. to --all-active-
arrays.

Note that
mdmon is automatically started by mdadm when needed and so does not need to be
considered when working with RAID arrays. The only times it is run other than by
mdadm is when the boot scripts need to restart it after mounting the new root
filesystem.

START UP AND SHUTDOWN

       As mdmon needs to be running whenever any filesystem on the monitored  device  is  mounted
       there  are  special  considerations  when  the  root  filesystem  is mounted from an mdmon
       monitored device.  Note that in general mdmon is needed even if the filesystem is  mounted
       read-only  as  some  filesystems can still write to the device in those circumstances, for
       example to replay a journal after an unclean shutdown.

       When the array is assembled by the initramfs code, mdadm will automatically start mdmon as
       required.   This  means  that mdmon must be installed on the initramfs and there must be a
       writable filesystem (typically tmpfs) in which mdmon can create a  .pid  and  .sock  file.
       The  particular  filesystem  to  use  is  given  to  mdmon at compile time and defaults to
       /run/mdadm.

       This filesystem must persist through to shutdown time.

       After the final root filesystem has be instantiated (usually with pivot_root) mdmon should
       be  run with --all --takeover so that the mdmon running from the initramfs can be replaced
       with one running in the main root, and  so  the  memory  used  by  the  initramfs  can  be
       released.

       At shutdown time, mdmon should not be killed along with other processes.  Also as it holds
       a file (socket actually) open in /dev (by default) it will not be possible to unmount /dev
       if it is a separate filesystem.

EXAMPLES

         mdmon --all-active-arrays --takeover
       Any mdmon which is currently running is killed and a new instance is started.  This should
       be run during in the boot sequence if an initramfs was used, so  that  any  mdmon  running
       from the initramfs will not hold the initramfs active.