NAME

       maildir - E-mail directory

SYNOPSIS

       $HOME/Maildir

DESCRIPTION

       A “Maildir” is a structured directory that holds E-mail messages. Maildirs were first
       implemented by the Qmail mail server. Qmail's maildirs were a simple data structure,
       nothing more than a single collection of E-mail messages. The Courier mail server builds
       upon Qmail's maildirs to provide extended functionality, such as folders and quotas. This
       document describes the Courier mail server's extended maildirs, without explicitly
       identifying The Courier mail server-specific extensions. See maildir(5) in Qmail's
       documentation for the original definition of maildirs.

       Traditionally, E-mail folders were saved as plain text files, called “mboxes”. Mboxes have
       known limitations. Only one application can use an mbox at the same time. Locking is
       required in order to allow simultaneous concurrent access by different applications.
       Locking is often problematic, and not very reliable in network-based filesystem
       requirements. Some network-based filesystems don't offer any reliable locking mechanism at
       all. Furthermore, even bulletproof locking won't prevent occasional mbox corruption. A
       process can be killed or terminated in the middle of updating an mbox. This will likely
       result in corruption, and a loss of most messages in the mbox.

       Maildirs allow multiple concurrent access by different applications. Maildirs do not
       require locking. Multiple applications can update a maildir at the same time, without
       stepping on each other's feet.

   Maildir contents
       A “maildir” is a directory that's created by maildirmake(1)[1]. Naturally, maildirs should
       not have any group or world permissions, unless you want other people to read your mail. A
       maildir contains three subdirectories: tmp, new, and cur. These three subdirectories
       comprise the primary folder, where new mail is delivered by the system.

       Folders are additional subdirectories in the maildir whose names begin with a period: such
       as .Drafts or .Sent. Each folder itself contains the same three subdirectories, tmp, new,
       and cur, and an additional zero-length file named maildirfolder, whose purpose is to
       inform any mail delivery agent that it's really delivering to a folder, and that the mail
       delivery agent should look in the parent directory for any maildir-related information.

       Folders are not physically nested. A folder subdirectory, such as .Sent does not itself
       contain any subfolders. The main maildir contains a single, flat list of subfolders. These
       folders are logically nested, and periods serve to separate folder hierarchies. For
       example, .Sent.2002 is considered to be a subfolder called “2002” which is a subfolder of
       “Sent”.

       Folder name encoding
           Folder names can contain any Unicode character, except for control characters.
           US-ASCII characters, U+0x0020 - U+0x007F, except for the period, forward-slash, and
           ampersand characters (U+0x002E, U+0x002F, and U+0x0026) represent themselves. The
           ampersand is represent by the two character sequence “&-”. The period, forward slash,
           and non US-ASCII Unicode characters are represented using the UTF-7 character set, and
           encoded with a modified form of base64-encoding.

           The “&” character starts the modified base64-encoded sequence; the sequence is
           terminated by the “-” character. The sequence of 16-bit Unicode characters is written
           in big-endian order, and encoded using the base64-encoding method described in section
           5.2 of RFC 1521[2], with the following modifications:

           •   The “=” padding character is omitted. When decoding, an incomplete 16-bit
               character is discarded.

           •   The comma character, “,” is used in place of the “/” character in the base64
               alphabet.

           For example, the word “Resume” with both "e"s being the e-acute character, U+0x00e9,
           is encoded as “R&AOk-sum&AOk-” (so a folder of that name would be a maildir
           subdirectory called “.R&AOk-sum&AOk-”).

       Other maildir contents
           Software that uses maildirs may also create additional files besides the tmp, new, and
           cur subdirectories -- in the main maildir or a subfolder -- for its own purposes.

   Messages
       E-mail messages are stored in separate, individual files, one E-mail message per file. The
       tmp subdirectory temporarily stores E-mail messages that are in the process of being
       delivered to this maildir.  tmp may also store other kinds of temporary files, as long as
       they are created in the same way that message files are created in tmp. The new
       subdirectory stores messages that have been delivered to this maildir, but have not yet
       been seen by any mail application. The cur subdirectory stores messages that have already
       been seen by mail applications.

   Adding new mail to maildirs
       The following process delivers a new message to the maildir:

       A new unique filename is created using one of two possible forms: “time.MusecPpid.host”,
       or “time.MusecPpid_unique.host”.  “time” and “usec” is the current system time, obtained
       from gettimeofday(2).  “pid” is the process number of the process that is delivering this
       message to the maildir.  “host” is the name of the machine where the mail is being
       delivered. In the event that the same process creates multiple messages, a suffix unique
       to each message is appended to the process id; preferrably an underscore, followed by an
       increasing counter. This applies whether messages created by a process are all added to
       the same, or different, maildirs. This protocol allows multiple processes running on
       multiple machines on the same network to simultaneously create new messages without
       stomping on each other.

       The filename created in the previous step is checked for existence by executing the
       stat(2) system call. If stat(2) results in ANYTHING OTHER than the system error ENOENT,
       the process must sleep for two seconds, then go back and create another unique filename.
       This is an extra step to insure that each new message has a completely unique filename.

       Other applications that wish to use tmp for temporary storage should observe the same
       protocol (but see READING MAIL FROM MAILDIRS below, because old files in tmp will be
       eventually deleted).

       If the stat(2) system call returned ENOENT, the process may proceed to create the file in
       the tmp subdirectory, and save the entire message in the new file. The message saved MUST
       NOT have the “From_” header that is used to mboxes. The message also MUST NOT have any
       “From_” lines in the contents of the message prefixed by the “>” character.

       When saving the message, the number of bytes returned by the write(2) system call must be
       checked, in order to make sure that the complete message has been written out.

       After the message is saved, the file descriptor is fstat(2)-ed. The file's device number,
       inode number, and the its byte size, are saved. The file is closed and is then immediately
       moved/renamed into the new subdirectory. The name of the file in new should be
       “time.MusecPpidVdevIino.host,S=cnt”, or “time.MusecPpidVdevIino_unique.host,S=cnt”.  “dev”
       is the message's device number, “ino” is the message's inode number (from the previous
       fstat(2) call); and “cnt” is the message's size, in bytes.

       The “,S=cnt” part optimizes the Courier[3] mail server's maildir quota enhancement; it
       allows the size of all the mail stored in the maildir to be added up without issuing the
       stat(2) system call for each individual message (this can be quite a performance drain
       with certain network filesystems).

   READING MAIL FROM MAILDIRS
       Applications that read mail from maildirs should do it in the following order:

       When opening a maildir or a maildir folder, read the tmp subdirectory and delete any files
       in there that are at least 36 hours old.

       Look for new messages in the new subdirectory. Rename new/filename, as
       cur/filename:2,info. Here, info represents the state of the message, and it consists of
       zero or more boolean flags chosen from the following: “D” - this is a 'draft' message, “R”
       - this message has been replied to, “S” - this message has been viewed (seen), “T” - this
       message has been marked to be deleted (trashed), but is not yet removed (messages are
       removed from maildirs simply by deleting their file), “F” - this message has been marked
       by the user, for some purpose. These flags must be stored in alphabetical order. New
       messages contain only the :2, suffix, with no flags, indicating that the messages were not
       seen, replied, marked, or deleted.

       Maildirs may have maximum size quotas defined, but these quotas are purely voluntary. If
       you need to implement mandatory quotas, you should use any quota facilities provided by
       the underlying filesystem that is used to store the maildirs. The maildir quota
       enhancement is designed to be used in certain situations where filesystem-based quotas
       cannot be used for some reason. The implementation is designed to avoid the use of any
       locking. As such, at certain times the calculated quota may be imprecise, and certain
       anomalous situations may result in the maildir actually going over the stated quota. One
       such situation would be when applications create messages without updating the quota
       estimate for the maildir. Eventually it will be precisely recalculated, but wherever
       possible new messages should be created in compliance with the voluntary quota protocol.

       The voluntary quota protocol involves some additional procedures that must be followed
       when creating or deleting messages within a given maildir or its subfolders. The
       deliverquota(8)[4] command is a tiny application that delivers a single message to a
       maildir using the voluntary quota protocol, and hopefully it can be used as a measure of
       last resort. Alternatively, applications can use the libmaildir.a library to handle all
       the low-level dirty details for them. The voluntary quota enhancement is described in the
       maildirquota(7)[5] man page.

   Maildir Quotas
       This is a voluntary mechanism for enforcing "loose" quotas on the maximum sizes of
       maildirs. This mechanism is enforced in software, and not by the operating system.
       Therefore it is only effective as long as the maildirs themselves are not directly
       accessible by their users, since this mechanism is trivially disabled.

       If possible, operating system-enforced quotas are preferrable. Where operating system
       quota enforcement is not available, or not possible, this voluntary quota enforcement
       mechanism might be an acceptable compromise. Since it's enforced in software, all software
       that modifies or accesses the maildirs is required to voluntary obey and enforce a quota.
       The voluntary quota implementation is flexible enough to allow non quota-aware
       applications to also access the maildirs, without any drastic consequences. There will be
       some non-drastic consequences, though. Of course, non quota-aware applications will not
       enforce any defined quotas. Furthermore, this voluntary maildir quota mechanism works by
       estimating the current size of the maildir, with periodic exact recalculation. Obviously
       non quota-aware maildir applications will not update the maildir size estimation, so the
       estimate will be thrown off for some period of time, until the next recalculation.

       This voluntary quota mechanism is designed to be a reasonable compromise between
       effectiveness, and performance. The entire purpose of using maildir-based mail storage is
       to avoid any kind of locking, and to permit parallel access to mail by multiple
       applications. In order to compute the exact size of a maildir, the maildir must be locked
       somehow to prevent any modifications while its contents are added up. Obviously something
       like that defeats the original purpose of using maildirs, therefore the voluntary quota
       mechanism does not use locking, and that's why the current recorded maildir size is always
       considered to be an estimate. Regular size recalculations will compensate for any
       occasional race conditions that result in the estimate to be thrown off.

       A quota for an existing maildir is installed by running maildirmake with the -q option,
       and naming an existing maildir. The -q option takes a parameter, quota, which is a
       comma-separated list of quota specifications. A quota specification consists of a number
       followed by either 'S', indicating the maximum message size in bytes, or 'C', maximum
       number of messages. For example:

           maildirmake -q 5000000S,1000C ./Maildir

       This sets the quota to 5,000,000 bytes or 1000 messages, whichever comes first.

           maildirmake -q 1000000S ./Maildir

       This sets the quota to 1,000,000 bytes, without limiting the number of messages.

       A quota of an existing maildir can be changed by rerunning the maildirmake command with a
       new -q option. To delete a quota entirely, delete the Maildir/maildirsize file.

SEE ALSO

       maildirmake(1)[1].

AUTHOR

       Sam Varshavchik
           Author

NOTES

        1. maildirmake(1)
           [set $man.base.url.for.relative.links]/maildirmake.html

        2. RFC 1521
           http://www.rfc-editor.org/rfc/rfc1521.txt

        3. Courier
           http://www.courier-mta.org

        4. deliverquota(8)
           [set $man.base.url.for.relative.links]/deliverquota.html

        5. maildirquota(7)
           [set $man.base.url.for.relative.links]/maildirquota.html