NAME

     locking — kernel synchronization primitives

DESCRIPTION

     The FreeBSD kernel is written to run across multiple CPUs and as such requires several
     different synchronization primitives to allow the developers to safely access and manipulate
     the many data types required.

   Mutexes
     Mutexes (also called "sleep mutexes") are the most commonly used synchronization primitive
     in the kernel.  Thread acquires (locks) a mutex before accessing data shared with other
     threads (including interrupt threads), and releases (unlocks) it afterwards.  If the mutex
     cannot be acquired, the thread requesting it will sleep.  Mutexes fully support priority
     propagation.

     See mutex(9) for details.

   Spin mutexes
     Spin mutexes are variation of basic mutexes; the main difference between the two is that
     spin mutexes never sleep - instead, they spin, waiting for the thread holding the lock,
     which runs on another CPU, to release it.  Differently from ordinary mutex, spin mutexes
     disable interrupts when acquired.  Since disabling interrupts is expensive, they are also
     generally slower.  Spin mutexes should be used only when necessary, e.g. to protect data
     shared with interrupt filter code (see bus_setup_intr(9) for details).

   Pool mutexes
     With most synchronization primitives, such as mutexes, programmer must provide a piece of
     allocated memory to hold the primitive.  For example, a mutex may be embedded inside the
     structure it protects.  Pool mutex is a variant of mutex without this requirement - to lock
     or unlock a pool mutex, one uses address of the structure being protected with it, not the
     mutex itself.  Pool mutexes are seldom used.

     See mtx_pool(9) for details.

   Reader/writer locks
     Reader/writer locks allow shared access to protected data by multiple threads, or exclusive
     access by a single thread.  The threads with shared access are known as readers since they
     should only read the protected data.  A thread with exclusive access is known as a writer
     since it may modify protected data.

     Reader/writer locks can be treated as mutexes (see above and mutex(9)) with shared/exclusive
     semantics.  More specifically, regular mutexes can be considered to be equivalent to a
     write-lock on an rw_lock. The rw_lock locks have priority propagation like mutexes, but
     priority can be propagated only to an exclusive holder.  This limitation comes from the fact
     that shared owners are anonymous.  Another important property is that shared holders of
     rw_lock can recurse, but exclusive locks are not allowed to recurse.  This ability should
     not be used lightly and may go away.

     See rwlock(9) for details.

   Read-mostly locks
     Mostly reader locks are similar to reader/writer locks but optimized for very infrequent
     write locking.  Read-mostly locks implement full priority propagation by tracking shared
     owners using a caller-supplied tracker data structure.

     See rmlock(9) for details.

   Shared/exclusive locks
     Shared/exclusive locks are similar to reader/writer locks; the main difference between them
     is that shared/exclusive locks may be held during unbounded sleep (and may thus perform an
     unbounded sleep).  They are inherently less efficient than mutexes, reader/writer locks and
     read-mostly locks.  They don't support priority propagation.  They should be considered to
     be closely related to sleep(9).  In fact it could in some cases be considered a conditional
     sleep.

     See sx(9) for details.

   Counting semaphores
     Counting semaphores provide a mechanism for synchronizing access to a pool of resources.
     Unlike mutexes, semaphores do not have the concept of an owner, so they can be useful in
     situations where one thread needs to acquire a resource, and another thread needs to release
     it.  They are largely deprecated.

     See sema(9) for details.

   Condition variables
     Condition variables are used in conjunction with mutexes to wait for conditions to occur.  A
     thread must hold the mutex before calling the cv_wait*(), functions.  When a thread waits on
     a condition, the mutex is atomically released before the thread is blocked, then reacquired
     before the function call returns.

     See condvar(9) for details.

   Giant
     Giant is an instance of a mutex, with some special characteristics:

     1.   It is recursive.

     2.   Drivers and filesystems can request that Giant be locked around them by not marking
          themselves MPSAFE.  Note that infrastructure to do this is slowly going away as non-
          MPSAFE drivers either became properly locked or disappear.

     3.   Giant must be locked first before other locks.

     4.   It is OK to hold Giant while performing unbounded sleep; in such case, Giant will be
          dropped before sleeping and picked up after wakeup.

     5.   There are places in the kernel that drop Giant and pick it back up again.  Sleep locks
          will do this before sleeping.  Parts of the network or VM code may do this as well,
          depending on the setting of a sysctl.  This means that you cannot count on Giant
          keeping other code from running if your code sleeps, even if you want it to.

   Sleep/wakeup
     The functions tsleep(), msleep(), msleep_spin(), pause(), wakeup(), and wakeup_one() handle
     event-based thread blocking.  If a thread must wait for an external event, it is put to
     sleep by tsleep(), msleep(), msleep_spin(), or pause().  Threads may also wait using one of
     the locking primitive sleep routines mtx_sleep(9), rw_sleep(9), or sx_sleep(9).

     The parameter chan is an arbitrary address that uniquely identifies the event on which the
     thread is being put to sleep.  All threads sleeping on a single chan are woken up later by
     wakeup(), often called from inside an interrupt routine, to indicate that the resource the
     thread was blocking on is available now.

     Several of the sleep functions including msleep(), msleep_spin(), and the locking primitive
     sleep routines specify an additional lock parameter.  The lock will be released before
     sleeping and reacquired before the sleep routine returns.  If priority includes the PDROP
     flag, then the lock will not be reacquired before returning.  The lock is used to ensure
     that a condition can be checked atomically, and that the current thread can be suspended
     without missing a change to the condition, or an associated wakeup.  In addition, all of the
     sleep routines will fully drop the Giant mutex (even if recursed) while the thread is
     suspended and will reacquire the Giant mutex before the function returns.

     See sleep(9) for details.

   Lockmanager locks
     Shared/exclusive locks, used mostly in VFS(9), in particular as a vnode(9) lock.  They have
     features other lock types don't have, such as sleep timeout, writer starvation avoidance,
     draining, and interlock mutex, but this makes them complicated to implement; for this
     reason, they are deprecated.

     See lock(9) for details.

INTERACTIONS

     The primitives interact and have a number of rules regarding how they can and can not be
     combined.  Many of these rules are checked using the witness(4) code.

   Bounded vs. unbounded sleep
     The following primitives perform bounded sleep: mutexes, pool mutexes, reader/writer locks
     and read-mostly locks.

     The following primitives block (perform unbounded sleep): shared/exclusive locks, counting
     semaphores, condition variables, sleep/wakeup and lockmanager locks.

     It is an error to do any operation that could result in any kind of sleep while holding spin
     mutex.

     As a general rule, it is an error to do any operation that could result in unbounded sleep
     while holding any primitive from the 'bounded sleep' group.  For example, it is an error to
     try to acquire shared/exclusive lock while holding mutex, or to try to allocate memory with
     M_WAITOK while holding read-write lock.

     As a special case, it is possible to call sleep() or mtx_sleep() while holding a single
     mutex.  It will atomically drop that mutex and reacquire it as part of waking up.  This is
     often a bad idea because it generally relies on the programmer having good knowledge of all
     of the call graph above the place where mtx_sleep() is being called and assumptions the
     calling code has made.  Because the lock gets dropped during sleep, one must re-test all the
     assumptions that were made before, all the way up the call graph to the place where the lock
     was acquired.

     It is an error to do any operation that could result in any kind of sleep when running
     inside an interrupt filter.

     It is an error to do any operation that could result in unbounded sleep when running inside
     an interrupt thread.

   Interaction table
     The following table shows what you can and can not do while holding one of the
     synchronization primitives discussed:

           You have: You want: spin mtx  mutex   sx      rwlock  rmlock sleep
           spin mtx            ok-1      no      no      no      no     no-3
           mutex               ok        ok-1    no      ok      ok     no-3
           sx                  ok        ok      ok-2    ok      ok     ok-4
           rwlock              ok        ok      no      ok-2    ok     no-3
           rmlock              ok        ok      no-5    ok      ok-2   no-5

     *1 Recursion is defined per lock.  Lock order is important.

     *2 Readers can recurse though writers can not.  Lock order is important.

     *3 There are calls that atomically release this primitive when going to sleep and reacquire
     it on wakeup (e.g.  mtx_sleep(), rw_sleep() and msleep_spin()).

     *4 Though one can sleep holding an sx lock, one can also use sx_sleep() which will
     atomically release this primitive when going to sleep and reacquire it on wakeup.

     *5 Read-mostly locks can be initialized to support sleeping while holding a write lock.  See
     rmlock(9) for details.

   Context mode table
     The next table shows what can be used in different contexts.  At this time this is a rather
     easy to remember table.

           Context:            spin mtx  mutex   sx      rwlock  rmlock sleep
           interrupt filter:   ok        no      no      no      no     no
           interrupt thread:   ok        ok      no      ok      ok     no
           callout:            ok        ok      no      ok      no     no
           syscall:            ok        ok      ok      ok      ok     ok

SEE ALSO

     witness(4), condvar(9), lock(9), mtx_pool(9), mutex(9), rmlock(9), rwlock(9), sema(9),
     sleep(9), sx(9), BUS_SETUP_INTR(9), LOCK_PROFILING(9)

HISTORY

     These functions appeared in BSD/OS 4.1 through FreeBSD 7.0.

BUGS

     There are too many locking primitives to choose from.