focal (7) pipe.7.gz

Provided by: manpages_5.05-1_all bug

NAME

       pipe - overview of pipes and FIFOs

DESCRIPTION

       Pipes  and FIFOs (also known as named pipes) provide a unidirectional interprocess communication channel.
       A pipe has a read end and a write end.  Data written to the write end of a pipe can be read from the read
       end of the pipe.

       A pipe is created using pipe(2), which creates a new pipe and returns two file descriptors, one referring
       to the read end of the pipe, the other referring to the write  end.   Pipes  can  be  used  to  create  a
       communication channel between related processes; see pipe(2) for an example.

       A  FIFO (short for First In First Out) has a name within the filesystem (created using mkfifo(3)), and is
       opened using open(2).  Any process may open a FIFO, assuming the file permissions allow it.  The read end
       is  opened  using  the  O_RDONLY  flag; the write end is opened using the O_WRONLY flag.  See fifo(7) for
       further details.  Note: although FIFOs have a pathname in the filesystem, I/O on FIFOs does  not  involve
       operations on the underlying device (if there is one).

   I/O on pipes and FIFOs
       The  only  difference  between  pipes and FIFOs is the manner in which they are created and opened.  Once
       these tasks have been accomplished, I/O on pipes and FIFOs has exactly the same semantics.

       If a process attempts to read from an empty pipe, then read(2) will block until data is available.  If  a
       process attempts to write to a full pipe (see below), then write(2) blocks until sufficient data has been
       read from the pipe to allow the write to complete.  Nonblocking I/O is possible  by  using  the  fcntl(2)
       F_SETFL operation to enable the O_NONBLOCK open file status flag.

       The communication channel provided by a pipe is a byte stream: there is no concept of message boundaries.

       If all file descriptors referring to the write end of a pipe have been closed, then an attempt to read(2)
       from the pipe will see end-of-file (read(2) will return 0).  If all file  descriptors  referring  to  the
       read  end of a pipe have been closed, then a write(2) will cause a SIGPIPE signal to be generated for the
       calling process.  If the calling process is ignoring this signal, then  write(2)  fails  with  the  error
       EPIPE.   An  application  that  uses  pipe(2)  and  fork(2)  should  use suitable close(2) calls to close
       unnecessary duplicate file descriptors; this ensures that end-of-file  and  SIGPIPE/EPIPE  are  delivered
       when appropriate.

       It is not possible to apply lseek(2) to a pipe.

   Pipe capacity
       A  pipe  has  a  limited capacity.  If the pipe is full, then a write(2) will block or fail, depending on
       whether the O_NONBLOCK flag is set (see below).  Different implementations have different limits for  the
       pipe  capacity.  Applications should not rely on a particular capacity: an application should be designed
       so that a reading process consumes data as soon as it is available, so that a writing  process  does  not
       remain blocked.

       In  Linux versions before 2.6.11, the capacity of a pipe was the same as the system page size (e.g., 4096
       bytes on i386).  Since Linux 2.6.11, the pipe capacity is 16 pages (i.e., 65,536 bytes in a system with a
       page  size  of  4096 bytes).  Since Linux 2.6.35, the default pipe capacity is 16 pages, but the capacity
       can be queried and set using the fcntl(2) F_GETPIPE_SZ and F_SETPIPE_SZ  operations.   See  fcntl(2)  for
       more information.

       The  following ioctl(2) operation, which can be applied to a file descriptor that refers to either end of
       a pipe, places a count of the number of unread bytes in the pipe in the int  buffer  pointed  to  by  the
       final argument of the call:

           ioctl(fd, FIONREAD, &nbytes);

       The FIONREAD operation is not specified in any standard, but is provided on many implementations.

   /proc files
       On Linux, the following files control how much memory can be used for pipes:

       /proc/sys/fs/pipe-max-pages (only in Linux 2.6.34)
              An  upper  limit,  in  pages,  on  the  capacity  that  an  unprivileged  user  (one  without  the
              CAP_SYS_RESOURCE capability) can set for a pipe.

              The default value for this limit is 16 times the default pipe  capacity  (see  above);  the  lower
              limit is two pages.

              This interface was removed in Linux 2.6.35, in favor of /proc/sys/fs/pipe-max-size.

       /proc/sys/fs/pipe-max-size (since Linux 2.6.35)
              The  maximum  size  (in  bytes)  of  individual  pipes  that  can  be  set  by  users  without the
              CAP_SYS_RESOURCE capability.  The value assigned to this file may be rounded  upward,  to  reflect
              the  value  actually employed for a convenient implementation.  To determine the rounded-up value,
              display the contents of this file after assigning a value to it.

              The default value for this file is 1048576 (1 MiB).  The minimum value that  can  be  assigned  to
              this file is the system page size.  Attempts to set a limit less than the page size cause write(2)
              to fail with the error EINVAL.

              Since Linux 4.9, the value on this file also acts as a ceiling on the default capacity  of  a  new
              pipe or newly opened FIFO.

       /proc/sys/fs/pipe-user-pages-hard (since Linux 4.5)
              The  hard  limit on the total size (in pages) of all pipes created or set by a single unprivileged
              user (i.e., one with neither the CAP_SYS_RESOURCE nor the CAP_SYS_ADMIN capability).  So  long  as
              the  total  number  of pages allocated to pipe buffers for this user is at this limit, attempts to
              create new pipes will be denied, and attempts to increase a pipe's capacity will be denied.

              When the value of this limit is zero (which is the default), no hard limit is applied.

       /proc/sys/fs/pipe-user-pages-soft (since Linux 4.5)
              The soft limit on the total size (in pages) of all pipes created or set by a  single  unprivileged
              user  (i.e.,  one with neither the CAP_SYS_RESOURCE nor the CAP_SYS_ADMIN capability).  So long as
              the total number of pages allocated to pipe buffers for this user is  at  this  limit,  individual
              pipes  created  by  a user will be limited to one page, and attempts to increase a pipe's capacity
              will be denied.

              When the value of this limit is zero, no soft limit is applied.  The default value for  this  file
              is 16384, which permits creating up to 1024 pipes with the default capacity.

       Before  Linux  4.9,  some bugs affected the handling of the pipe-user-pages-soft and pipe-user-pages-hard
       limits; see BUGS.

   PIPE_BUF
       POSIX.1 says that write(2)s of less than PIPE_BUF bytes must be atomic: the output data is written to the
       pipe  as  a  contiguous  sequence.   Writes  of more than PIPE_BUF bytes may be nonatomic: the kernel may
       interleave the data with data written by other processes.  POSIX.1 requires PIPE_BUF to be at  least  512
       bytes.   (On Linux, PIPE_BUF is 4096 bytes.)  The precise semantics depend on whether the file descriptor
       is nonblocking (O_NONBLOCK), whether there are multiple writers to the pipe, and  on  n,  the  number  of
       bytes to be written:

       O_NONBLOCK disabled, n <= PIPE_BUF
              All  n  bytes  are  written  atomically; write(2) may block if there is not room for n bytes to be
              written immediately

       O_NONBLOCK enabled, n <= PIPE_BUF
              If there is room to write n bytes to the pipe, then write(2) succeeds immediately, writing  all  n
              bytes; otherwise write(2) fails, with errno set to EAGAIN.

       O_NONBLOCK disabled, n > PIPE_BUF
              The  write  is  nonatomic:  the  data given to write(2) may be interleaved with write(2)s by other
              process; the write(2) blocks until n bytes have been written.

       O_NONBLOCK enabled, n > PIPE_BUF
              If the pipe is full, then write(2) fails, with errno set to EAGAIN.  Otherwise, from 1 to n  bytes
              may  be  written (i.e., a "partial write" may occur; the caller should check the return value from
              write(2) to see how many bytes were actually written), and these bytes  may  be  interleaved  with
              writes by other processes.

   Open file status flags
       The  only  open  file  status flags that can be meaningfully applied to a pipe or FIFO are O_NONBLOCK and
       O_ASYNC.

       Setting the O_ASYNC flag for the read end of a pipe causes a signal (SIGIO by default)  to  be  generated
       when  new  input becomes available on the pipe.  The target for delivery of signals must be set using the
       fcntl(2) F_SETOWN command.  On Linux, O_ASYNC is supported for pipes and FIFOs only since kernel 2.6.

   Portability notes
       On some systems (but not Linux), pipes are bidirectional: data can  be  transmitted  in  both  directions
       between  the  pipe ends.  POSIX.1 requires only unidirectional pipes.  Portable applications should avoid
       reliance on bidirectional pipe semantics.

   BUGS
       Before Linux 4.9, some bugs affected the handling of the  pipe-user-pages-soft  and  pipe-user-pages-hard
       limits when using the fcntl(2) F_SETPIPE_SZ operation to change a pipe's capacity:

       (1)  When  increasing  the  pipe  capacity, the checks against the soft and hard limits were made against
            existing consumption, and excluded the memory required for the increased  pipe  capacity.   The  new
            increase in pipe capacity could then push the total memory used by the user for pipes (possibly far)
            over a limit.  (This could also trigger the problem described next.)

            Starting with Linux 4.9, the limit checking includes the memory required for the new pipe capacity.

       (2)  The limit checks were performed even when the new pipe capacity was  less  than  the  existing  pipe
            capacity.  This could lead to problems if a user set a large pipe capacity, and then the limits were
            lowered, with the result that the user could no longer decrease the pipe capacity.

            Starting with Linux 4.9, checks against the limits are  performed  only  when  increasing  a  pipe's
            capacity; an unprivileged user can always decrease a pipe's capacity.

       (3)  The accounting and checking against the limits were done as follows:

            (a) Test whether the user has exceeded the limit.
            (b) Make the new pipe buffer allocation.
            (c) Account new allocation against the limits.

            This  was  racey.   Multiple  processes  could pass point (a) simultaneously, and then allocate pipe
            buffers that were accounted for only in step (c), with  the  result  that  the  user's  pipe  buffer
            allocation could be pushed over the limit.

            Starting  with  Linux  4.9,  the  accounting  step is performed before doing the allocation, and the
            operation fails if the limit would be exceeded.

       Before Linux 4.9, bugs similar to points (1) and (3) could also occur when the  kernel  allocated  memory
       for a new pipe buffer; that is, when calling pipe(2) and when opening a previously unopened FIFO.

SEE ALSO

       mkfifo(1),  dup(2),  fcntl(2),  open(2),  pipe(2), poll(2), select(2), socketpair(2), splice(2), stat(2),
       tee(2), vmsplice(2), mkfifo(3), epoll(7), fifo(7)

COLOPHON

       This page is part of release 5.05 of  the  Linux  man-pages  project.   A  description  of  the  project,
       information   about   reporting   bugs,   and   the  latest  version  of  this  page,  can  be  found  at
       https://www.kernel.org/doc/man-pages/.