xenial (1) sluice.1.gz

Provided by: sluice_0.02.02-1_amd64 bug

NAME

       sluice - a tool to control data flow at a specified rate

SYNOPSIS

       sluice [options]

DESCRIPTION

       sluice  reads  input  and  outputs a specified data rate. It has various data rate controlling mechanisms
       that can be tuned for specific use cases where necessary.

OPTIONS

       sluice options are as follow:

       -a     append output to a file (used in conjunction with the -t 'tee' or -O options). Instead of creating
              a new file or truncating an existing file, this option appends data to the file.

       -c delay
              enables  a  constant delay time (in seconds) between writes. This option adjusts the output buffer
              size to try and keep the data rate constant.  The output buffer size in this mode is initially set
              to the data rate × the delay.

              This  option  is  mutually exclusive to the -i option and implicitly enables the -o overrun and -u
              underrun buffer management options to dynamically re-size the read/write buffer to keep  the  data
              rate  constant.   By default this adjusts the buffer based on the total amount of data transferred
              and the time to write this (equivalent to the -s 0 turning mode).  However, if the -s shift  value
              is  greater  than 0, then the new size is adjusted by the previous size right shifted by the shift
              value.

       -d     discard data, do not copy it to stdout. This makes sluice act as a data sink.

       -D mode
              select the delay mode.  The are various approaches to when to perform the data  rate  delays.  The
              default  is to perform the read, then write and finally the delay for each iteration. However, the
              -D option allows one to select the delay mode as follows:

                                 Mode            Delay strategy              Delay Duration
                                  0     Read, Write, Delay (default)       1 × delay time
                                  1     Delay, Read, Write                 1 × delay time
                                  2     Read, Delay, Write                 1 × delay time
                                  3     Delay, Read, Delay, Write          2 × 1/2 delay time
                                  4     Read, Delay, Write, Delay          2 × 1/2 delay time
                                  5     Delay, Read, Delay, Write, Delay   3 × 1/3 delay time

              Note that modes 3 and 4 perform two delays each comprising of  1/2  the  delay  time  and  mode  5
              performs 3 delays each comprising of 1/3 the delay time.

              Modes  1,  3,  5  maybe considered as not entirely accurate in terms of the total run duration. In
              these modes an extraneous delay occurs before the final end-of-file empty read is performed.

       -e     ignore read errors. The failed read is replaced by zeros.

       -f freq
              specify the frequency of -v verbose statistics updates. The default is 1/4 of a second. Note  that
              sluice  will try to emit updates close to the requested frequency, however, if the read/write rate
              is less than the frequency then the updates occur only at the read/write rate.

       -h     show help

       -i size
              specify the read/write size in bytes. The K, M, G, T and P suffixes allow one to specify  size  in
              Kilobytes,  Megabytes,  Gigabytes,  Terabytes and Petabytes respectively.  This option is mutually
              exclusive to the -c option.

              In this mode, the delays between writes are used to control the data rate. By default the delay is
              based  on  the  total  amount  of  data  transferred  and  the  time taken to write this.  This is
              equivalent to the -s 0 tuning mode.   However, if the -s shift value is greater than 0,  then  the
              new delay is adjusted by the previous delay right shifted by the shift value.

              A  special  hybrid rate control mode can be invoked by also using the -o overflow and -u underflow
              options to also enable dynamic re-sizing of the read/write buffer.  By default  this  adjusts  the
              buffer based on the total amount of data transferred and the time to write this (equivalent to the
              -s 0 turning mode).  However, if the -s shift value is greater  than  0,  then  the  new  size  is
              adjusted by the previous size right shifted by the shift value.

       -I file
              read input from file rather than from stdin.

       -m size
              specify amount of data to process, the default size is in bytes, but the K, M, G, T and P suffixes
              can specify size in Kilobytes, Megabytes, Gigabytes, Terabytes and Petabytes respectively. If this
              size is less than the write size, then the write size is truncated to be the -m size.

       -n     no rate control. This is just a straight data copy much like cat and all data rate controls cannot
              be used. Combined with the -v and -S options one can observe the data rates of the copy.

       -o     detect overrun and re-size read/write buffer size to try and stop overrun. This  will  shrink  the
              buffer  each time consecutive overruns are detected. See the -s option for details of the size re-
              adjustment mechanism.

       -O file
              send output to file, equivalent to -dt file

       -p     enable verbose stats showing % progress and ETA information. This is only valid using the -I or -m
              option and the if file size is non-zero. See the -v option for more details.

       -P pidfile
              write the process ID of sluice in file pidfile. The file is removed when sluice exits.

       -r rate
              specify  the  data  rate  in  bytes per second. The K, M, G and T suffixes can specify the rate in
              Kilobytes/sec, Megabytes/sec, Gigabytes/sec  and  Terabytes/sec  respectively.  This  option  must
              always be provided except when the -n option is used.

       -R     do not read from stdin, instead read random data from /dev/urandom.

       -s shift
              modify the rate adjustment shift. This is a data rate tuning scaling factor used by the -r, -c, -o
              and - options.

              For the -r option, the delay between each write is controlled by modifying the previous  delay  by
              adding  or subtracting the previous delay right shifted by this shift value.  The larger the shift
              value the longer it takes to adjust up/down to the specified rate.  The smaller  the  shift  value
              the  quicker it takes to reach the optimal delay, however, this can result in a highly fluctuating
              rates at the the beginning because the delay varies by a large amount causing large  overruns  and
              underruns.  A shift value of 3 works well for most fast rates.

              For  the  -c,  -o  and -u options, the size of the buffer is modified by adding or subtracting the
              previous size shifted by the shift value. Again, a shift value of  3  works  well  for  most  fast
              rates.

              If  the  shift  value  is  set to 0, then the shift rate adjustment tuning mechanism is explicitly
              turned off and data rates are adjusted based on the total amount of data transferred and the  time
              to write this.

              Small -s shift values of 1 and 2 can cause rapid oscillations before data rate damping fully kicks
              into action. The value of -s 0 (the default) is recommended for accurate low-speed data transfers.

       -S     print various performance and buffering statistics to stderr when end of file is reached.

       -t file
              tee output to the specified file. Output is written to both stdout  and  to  the  named  file.  By
              default, the file will be created if it does not exist or re-written if it already exists. Use the
              -a option to append to an existing file.

       -T t   stop slice test after t seconds. One can also specify the  units  of  time  in  seconds,  minutes,
              hours, days or years with the suffix s, m, h, d or y.

       -u     detect  underrun and re-size read/write buffer size to try and stop underrun. This will expand the
              buffer each time consecutive underruns are detected. The buffer will not be expanded any more than
              4MB in size.  See the -s option for details of the size re-adjustment mechanism.

       -v     write  verbose statistics to stderr. By default, this will display the current data rate, the last
              data rate adjusment ('-' = underrun, '+' = overrun), total bytes  transferred,  duration  and  the
              current buffer size.

              With the -p option, the progess statistics are displayed. This will display the current data rate,
              total bytes  transferred,  duration,  percentage  complete  so  far  and  the  estimated  time  to
              completion.   Note  that  the  estimation is available using the -I and -m options and if the file
              size is non-zero.

       -V     print version information to standard out and exit successfully.

       -w     warn if a long burst of continuous data rate underrun occurs, the warning is issued just once.  To
              overcome  the  underrun increase the -i read/write buffer size or use the -u option to auto-expand
              the read/write buffer.  Too many underruns implies that too small a buffer or not  enough  CPU  is
              available to keep up with the required data rate.

       -z     do not read from stdin, instead generate a stream of zeros (equivalent to reading from /dev/zero).

       SIGUSR1 SIGINFO
              Sending SIGUSR1 (or SIGINFO on BSD systems) will toggle the verbose data rate mode on/off.

       SIGUSR2
              Toggle underrun/overrun (-u, -o) options on/off.

NOTES

       If  neither  -i  or -c options are used, then sluice defaults to using a write buffer size of 1/32 of the
       data rate and bounded between the limits of 1 byte and 64MB. Sluice will try to keep the data rate steady
       by adjusting the delay between writes. To tune this, see the -s option.

EXAMPLES

       Read /dev/zero and write in 4K sizes at the rate of 1MB/sec to the file 'example.dat'
               cat /dev/zero | sluice -i 4K -r 1M > example.dat

       Read  32MB from /dev/zero and write at the rate of 64K/sec to stdout with feedback on duration and ETA on
       stderr using 4K buffer writes and a tuning shift of 4.
               cat /dev/zero | sluice -r 64K -vp -m 32M -i 4K -s 4

       Generate a stream of zeros and write at a rate of 1MB/sec to a fifo  named  'myfifo'  with  underrun  and
       overrun buffer management
               sluice -z -u -o -r 1MB -O myfifo

       Write random data at 5MB per second to the file 'myfile' doing a write every 0.1 seconds
               sluice -R -r 5M -c 0.1 > myfile

       Write zeros to the file 'example-file' in 64K chunks and measure write rate as a crude throughput test
               sluice -nzSv -f 0.1 -i 64K > example-file

       Read data from somehost.com on port 1234 at a rate of 2MB per second and discard the data, e.g. this is a
       constant rate data sink.
               nc somehost.com 1234 | sluice -d -r 2MB -i 8K

EXIT STATUS

       Sluice sets the exit status as follows:

                                   Status                  Decription

                                     0      Exited successfully.
                                     1      Invalid or out of range option provided.
                                     2      File open error.
                                     3      Sleep error.
                                     4      Failed to get time of day.
                                     5      Signal handler setup error.
                                     6      Read error (file or stdin).
                                     7      Write error (file or stdout).
                                     8      Buffer allocation failed.

BUGS

       Stopping and starting sluice using SIGSTOP and SIGCONT will interfere with the  internal  buffering  rate
       calculations  causing  sluice to try to catch up and this may affect the short term data rate immediately
       after the SIGCONT.

SEE ALSO

       cat(1), pv(1), cstream(1)

AUTHOR

       sluice was written by Colin King <colin.king@canonical.com> with testing feedback  and  help  from  Kamal
       Mostafa.

       This manual page was written by Colin King <colin.king@canonical.com>, for the Ubuntu project (but may be
       used by others).

       Copyright © 2014-2015 Canonical Ltd.
       This is free software; see the source for copying  conditions.   There  is  NO  warranty;  not  even  for
       MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

                                                  May 18, 2015                                         SLUICE(1)