Ubuntu Manpage: nbdkit-lzip-filter

name
synopsis
description
parameters
files
version
see also
authors
copyright
license

Provided by: nbdkit_1.42.0-1ubuntu1_amd64

NAME

       nbdkit-lzip-filter - nbdkit lzip filter

SYNOPSIS

        nbdkit --filter=lzip file FILENAME.lz

        nbdkit --filter=lzip curl https://example.com/FILENAME.lz

DESCRIPTION

"nbdkit-lzip-filter" is a filter for nbdkit(1) which transparently decompresses an lzip(1)-compressed
file. You can place this filter on top of nbdkit-file-plugin(1) to decompress a local .lz file, or on
top of other plugins such as nbdkit-curl-plugin(1):

nbdkit curl --filter=lzip https://example.com/disk.lz

With nbdkit-partition-filter(1) it can be used to mount a specific partition of a compressed disk image:

nbdkit curl --filter=partition --filter=lzip \
https://example.com/disk.img.lz partition=2

The filter only supports read-only connections.

Note that this filter was made primarily for multi-member files. Single-member files are currently only
supported up to a size of "lzip-max-block" and their contents are kept in memory.

Single-member files and multi-member files
An lzip-compressed file consists of one or more members, with each member containing an LZMA stream and
some metadata. Most lzip files consist of only a single member holding all of the compressed data. You
can find out the number of members in an lzip file by doing:

$ lzip --list --verbose lfs.img.lz
dict memb trail uncompressed compressed saved name
8 MiB 1 0 10737418240 1808781538 83.15% lfs.img.lz
↑↑↑
members

Seeking an lzip file is done by seeking directly to the lower member boundary, then decompressing data
until the precise byte is reached. Because of this, single-member files are not seekable and must be
decompressed entirely.

Getting the best random access performance
Use the plzip(1) parallel compressor with the --data-size option set to a small-ish block size. plzip
first splits the input file into fixed-size data blocks before compressing each block as a separate
member. For example this is the same image as above compressed with plzip on default settings, resulting
in a block size of 16 MiB:

$ lzip --list --verbose lfs.img.lz
dict memb trail uncompressed compressed saved name
8 MiB 640 0 10737418240 1839569039 82.87% lfs.img.lz
↑↑↑
members

This file allows random access in constant time. At most 16 MiB will have to be decompressed to seek to
any byte. The position of the member that needs to be decompressed can be computed directly.

If you don't have access to plzip, you can achieve the same result using the split(1) utility together
with regular lzip.

$ split -b 16m disk.img disk.img.part
$ lzip disk.img.part*
$ cat disk.img.part*.lz > disk.img.lz

Do not just use lzip with the --member-size option, since that option refers to member size, not data
block size.

Multi-member files with variable-size data blocks
Some ways of creating multi-member lzip files, such as the --member-size option or the tarlz(1) utility,
result in data blocks of varying sizes. In this case, seeking requires a binary search on the member
index and random access takes logarithmic time.

PARAMETERS

       lzip-max-block=SIZE
           The maximum block size that the filter will read.  The filter will refuse to read lzip files that
           contain any data block larger than this size.  This value refers to size of the uncompressed block,
           not the size of the member holding it.

           This parameter is optional.  If not specified it defaults to 512M.

       lzip-max-depth=N
           Maximum number of blocks stored in the LRU block cache.

           This parameter is optional.  If not specified it defaults to 8.

           The filter may allocate up to maximum block size in file × maxdepth bytes of memory per connection.

FILES

       $filterdir/nbdkit-lzip-filter.so
           The filter.

           Use "nbdkit --dump-config" to find the location of $filterdir.

VERSION

       "nbdkit-lzip-filter" first appeared in nbdkit 1.42.  It is derived from "nbdkit-xz-filter" which first
       appeared in nbdkit 1.10.

AUTHORS

       Jan Felix Langenbach

       Richard W.M. Jones

COPYRIGHT

       Copyright Jan Felix Langenbach

       Copyright Red Hat

LICENSE

       Redistribution and use in source and binary forms, with or without modification, are permitted provided
       that the following conditions are met:

       •   Redistributions of source code must retain the above copyright notice, this list of conditions and
           the following disclaimer.

       •   Redistributions in binary form must reproduce the above copyright notice, this list of conditions and
           the following disclaimer in the documentation and/or other materials provided with the distribution.

       •   Neither the name of Red Hat nor the names of its contributors may be used to endorse or promote
           products derived from this software without specific prior written permission.

       THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
       INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
       PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
       INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
       SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
       ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
       OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
       DAMAGE.