Provided by: jigdo-file_0.8.1-1_amd64 bug

NAME

       jigdo-file  -  Prepare  files  for  Jigsaw  Download  (distribution of huge files, e.g. CD
       images).

SYNOPSIS

       jigdo-file   COMMAND
        [ --image=cdrom.iso ] [ --jigdo=cdrom.jigdo ] [ --template=cdrom.template ] [ --force ] [
       MORE OPTIONS ] [ FILES ... | --files-from=f ]
        Common COMMANDs: make-template, make-image, verify

DESCRIPTION

       Jigsaw  Download,  or  short  jigdo,  is  a  scheme developed primarily to make it easy to
       distribute huge filesystem images (e.g.  CD  (ISO9660)  or  DVD  (UDF)  images)  over  the
       internet,  but  it could also be used for other data which is awkward to handle due to its
       size, like audio/video files or large software packages.

       jigdo tries to ensure that the large file (always called image from now on) is  downloaded
       in  small  parts which can be stored on different servers. People who want to download the
       image do so by telling the jigdo(1) (NOT IMPLEMENTED YET) download  tool  to  process  one
       `.jigdo'  file;  using it, jigdo downloads the parts and reassembles the image. jigdo-file
       is used to prepare the files for download.

       What makes jigdo special is that the parts that are used to reconstruct the image can have
       any  size  and content - they only need to be contained in a contiguous region anywhere in
       the image.

       For example, if you wish to distribute an ISO9660 image which contains a  snapshot  of  an
       FTP  server,  you  can instruct jigdo-file to prepare the download data in such a way that
       when people use jigdo to download the image, jigdo actually fetches the  individual  files
       from  the  FTP  server  and  assembles  them into an exact copy of your image - during the
       download! (If the image is not a filesystem dump, you can use split(1) to create the small
       parts that the image will be reassembled from.)

       You are completely free to choose where the individual parts of the image are stored: They
       may  be  in  entirely  different  directories  on  different  servers  (e.g.  because   of
       storage/bandwidth  constraints),  but  this  is  invisible  to the people downloading your
       image. The information about available servers only needs to be added to the `.jigdo' file
       by you before distributing it.

       The  `DETAILS' section below contains technical details on how jigdo works. The `EXAMPLES'
       section lists a number of common scenarios and may help you to get an idea of  what  jigdo
       is useful for.

OPTIONS

       Many  options  are specific to a particular COMMAND; the ones below are general or used by
       several commands. Further options are listed  below  with  the  individual  commands.  All
       options  are  silently  ignored if they are not applicable to the current command. For any
       BYTES parameters to options, you can append one of the letters `k',  `M'  or  `G'  to  the
       amount you specify, to indicate kilobytes, megabytes or gigabytes.

       -h --help
              Output short summary of commands and options.

       -H --help-all
              Output complete summary of commands and options.

       -v --version
              Output program version.

       -i --image=cdrom.iso
              Specify location of the file containing the image. The image is the large file that
              you want to distribute.

       -j --jigdo=cdrom.jigdo
              Specify location of the Jigsaw Download description  file.  The  jigdo  file  is  a
              human-readable file generated by jigdo-file, to which you add information about all
              the servers you are going to upload the files to.  jigdo will download this file as
              the first step of retrieving the image.

       -t --template=cdrom.template
              Specify  location  of the image `template' file. The template file is a binary file
              generated by jigdo-file, it contains information on how to reassemble the image and
              also (in compressed form) all the data from the image which was not found in any of
              the parts.

              Depending on the command, each of these three files is used  sometimes  for  input,
              sometimes for output. If the file is to be used for output for a particular command
              and the output file already exists, jigdo-file exits with an error, unless  --force
              is present.

              In  most  cases,  you  will  only  need to specify one out of -i -j -t, because any
              missing filenames will be deduced from the one you specify. This is done  by  first
              stripping  any  extension  from  the  supplied  name and then appending nothing (if
              deducing --image), `.jigdo' or `.template'.

       -r --report=default|noprogress|quiet|grep
              Control how verbose the program is, and what format the output has:  noprogress  is
              the  same as default except that no `x% done' progress messages are printed.  quiet
              restricts the output to what is absolutely necessary, mostly error messages.   grep
              is  only different from default for the make-template command: It enables output in
              a simple `<offset> <file>' format which is useful when searching for  binary  files
              in other binary files.

       -f --force
              Overwrite existent output files without complaining.

       --no-force
              This is the default. Refuse to overwrite existent output files.

       -c --cache=jigdo-cache.db
              jigdo-file  usually needs to read the entire contents of all the FILES you specify.
              If you use it repeatedly (e.g. because you make a new CD  image  available  daily),
              caching  the  file information will increase the program's speed significantly. The
              cache file is automatically created if it is not yet present. Data is usually  both
              read from and written to it.

       --no-cache
              This is the default. Do not use a cache.

       --cache-expiry=SECONDS
              Set  maximum age of cache entries. Any entries older than this will be removed from
              the cache. The default is 30 days. You can append one of the letters `h', `d', `w',
              `m',  `y'  to  denote hours, days, weeks, months or years, respectively. A value of
              `0' or `off' disables expiry, so that all entries will stay in the  cache  forever.
              See the section `CACHE FILES' below for more information.

       --readbuffer=BYTES
              Set  size  of  internal  buffers.  The  default  is 128k - if you have a fast disc,
              increasing this value may make jigdo-file faster, but in general,  changing  it  is
              not necessary.

       -C --checksum-algorithm=ALGO
              Choice  of  checksum  algorithm  to  use in describing the image and matched diles.
              Valid options are md5 and sha256. sha256 is a more secure algorithm, but will  make
              the  template file and jigdo file slightly larger. It will also create template and
              jigdo files that are not compatible with older versions  of  jigdo,  before  0.8.0.
              Default is currently md5, but this may change in future.

       --checksum-block-size --md5-block-size=BYTES
              Uninteresting  internal  parameter.   Set  size  of  blocks  into  which  files are
              subdivided. The default is 128k. If you change it, any cache file will have  to  be
              regenerated.  Internally, jigdo-file may choose to use a slightly larger or smaller
              value.

       -T --files-from=file
              Read file and directory names from the specified file.  If file is `-', read  names
              from  standard  input.  Each  line in the file is taken as a name, so the names may
              contain spaces, but not newline characters. An empty line causes jigdo-file to stop
              reading from the file.

              find(1)  is  a  powerful tool for generating file lists, but make sure to use `find
              -type f' if possible - otherwise, if you instruct find to output  both  a  filename
              and a symlink to that filename, jigdo-file will read the file contents twice.

       --hex  Output  checksums  in hexadecimal instead of Base64-like format. This should not be
              used with the make-template command, because the resulting `.jigdo'  file  violates
              the `.jigdo' file format. Its intended use is to make jigdo-file more interoperable
              with other Unix shell utilities like md5sum(1).

       --no-hex
              This is the default. Use jigdo's own Base64-like encoding of checksums.

       --debug[=help|=all|=UNIT,~UNIT... ]
              Switch on or off debugging output. Just `--debug' is equivalent  to  `--debug=all'.
              The  argument is a comma-separated list of unit names for which debugging output is
              to be enabled, or disabled if the name is preceded by `~'. The special  name  `all'
              means  all units. By default, debugging output is switched off except for the units
              `assert' and `general'. The exact list of available units for which  debugging  can
              be  switched  on depends on whether jigdo was compiled with debugging support - the
              list can be printed with `--debug=help'.

       FILES  Names of files or directories to use  as  input.  These  are  the  parts  that  are
              contained  in  the  image.  In  case  one  of the names is a directory, the program
              recursively scans the directory and adds all files contained  in  it.  While  doing
              this, it follows symbolic links, but avoids symlink loops.

              If one of the filenames starts with the character `-', you must precede the list of
              files with `--'. A value of `-' has no special meaning in this list, it stands  for
              a file whose name is a single hyphen.

COMMANDS

       The command name is the first non-option argument passed to jigdo-file. Most commands have
       short abbreviations as well as long names. The short command names should not be  used  in
       scripts - there may be incompatible changes to them in the future!

   MAKE-TEMPLATE, MT
       Reads image and FILES, creates `.jigdo' and `.template'. This is the main functionality of
       jigdo-file.

       It is possible to specify both --image=- and --files-from=-. In this case, first the  list
       of  files  is  read  from  standard  input  until an empty line is encountered. Everything
       following it is assumed to be the image data. This can be useful if you use mkisofs(1)  or
       similar  programs  that  can  output  the complete image on their standard output, because
       there is no need to store the image on disc temporarily.

       If a FILES argument contains the characters `//'  (Unix)  or  `\.\'  (Windows),  this  has
       special  meaning.  In  the final jigdo file that users will download, each of the parts is
       referenced in the `[Parts]' section with a URI of  the  form  `Label:some/filename'.  (See
       `FORMAT  OF .JIGDO FILES' below for a detailed description.) The `[Servers]' section gives
       a   mapping   of   labels   to   servers   on    the    internet,    with    lines    like
       `Label=http://myserver.org/jigdofiles/'.  Using  this  information,  jigdo will create the
       final  download  URI   for   the   part,   `http://myserver.org/jigdofiles/some/filename'.
       Specifying  `//'  (or  `\.\') in a file or directory name serves to `cut off' the names at
       the right directory level. For example,  if  the  Unix  path  of  one  of  your  FILES  is
       `/path/some/filename',  you can tell jigdo-file to cut off after the `/path' by passing it
       the argument `/path//some/filename', or `/path//' if you want the whole directory scanned.
       The path names need not be absolute; `somedirectory//' is also possible.

       --label Label=/path
              Specify  a name to use as the label name for a path on disc. (Influences the output
              jigdo file.) If you used `//' in the FILES arguments as described above, jigdo-file
              will  by  default pick label names automatically (`A', `B' etc.). With this option,
              you can give labels more meaningful names. Note that the label name  will  only  be
              used if one or more FILES begin with `/path//'.

              Try  to  use label names that start with uppercase characters, to disambiguate them
              clearly from protocol names like `http', `ftp'.

       --uri Label=http://some.server.org/
              By default, using  --label  as  described  above  will  cause  lines  of  the  form
              `Label=file:/path/'  to  be  written to the `[Servers]' section of the output jigdo
              file.  If  you  want  to  override  the  `file:'  URI  so  that  the   line   reads
              `Label=http://some.server.org/',  you  can  do  so  by  specifying --uri along with
              --label. Giving just --uri Label=... without the  corresponding  --label  Label=...
              has  no  effect,  and  even  if  you  specify  both,  an entry is only added to the
              `[Servers]' section if the label is referenced by at least one `[Parts]' entry.

              The supplied value is not quoted by the program; if it contains characters such  as
              space  or  any of the characters #"'\ then you must quote it.  (Under Unix, you may
              need to quote the value twice to also protect it from the shell, e.g. \\\\ or  '\\'
              to get a single backslash in the URI.)

              The  mapping  specified with an --uri option is ignored if it is already present in
              the output jigdo file.

              Users of the Windows version may notice  that  the  `\'  directory  separators  are
              converted  into `/' in the `file:' URIs that are generated by default. This is done
              to increase cross-platform compatibility of `file:' - the print-missing command  of
              the Windows version will automatically re-convert the characters when it prints the
              URIs. In case you supply your own `file:' URIs under Windows using --uri, you  must
              also exchange `/' and `\'.

       -0 to -9
              Set  amount of compression in the output template file, from -0 (no compression) to
              -9 (maximum compression). The default is -9, which can make the template generation
              quite slow. By default, the compression algorithm used is the same as for gzip(1).

       --gzip and --bzip2
              Choose  between  the  gzip  and  bzip2 compression algorithms. The default is gzip.
              Bzip2 usually gives a better compression ratio, but  compression  is  significantly
              slower than with gzip.

       --min-length=BYTES
              Set  minimum  length  of  a  part  for jigdo-file to look for it in the image.  The
              default is 1k. Parts smaller than this will never be found in the image,  so  their
              data will be included in the template file. The search algorithm used requires such
              a minimum length, otherwise template generation could become extremely slow. If you
              know  for  sure  that  all  your  FILES  are  larger than a certain amount, you can
              increase jigdo-file's speed slightly by specifying the  amount  with  this  option.
              There  is a hard-wired absolute minimum of 256 bytes - anything lower will silently
              be set to 256.

       --merge=FILE
              Include the contents of FILE in the output `.jigdo' file. The file can contain data
              which  you want added to the output (for example, a `[Servers]' section with a list
              of your servers as entries), or it can be the jigdo file output by an  earlier  run
              of jigdo-file.

              It  is possible to specify the same file for input with --merge and for output with
              --jigdo. However, you will also need to use --force to make the  program  overwrite
              the  old  version of the jigdo file with the new one.  FILE can be `-' for standard
              input.

              When adding new information to the supplied file, jigdo-file will  not  insert  new
              lines  into  the  `[Parts]'  section  if  an  entry  for the same checksum (but not
              necessarily with the same URI!) already exists, and it will not  insert  new  lines
              into the `[Servers]' section if a completely identical entry already exists.

              When reading in the existing FILE, the behaviour is slightly different: The program
              preserves entries in the `[Parts]' section with identical checksum,  but  different
              URIs.  For  completely identical entries (same checksum and URI), only one entry is
              preserved  and  the  duplicates  are  removed.  The  `[Servers]'  section  is  left
              untouched.

       --image-section
              This  is the default. Causes jigdo-file to add an `[Image]' section to the `.jigdo'
              file.

              As an exception, a new `[Image]' section is not added if you use  --merge  and  the
              file  to  merge  contains  an  `[Image]' section with a line which reads `Template-
              MD5Sum=' (end of line after the `='). In this case, the generated  template  data's
              MD5  checksum  value  is just added after the `=' of the first line of this form in
              the file - no whole new `[Image]' section is appended.  This  behaviour  is  useful
              because  it  allows  you  to  pass  via --merge an `[Image]' section with arbitrary
              content and then have the MD5  checksum  automatically  added  by  jigdo-file.  The
              section  `FORMAT  OF .JIGDO FILES' below explains the `[Image]' section contents in
              greater detail.

       --no-image-section
              Do not include an `[Image]' section in the `.jigdo'  file.  You  need  to  add  one
              yourself  if you use this option. However, doing that is not easy (you also need to
              add a `Template-MD5Sum' line with the correct checksum, or jigdo will complain), so
              use of this option is discouraged.

       --servers-section
              This is the default. Causes jigdo-file to add a `[Servers]' section to the `.jigdo'
              file. This default section uses `file:' URIs, which allows for immediate reassembly
              of  the image from the local filesystem, and is also useful if you want to edit the
              file manually and replace the `file:' URIs with other URIs.

       --no-servers-section
              Do not add a `[Servers]' section at the end of the `.jigdo' file.  Useful  e.g.  if
              you are going to append the section with a script.

       --match-exec=SHELLCOMMAND
              Whenever  a  file  is  found  in  the image, execute the supplied command string by
              passing it to a shell.  jigdo-file sets up a number of environment  variables  with
              information  about  the  file match. For example, if the file `/path//a/b/file' was
              found in the image and `Label:a/b/file' is going to  be  written  to  the  `.jigdo'
              file:

              • LABEL="Label" - Name of the label for the file. The example assumes that `--label
                Label=/path' was specified by you.  In the absence of such an option, LABEL  will
                be set but empty.

              • LABELPATH="/path/"  - The path corresponding to the label, or in other words, the
                prefix of the matched file's path that will not appear  in  the  output  `.jigdo'
                file. Is set even without any `--label' option present.  Ends with a slash.

              • MATCHPATH="a/b/"  -  The  rest  of  the path, without the leafname of the matched
                file. Is either empty or ends with a slash.

              • LEAF="file" - The leafname of the matched file.

              • MD5SUM="lNVdUSqbo2yqm33webrhnw" - The md5sum of the matched file, in  Base64-like
                format.

              • SHA256SUM="QXBJ8VZKeh0NXH0uOhdhgguPPE5tT1wvYO27sLx9Fsc"  -  The  sha256sum of the
                matched file, in Base64-like format.

              • FILE="/path//a/b/file" - For convenience, the complete  path  of  the  file.  The
                variable is always set to $LABELPATH$MATCHPATH$LEAF.

       Please  be  careful  to  correctly  quote the string passed to this option, otherwise your
       supplied command will not work with filenames that  contain  spaces.  As  an  example,  to
       create  a  backup  of  hard links to the matched files, use the following option: --match-
       exec='mkdir -p "${LABEL:-.}/$MATCHPATH" && ln -f "$FILE" "${LABEL:-.}/$MATCHPATH$LEAF"'

       By default, no command is executed. Use --match-exec="" to remove a command  string  which
       was set with an earlier use of this option.

       --greedy-matching
              This  is  the  default.  Imagine that your image contains a .tar file which in turn
              contains another file x, and that you provide both the .tar and the files inside it
              on  the  command line. When jigdo-file scans the image, it encounters the beginning
              of the .tar file, and then the file x.

              At this point, a decision must be made: Should the smaller file x  be  recorded  as
              matched, or should it be ignored in favour of the larger (and thus better) match of
              the .tar file? Unfortunately, at this point it is  not  clear  whether  there  will
              actually  be a full match of the .tar, so by default, the program prefers the small
              match.

       --no-greedy-matching
              In the case where a large partial match is present and a  shorter  match  has  been
              confirmed, ignore the small match. (See the option above.)

   MAKE-IMAGE, MI
       Reads  `.template'  and  FILES, creates image (or `imagename.tmp'). Provides a rudimentary
       way of reassembling images - jigdo is usually better suited for  this  task.  However,  in
       contrast to jigdo, no `.jigdo' file is required.

       If  the  image  is to be written to a file (and not to standard output), it is possible to
       create the image in several steps, with several invocations of `jigdo-file make-image', as
       follows:  You  first  invoke jigdo-file, specifying as many files as are available at this
       time. The program scans the files, and those that are contained in the image are copied to
       a temporary file, whose name is formed by appending `.tmp' to the image filename.

       For  all further files which could be parts of the image, you repeat this process. As soon
       as all parts are present, the temporary file will be truncated slightly  (to  delete  some
       administrative  data  that  jigdo-file  appends at the end) and renamed to the final image
       name. The possibility of reassembling the image in several steps is especially useful  for
       gathering files from removable media, e.g. several older CDs.

       Scripts  using  make-image  can  detect whether image creation is complete by checking the
       exit status: 0 signals successful creation, whereas 1 means that more  files  need  to  be
       supplied.  Other errors result in an exit status of 2 (`recoverable', e.g. file not found)
       or 3 (non-recoverable, e.g.  write error).

       --check-files
              This is the default. Whenever any  part  is  copied  to  the  image,  re-check  its
              checksum  against  the  checksum stored in the template. It is recommended that you
              leave this switched on, even if it slows down image creation a bit.

       --no-check-files
              Do not check files' checksums when copying them to the image. This  can  be  safely
              used  when  no  cache  file  is used (which means that files will be written to the
              image immediately after being scanned) or the whole image is checked later with the
              verify command.

   PRINT-MISSING, PM
       Reads `.jigdo', `.template' and (if present) `imagename.tmp', outputs a list of URIs still
       needed to completely reassemble the image.

       Together with the make-image command, this provides most of the functionality of jigdo  on
       the command line.

       For  each  part  that is not yet present in the temporary image file, the file checksum is
       looked up in the `[Parts]' section of the jigdo file. Any label in the corresponding entry
       is then expanded according to the label definitions in the `[Servers]' section and printed
       on standard output. jigdo allows you to specify several  alternative  locations  for  each
       label  in  this section, but print-missing will only output the first one for each missing
       part.

       If the checksum cannot be found in the `[Parts]' section (this Should  Not  Happen  unless
       you  deleted  that  section),  a  lookup  is  instead  made  for  `MD5Sum:<checksum>'  and
       `SHA256Sum:<checksum>', just like with jigdo. (Thus,  if  you  want  to  get  rid  of  the
       `[Parts]' section, you can do so if you rename each part to its own checksum.)

       --uri Label=http://some.server.org/
              Override  the entries in the `.jigdo' file for any label with a URI of your choice.
              With the example above, a `[Parts]' entry of `Label:some/filename' will  cause  the
              line `http://some.server.org/some/filename' to be printed.

              The  supplied value is not quoted by the program; if it contains characters such as
              space or any of the characters #"'\ then you must quote it.  (Under Unix,  you  may
              need  to quote the value twice to also protect it from the shell, e.g. \\\\ or '\\'
              to get a single backslash in the URI.)

   PRINT-MISSING-ALL, PMA
       Just like print-missing, this command outputs a list of URIs still  needed  to  completely
       reassemble  the  image. However, all alternative download locations are printed instead of
       just one. In the output, the URIs for a file are separated from  other  files'  URIs  with
       blank lines. The --uri option has the same effect as for print-missing.

   VERIFY, VER
       Reads  image  (presumably  generated  with make-image) and `.template', checks for correct
       checksum of image.

       The template data does not only contain checksums of the individual parts, but also of the
       image  as  a  whole.   make-image already performs a number of internal checks, but if you
       like, you can additionally check the image with this command.

   SCAN, SC
       Reads all the FILES and enters them into the cache, unless they are  already  cached.  The
       --cache option must be present for this command.

       --no-scan-whole-file
              This is the default. This only causes the first --checksum-block-size bytes of each
              file to be read. If the cache is used later by jigdo-file make-image, the  rest  of
              the file will be read once these first bytes are recognized in the input image.

       --scan-whole-file
              Immediately read the entire file contents and store them in the cache.

   MD5SUM, MD5
       Reads  all the FILES and prints out MD5 checksums of their contents. This command is quite
       similar to md5sum(1), except that the checksum is output in the Base64-like encoding which
       is also used elsewhere by jigdo-file.

       The  FILES arguments are processed in the same way as with the other commands, which means
       that recursion automatically takes place for any arguments that are directories, and  that
       symbolic  links  are  not  listed  except when the file(s) they point to are not reachable
       directly.

       In the checksum list printed on standard output, only the part of the  filename  following
       any  `//'  (or  `\.\' on Windows) is printed. Any --cache will be used for querying files'
       MD5 checksums and/or writing the checksums of scanned files.

   SHA256SUM, SHA256
       Reads all the FILES and prints out SHA256 checksums of their  contents.  This  command  is
       quite  similar  to  sha256sum(1),  except  that  the checksum is output in the Base64-like
       encoding which is also used elsewhere by jigdo-file.

       The FILES arguments are processed in the same way as with the other commands, which  means
       that  recursion automatically takes place for any arguments that are directories, and that
       symbolic links are not listed except when the file(s) they  point  to  are  not  reachable
       directly.

       In  the  checksum list printed on standard output, only the part of the filename following
       any `//' (or `\.\' on Windows) is printed. Any --cache will be used  for  querying  files'
       SHA256 checksums and/or writing the checksums of scanned files.

   LIST-TEMPLATE, LS
       Reads  a  `.template' file and outputs low-level information about the image and all parts
       contained in it, including offset, length and checksum.

       You can also use this command with temporary image files  (by  specifying  something  like
       --template=imagename.tmp) - in that case, the output also distinguishes between parts that
       have been written to the image and parts that haven't.

       The exact output format may change incompatibly  between  different  jigdo  releases.  The
       following  different  types  of  lines  can  be output. `have-file' only occurs for `.tmp'
       files, indicating a file that has already been successfully written to the temporary file:

       in-template offset-in-image length
       need-file-md5 offset-in-image length file-md5sum filestart-rsyncsum
       have-file-sha256 offset-in-image length file-sha256sum filestart-rsyncsum
       image-info-sha256 image-length rsyncsum-size image-sha1sum

DETAILS

       Jigsaw Download was created with the format of ISO9660 CD images in mind  -  however,  the
       following  also applies to many other filesystem formats, as well as to `tar' archives and
       uncompressed `zip' archives. A CD image  contains  both  information  for  organizing  the
       filesystem (header with disc name etc., ISO9660 directory data, data of extensions such as
       Joliet or RockRidge, zero padding) and  the  files  contained  on  the  CD.  An  important
       property that jigdo relies on is that each file is stored in one contiguous section of the
       image; it is not split into two or more parts.

       When jigdo-file is given a number of files that might be contained in an image, it detects
       whether  any  of the files are present using a `rolling checksum' inspired by the one used
       by rsync(1). The resulting data is written to the `.template' file: If a  section  of  the
       image could not be matched (e.g. it was directory information), the data is compressed and
       written directly to the template. However, if a matching  file  was  found,  its  data  is
       omitted  from  the  template.  Instead, only a reference (an MD5 or SHA256 checksum of the
       file) is inserted in the template.

       Note that the template data only contains binary data, it does not contain  any  filenames
       or  URIs, since it cannot be easily edited in case any of these values need to be changed.
       All that information is stored in the `.jigdo' file, a text file to which you can add URLs
       for  your server(s). The jigdo file provides a mapping for each MD5/SHA256 checksum to one
       or more alternative download locations for the corresponding part.

       Apart from the mapping of MD5/SHA256 sums to URIs, the jigdo file  also  contains  an  URI
       pointing  to  a download location for the template file. This way, the jigdo download tool
       only needs to be given one URI (that of the `.jigdo' file) to  be  able  to  download  and
       reassemble the complete image.

FORMAT OF .JIGDO FILES

       The  overall  format  of  `.jigdo' files follows that of `.ini' files, as also used by the
       Gnome and KDE projects for some data. The file is organized into sections, each  of  which
       is  preceded  by  a line reading `[Sectionname]'. Within each section, lines have the form
       `Label=Value'. Such lines are also called `entries' below. All `.jigdo' files use UTF-8 as
       their character encoding.

       Comments  are  introduced  with  the  `#'  character  and  extend  to the end of the line.
       Whitespace is ignored at line start and end as well as to the left and  right  of  section
       names  and  the  `=' in entries. Furthermore, the jigdo utilities split up the text of the
       entry value (i.e. the part after the `=') into whitespace-separated words, much  like  the
       Unix  shell.  Single  ''  and  double  ""  quotes  can  be  used to prevent that e.g. URIs
       containing whitespace are split apart. Similarly, characters  with  special  meaning  (the
       characters  '"#\  and space/tab) must be quoted with \ to appear in the value. As with the
       shell, there is a difference between ' ' and " ": Within  '  ',  the  characters  "#\  and
       whitespace  lose their special meaning and become ordinary characters, whereas within " ",
       only the characters '# and whitespace  lose  their  special  meaning  -  in  other  words,
       backslash escapes still work inside " ", but not ' '.

       `.jigdo'  files  can  optionally  be  compressed  with  gzip(1). jigdo-file always outputs
       uncompressed  files,  which  you  can  compress  yourself.   jigdo-lite  supports   single
       uncompressed and compressed files.

       (Behaviour  which  may  change  in  the  future and which should not be relied upon: jigdo
       additionally supports any number of concatenated plaintext and gzipped parts in the  files
       -  for  example,  you  can  compress  a  `.jigdo'  file  and then add a couple of lines of
       uncompressed data to the end.)

       In all cases, the `.gz' extension should be removed from the filename  -  the  tools  will
       determine automatically from the file contents whether a file is compressed or not.

       Below is a description of the individual section names used by jigdo.

   JIGDO SECTION
       [Jigdo]
       Version=1.1
       Generator=jigdo-file/1.0.0

       Information  about  the  version  of  the  jigdo  file  format  used, and the program that
       generated it. There should be one such section per `.jigdo' file.

   IMAGE SECTION
       [Image]
       Filename="filename for saving on user's disc"
       Template="URI where to fetch template file"
       Template-MD5Sum=OQ8riqT1BuyzsrT9964A7g
       Template-SHA256Sum=MVJIxGifflRF9K8ERdbqoyns4Ucw9Xy1ubdnE6CtMbo
       ShortInfo=single-line description of the image (200 characters max.)
       Info=long description (5000 characters max.)

       The value for the `Template' entry can be either an URL (absolute or relative to  the  URL
       of  the jigdo file) or a string of the form `Label:pathname' (UNIMPLEMENTED), as described
       below.

       The `Template-MD5Sum' and/or `Template-MD5Sum' entries are added by jigdo-file and specify
       the  checksum of the generated `.template' file. It is used by jigdo to detect cases where
       the downloaded template data is corrupted or belongs to a different image.

       Unlike other entry values, the values of the `ShortInfo' and `Info' entries are not  split
       up into words, instead all quoting is preserved.

       The  value of the `Info' entry is special in that jigdo(1) can optionally parse XML markup
       it contains. If the markup has errors such as unbalanced/unsupported tags, the  string  is
       displayed  literally,  without  XML  parsing.  Supported  tags are <b></b> (bold), <i></i>
       (italic), <tt></tt> (typewriter font), <u></u>  (underline),  <big></big>  (larger  font),
       <small></small>  (smaller  font)  and  <br/>  (linebreak). Supported entities include &lt;
       (`<'), &gt; (`>') and &amp; (`&'). Note that the whole `Info' entry must be on one line in
       the jigdo file.

       This  section may occur multiple times, but all except the first one will be ignored. This
       is useful e.g. when creating a `.jigdo' file  for  a  DVD  image  when  you  already  have
       `.jigdo'  files  for CDs with the same content: You can simply `[Include]' (see below) the
       CDs' jigdo files at the end of the DVD jigdo file, after its `[Image]' section.

   PARTS SECTION
       [Parts]
       xJNkjrq8NYMraeGavUpllw=LabelA:part0
       GoTResP2EC6Lb_2wTsqOoQ=LabelA:part1
       kyfebwu6clbYqqWUdFIyaw=LabelB:some/path/part2
       -J9UAimo0Bqg9c0oOXI1mQ=http://some.where.com/part3

       All lines in the section, which provides the mapping from checksums to URIs, have the same
       format:  On the left side of the `=' the checksum (encoded with a Base64-like encoding) is
       given, and on the right a string corresponding to the part with this  checksum;  either  a
       complete  URI or a string of the form `Label:pathname', which is expanded into one or more
       URIs by looking up the definition(s) for the Label in the `[Servers]' section.

       In case a particular checksum cannot be found in  any  `[Parts]'  section  by  jigdo,  the
       program will perform a lookup for `MD5Sum:<checksum>' and `SHA256Sum:<checksum>', e.g. for
       `MD5Sum:xJNkjrq8NYMraeGavUpllw' if you deleted the line for `part0' above.

       A checksum appearing  multiple  times  in  this  section  indicates  alternative  download
       locations for the part.

       There  may  be  any number of `[Parts]' sections in the file; they are all considered when
       looking up checksums.

       jigdo-file always puts the `[Parts]'  section  at  the  end  of  the  file,  and  it  even
       rearranges  any file specified with --merge to have only one such section at the end. This
       is done to allow jigdo to display the information from the  `[Image]'  section  while  the
       rest of that file is still being downloaded.

   SERVERS SECTION
       [Servers]
       LabelA=http://myserver.org/
       LabelA=ftp://mirror.myserver.org/
       LabelB=LabelC:subdirectory/
       LabelC=http://some.where.com/jigdo/

       All  lines  in  the  section,  which  provides  the  mapping  from server labels to server
       locations, have the same format: On the left side of the `=' the label name is given,  and
       on the right the value to expand the label name to.

       A  label  name  appearing  multiple  times  in this section indicates alternative download
       locations for the parts that use the label in the `[Parts]' section. This  notation  makes
       it very easy to add mirrors to the jigdo file.

       As  shown by the example above, the label values may themselves reference other labels. In
       this case, the entry `LabelB:some/path/part2' in the  `[Parts]'  section  will  expand  to
       `http://some.where.com/jigdo/subdirectory/some/path/part2'.     Loops    in    the   label
       definitions result in undefined behaviour and must be avoided.

       There may be any number of `[Servers]' sections in the file; they are all considered  when
       looking  up  labels. Either of `[Parts]' or `[Servers]', but not both, can be omitted from
       the jigdo file.

   COMMENT SECTION
       [Comment]
       Any text, except that lines must not begin with `['.

       All text following a `[Comment]' or `[comment]' line is ignored, up to the next line  with
       a section label.

   INCLUDE DIRECTIVE
       [Include http://some.url/file.jigdo]

       Lines  of  this  form  cause  the content of the specified jigdo file to be downloaded and
       parsed just like the main jigdo file. The effect will be the same as copying the  included
       file's  contents  into  the  file  which  contains  the include directive. (Exception: Any
       relative URLs are always resolved using the URL of the `.jigdo' file  that  contains  that
       relative URL.)

       The  URL  argument  can  be  an absolute or relative URL.  Relative URLs are assumed to be
       relative to the URL of the jigdo file which contains the include directive.  Includes  can
       be  nested,  but it is an error to create a loop of include directives. It is not possible
       to use URLs of the form `Label:pathname'.

       The URL cannot be quoted with "". Any `]' characters in the argument must  be  escaped  as
       `%5D', and any spaces as `%20'.

       Include directives are only supported by jigdo, they are ignored by jigdo-lite.

       An  include directive terminates any previous section, but it does not start a new one. In
       other words, a new section must always be started after the include line, jigdo  does  not
       allow normal entries to appear below the `[Include]'.

CACHE FILES

       Any  file  specified  with the --cache option is used to store information about the FILES
       presented to jigdo-file. When querying the cache, a file is considered unchanged (and  the
       cached  data is used) only if filename, file size and last modification time (mtime) match
       exactly. For the filename match, not the entire file name  is  used,  but  only  the  part
       following  any  `//',  so that any changes to the part before the `//' will not invalidate
       the cache.

       Old cache entries are removed from the cache if they have not been read from or written to
       for  the amount of time specified with --cache-expiry. Entries are not immediately removed
       from the cache if the file they refer to no longer exists -  this  makes  it  possible  to
       cache information about files on removable media.

       Cache expiry only takes place after jigdo-file has done its main work - if any old entries
       are accessed before expiry takes place, they will be kept.  For example, if the program is
       run  using  the  default  expiry  time  of 30 days, but accesses a cache file with entries
       generated 2 months ago, then entries in that cache will  be  considered,  and  only  those
       cache entries that were not needed during the program run will be expired.

       Due  to  a  peculiarity  of  the  underlying  database library (libdb3), cache files never
       shrink, they only grow. If a large number of entries was expired from your cache file  and
       you  want it to shrink, you can either just delete it (of course then everything will have
       to be regenerated) or use the utilities  accompanying  libdb3  to  dump  and  restore  the
       database, with a command like `db3_dump old-cache.db | db3_load new-cache.db'. For Debian,
       these programs are supplied in the package `libdb3-util'.

       If a different --md5-block-size is specified, the entire  file  needs  to  be  re-read  to
       update  its  cache  entry.  If  a  different  --min-length  is  specified,  only the first
       `md5-block-size' bytes of the file need to be re-read.

EXAMPLES

   PREPARING YOUR CD IMAGE FOR DISTRIBUTION
       You have created a CD image `image.iso' from some of the files  stored  in  the  directory
       `/home/ftp'  on  your  harddisc, which is also available online as `ftp://mysite.org'.  As
       you don't want to waste space by effectively hosting the same data twice (once as files on
       the  FTP server, once inside the image), and you are fed up with users' downloads aborting
       after 200MB and their restarting the download dozens of times, you decide  to  use  jigdo.
       How do you prepare the image for download?

       In fact, only one command is necessary:

              jigdo-file     make-template     --image=image.iso    --jigdo=/home/ftp/image.jigdo
              --template=/home/ftp/image.template  /home/ftp//  --label  Mysite=/home/ftp   --uri
              Mysite=ftp://mysite.org/

       People  can  now point jigdo at `ftp://mysite.org/image.jigdo' to download your image. The
       template file needs to be accessible as `ftp://mysite.org/image.template'.

       Note that nothing prevents  you  from  doing  the  same  for  an  FTP  server  that  isn't
       administrated  by  you  - in that case, you only need to host the `.jigdo' and `.template'
       files on your own server/homepage.

   PREPARING AN ARBITRARY LARGE FILE FOR DISTRIBUTION
       We assume that you have a large file that is not a filesystem, e.g. `movie.mpeg'.  Because
       of space problems, you want to distribute the data on two servers.

       In  this  case,  the  parts  of the image need to be generated artificially with the split
       command. For example, to create chunks of 4MB each, use `split  -b  4m  movie.mpeg  part'.
       Copy  the  resulting  files  `partXX'  into  two  directories `1' and `2' that you create,
       according to how you want the files distributed between  the  servers.  Next,  create  the
       jigdo  and  template files with `jigdo-file make-template --image=movie.mpeg 1// 2//'. You
       will need to edit the `.jigdo' file and provide the right URIs for the  two  servers  that
       you are going to upload the `partXX' files to.

   CUSTOMIZED VERSIONS OF IMAGES
       Because  it  is possible to assign a different URI for each part of an image if necessary,
       jigdo is very flexible. Only one example is the  possibility  of  customized  versions  of
       images:  Suppose  that someone is distributing a CD image, and that you want to make a few
       small changes to it and redistribute your own version. You download the `official.iso'  CD
       image  with  jigdo  (passing  it the URL of `official.jigdo'), write it to CD-R, make your
       changes (say, adding files from the `myfiles' directory on your harddisc) and produce your
       own  version,  `myversion.iso'.   Next,  you  instruct  jigdo-file to create the jigdo and
       template files for your modified image, using the command

              jigdo-file  make-template  --image=myversion.iso  /mnt/cdrom/   myfiles//   --label
              My=myfiles/ --uri My=http://my.homepage.net/ --merge=official.jigdo
       while `official.iso' is mounted under `/mnt/cdrom'. By using --merge, you have told jigdo-
       file to take the contents of `official.jigdo', add to  it  a  new  `[Image]'  section  for
       `myversion.iso'  and  write  the  resulting  jigdo  file  to  `myversion.jigdo'  -  so now
       `myversion.jigdo' offers two images for download, the original version and  your  modified
       version.  (If  you  do  not  want it to offer the official version, edit it and remove the
       `[Image]' section that lists `official.iso'.)

       Now you can upload the `.jigdo' file, the `.template' file and also the files in `myfiles'
       to  `http://my.homepage.net/'.   Thus,  for people to download your modified image, you do
       not need to upload the complete image contents to your web space, but only the changes you
       made!

       (In  case  you  only made very few changes, you could also omit the `myfiles' parameter in
       the command above, then all your changes end up in the new template file.)

   COMBINING MANY JIGDO-MANAGED IMAGES INTO ONE
       It is also no problem to combine data from several sources that use jigdo. For example, if
       of  five  different  and  unrelated  servers each one distributes a different CD image via
       jigdo, you can create a customized DVD image that contains the data from  all  these  CDs.
       When  people use jigdo to download your image, the individual files on the DVD are fetched
       from the same sources as the original CDs.

       Consequently, even though you will be distributing a 3.2GB file via your  web  space,  the
       actual  amount  of data that is stored on your server will only be in the order of several
       MBs.

BUGS

       For certain contents of one of the input files, most notably  a  sequence  of  zero  bytes
       longer  than --min-length at the start of the file and an area of zeros preceding the file
       data in the image, jigdo-file make-template may fail  to  find  the  file  in  the  image.
       Unfortunately,  this  restriction  cannot be avoided because the program could become very
       slow otherwise. If you use the --debug option,  all  instances  of  jigdo-file  discarding
       possible matches are indicated by lines containing the word `DROPPED'.

       In fact, not only all-zeroes files trigger this behaviour, but also files which contain at
       their start a long  sequence  of  short  identical  strings.  For  example,  both  a  file
       containing only `a' characters and one containing `abcabcabcabc...' are problematic.

SEE ALSO

       jigdo(1)  (NOT  YET  IMPLEMENTED),  jigdo-lite(1),  jigdo-mirror(1),  split(1)  (or  `info
       split'), find(1) (or `info find'), mkisofs(1), md5sum(1), sha256sum(1), jigit-mkimage(1)

AUTHOR

       Jigsaw Download <URL:http://atterer.org/jigdo/> was  written  by  Richard  Atterer  <jigdo
       atterer.org>,  to make downloading of CD ROM images for the Debian Linux distribution more
       convenient.

       Steve McIntyre <93sam@debian.org> picked up later development of jigdo after  Richard  had
       moved     on    -    see    the    git    repo    work    <URL:https://git.einval.com/cgi-
       bin/gitweb.cgi?p=jigdo.git;a=summary> or packages in Debian for more recent releases.

                                           Oct 31, 2021                             JIGDO-FILE(1)