Provided by: reprotest_0.7.26_all bug

NAME

       reprotest - Build packages and check them for reproducibility.

SYNOPSIS

       reprotest --help [<virtual_server_name>]
       reprotest [options] [-c <build_command>] <source_root> [<artifact_pattern>] [-- <virtual_server_args> [<virtual_server_args> ...]]
       reprotest [options] [-s <source_root>] <build_command> [<artifact_pattern>] [-- <virtual_server_args> [<virtual_server_args> ...]]

()                                                                                             ()

DESCRIPTION

       reprotest builds the same source code twice in different environments, and then checks the
       binaries produced by each build for differences. If any are found, then diffoscope(1)  (or
       if unavailable then diff(1)) is used to display them in detail for later analysis.

       See  the  COMMAND-LINE  EXAMPLES section further below to get you started, as well as more
       detailed explanations of all the  command-line  options.  The  same  information  is  also
       available in /usr/share/doc/reprotest/README.rst or similar.

       source_root|build_command
              The  first  argument is treated either as a source_root (see the -s option) or as a
              build-command (see the -c option) depending on what it  looks  like.  Most  of  the
              time,  this  should  "just work"; but specifically: if neither -c nor -s are given,
              then: if this exists as a file or directory and is not "auto", then this is treated
              as a source_root, else as a build_command.  Otherwise, if one of -c or -s is given,
              then this is treated as the other one. If both are given, then  this  is  a  syntax
              error and we exit code 2.

       artifact_pattern
              Build  artifact  to test for reproducibility. May be a shell pattern such as "*.deb
              *.changes".

       virtual_server_args
              Arguments to pass to the virtual_server, the first argument being the name  of  the
              server.  If  this itself contains options (of the form -xxx or --xxx), or if any of
              the previous arguments are omitted, you should put a "--" between  these  arguments
              and  reprotest's  own  options. Default: "null", to run directly in /tmp.  Choices:
              chroot, lxc, lxd, null, qemu, schroot, ssh

   options:
       --help [VIRTUAL_SERVER_NAME]
              Show this help message and exit. When given an  argument,  show  instead  the  help
              message for that virtual server and exit.

       -f CONFIG_FILE, --config-file CONFIG_FILE
              File to load configuration from. (Default: None)

   basic options:
       --verbosity VERBOSITY
              An integer. Control which messages are displayed (0: quiet (warning/error only), 1:
              info, 2: debug).

       -v, --verbose
              Like --verbosity, but given multiple times without arguments.

       --host-distro HOST_DISTRO
              The distribution that will run the tests (Default: None)

       -s PATH, --source-root PATH
              Root of the source tree, that is copied to the virtual server  and  made  available
              during  the  build.  If  a  file  is  given here, then its parent directory is used
              instead. Default: "." (current working directory).

       --source-pattern PATTERNS
              Shell glob pattern to restrict the files in <source_root> that are  made  available
              during  the build. Default: empty, i.e. copy the whole <source_root> directory with
              no restrictions.

       -c COMMANDS, --build-command COMMANDS
              Build command to execute. If this is "auto" then reprotest will guess how to  build
              the  given  source_root,  in  which case various other options may be automatically
              set-if-unset. Default: auto

       --store-dir DIRECTORY
              Save the artifacts  in  this  directory,  which  must  be  empty  or  non-existent.
              Otherwise,  the  artifacts  will  be  deleted  and  you  only  see their hashes (if
              reproducible) or the diff output (if not). See also --no-clean-on-error.

       --variations VARIATIONS
              Build variations to test as a comma-separated list of variation names.  Default  is
              "+all", equivalent to "+environment, +build_path, +kernel, +aslr, +num_cpus, +time,
              +user_group, +fileordering, +domain_host, +home, +locales,  +exec_path,  +timezone,
              +umask",  testing  all available variations. See the man page section on VARIATIONS
              for more advanced syntax options, including tweaking how certain variations work.

       --vary VARIATIONS
              Like --variations, but appends to previous --vary  values  instead  of  overwriting
              them.  The  last  value  set  for  --variations is treated implicitly as the zeroth
              --vary value.

       --extra-build VARIATIONS
              Perform another build with the given VARIATIONS (which may be empty) to be  applied
              on  top of what was given for --variations and --vary. Each occurrence of this flag
              specifies another build, so e.g. given twice this will  make  reprotest  perform  4
              builds in total.

       --auto-build
              Automatically  perform  builds  to try to determine which specific variations cause
              unreproducibility,  potentially  up  to  and  including  the  ones   specified   by
              --variations and --vary. Conflicts with --extra-build.

       --env-build
              Automatically  perform  builds  to  try  to  determine  which  specific environment
              variables cause unreproducibility, based on a hard-coded whitelist  and  blacklist.
              You  probably  want  to set --vary=-all as well when setting this flag; see the man
              page for details. Conflicts with --extra-build and --autobuild.

       --min-cpus NUM
              Minimum CPUs to use when fixing num_cpus. Default: 1.

   diff options:
       --diffoscope-arg ARG
              Give   extra   arguments    to    diffoscope    when    running    it.     Default:
              ['--exclude-directory-metadata=yes'].   Arguments  are  {}-formatted  with: {0} the
              name of each experiment run, and {1} the path of the experiment output.

       --diffoscope PATH
              Path to diffoscope(1). Default: diffoscope

       --no-diffoscope
              Don't run diffoscope; instead run diff(1). Useful if  you  don't  want  to  install
              diffoscope  and/or  just  want  a  quick  answer  on  whether  the reproduction was
              successful or not, without spending time to compute all the detailed differences.

   advanced options:
       --testbed-pre COMMANDS
              Shell commands to run before starting the test bed, in the context of  the  current
              system  environment.  This  may  be  used to e.g. compute information needed by the
              build, where the computation needs packages you don't want installed in the testbed
              itself.

       --testbed-init COMMANDS
              Shell  commands  to  run after starting the test bed, before running anything else.
              Used to e.g. install disorderfs in a chroot.

       --testbed-build-pre COMMANDS
              Shell commands to run before each build, even before applying variations  for  that
              build. Used to e.g.  install build-dependencies.

       --auto-preset-expr PYTHON_EXPRESSION
              This  may  be used to transform the presets returned by the auto-detection feature.
              The value should be a python expression that transforms the _ variable, which is of
              type  reprotest.presets.ReprotestPreset.   See  that class's documentation for ways
              you can write this expression. Default: _

       --no-clean-on-error
              Don't clean the virtual_server if there was an error.   Useful  for  debugging  but
              will leave cruft on your system depending on the virtual_server used; we hint about
              some but there may be others.

       --dry-run
              Don't run the builds, just print what would happen.

       --print-sudoers
              Print a sudoers file for  passwordless  operation  using  the  given  --variations,
              useful for user_group.available, domain_host.use_sudo.

       --control-build CONTROL_BUILD
              Override control build with artifacts located on this path. Allows reusing previous
              build as baseline.

       --append-build-command COMMANDS
              Append arguments to the build command

COMMAND-LINE EXAMPLES

       The easiest way to run reprotest is via our presets:

          # Build the current directory in a null server (/tmp)
          $ reprotest .
          $ reprotest . -vv -- null -d # for very verbose output

          # Build a make-based program
          $ reprotest "make clean && make" mybinary

          # Build a Debian package
          $ apt-get source hello && cd hello-2.10
          $ reprotest  auto -- null
          # Build a Debian package and disable some variations
          $ reprotest --vary=-user_group,-domain_host,-fileordering auto -- null

          # Build the given Debian source package in an schroot
          # See https://wiki.debian.org/sbuild for instructions on setting that up.
          $ reprotest reprotest_0.3.3.dsc -- schroot unstable-amd64-sbuild

          # Build the given RPM source package
          # Only null server (/tmp) is currently supported.
          $ reprotest reprotest-0.7.16.src.rpm

          # Build the given RPM source package and automatically install dependencies
          $ reprotest --testbed-build-pre 'dnf -y builddep ./*.src.rpm' reprotest-0.7.16.src.rpm

       Currently, we only support this for Debian and RPM based packages, but are keen on  adding
       more.  If  we  don't have knowledge on how to build your file or directory, you can send a
       patch to us on adding this intelligence - see the  reprotest.presets  python  module,  and
       adapt the existing logic.

       In  the  meantime,  you  can use other parts of the CLI to build arbitrary things.  You'll
       need to give two mandatory arguments, the build command to  run  and  the  build  artifact
       file/pattern to test after running the build. For example:

          $ reprotest 'python3 setup.py bdist' 'dist/*.tar.gz'

       This  runs the command on ., the current working directory. To run it on a project located
       elsewhere:

          $ reprotest -s ../path/to/other/project 'python3 setup.py bdist' 'dist/*.tar.gz'
          $ reprotest -c 'python3 setup.py bdist' ../path/to/other/project 'dist/*.tar.gz'

       These two invocations are equivalent; you can  pick  the  most  convenient  one  for  your
       use-case. When using these from a shell:

          • If  the  build  command  has  spaces,  you  will  need to quote them, e.g.  reprotest
            "dpkg-buildpackage -b --no-sign" [..].

          • If you want to use several build artifact patterns, or  if  you  want  to  use  shell
            wildcards  as  a  pattern,  you  will  also  need to quote them, e.g.  reprotest [..]
            "*.tar.gz *.tar.xz".

          • If your build artifacts have spaces in their names, you  will  need  to  quote  these
            twice,  e.g. '"a file with spaces.gz"' for a single artifact or '"dir 1"/* "dir 2"/*'
            for multiple patterns.

       To get more help for the CLI, including documentation on optional arguments and what  they
       do, run:

          $ reprotest --help

RUNNING IN A VIRTUAL SERVER

       You  can  also  run  the  build inside what is called a "virtual server".  This could be a
       container, a chroot, etc. You run them like this:

          $ reprotest 'python3 setup.py bdist_wheel'   'dist/*.whl' -- qemu    /path/to/qemu.img
          $ reprotest 'dpkg-buildpackage -b --no-sign' '../*.deb'   -- schroot unstable-amd64

       There are different server types available. See --help for a list of them,  which  appears
       near the top, in the "virtual_server_args" part of the "positional arguments" section.

       For each virtual server (e.g. "schroot"), you see which extra arguments it supports:

          $ reprotest --help schroot

       When  running  builds  inside  a  virtual  server,  you  will  probably have to give extra
       commands, in order to set up your  build  dependencies  inside  the  virtual  server.  For
       example, to take you through what the "Debian directory" preset would look like, if we ran
       it using the full CLI:

          # "Debian directory" preset
          $ reprotest . -- schroot unstable-amd64-sbuild
          # This is exactly equivalent to this:
          $ reprotest -c auto . -- schroot unstable-amd64-sbuild
          # In the non-preset full CLI, this is roughly similar to:
          $ reprotest \
              --testbed-init 'apt-get -y --no-install-recommends install \
                              disorderfs faketime locales-all sudo util-linux; \
                              test -c /dev/fuse || mknod -m 666 /dev/fuse c 10 229; \
                              test -f /etc/mtab || ln -s ../proc/self/mounts /etc/mtab' \
              --testbed-build-pre 'apt-get -y --no-install-recommends build-dep ./' \
              --build-command 'dpkg-buildpackage --no-sign -b' \
              . \
              '../*.deb' \
              -- \
              schroot unstable-amd64-sbuild

       The --testbed-init argument is needed to set up basic  tools,  which  reprotest  needs  in
       order  to  make  the  variations in the first place. This should be the same regardless of
       what package is being built, but might differ depending on what  virtual_server  is  being
       used.

       Next,  we  have  --testbed-build-pre,  then  --build-command  (or  -c).   For  our  Debian
       directory, we install build-dependencies using apt-get,  then  we  run  the  actual  build
       command itself using dpkg-buildpackage(1).

       Then,  we  have  the source_root and the artifact_pattern. For reproducibility, we're only
       interested in the binary packages.

       Finally, we specify that this is to  take  place  in  the  "schroot"  virtual_server  with
       arguments "unstable-amd64-sbuild".

       Of course, all of this is a burden to remember, if you must run the same thing many times.
       So that is why adding new presets for new package types would be good.

       Here is a more complex example. It tells  reprotest  to  store  the  build  products  into
       ./artifacts  to analyse later; and also tweaks the "Debian dsc" preset so that it uses our
       experimental toolchain:

          $ reprotest --store-dir=artifacts \
              --auto-preset-expr '_.prepend.testbed_init("apt-get install -y wget; \
                  echo deb http://reproducible.alioth.debian.org/debian/ ./ >> /etc/apt/sources.list; \
                  wget -q -O- https://reproducible.alioth.debian.org/reproducible.asc | apt-key add -; \
                  apt-get update; apt-get upgrade -y; ")' \
              ./bash_4.4-4.0~reproducible1.dsc \
              -- \
              schroot unstable-amd64-sbuild

       Alternatively, you can clone your unstable-amd64-sbuild chroot, add our repo to the cloned
       chroot, then use this chroot in place of "unstable-amd64-sbuild".  That would allow you to
       omit the long --auto-preset-expr flag above.

CONFIG FILE

       You can also give options to reprotest via a config file. This is  a  time-saving  measure
       similar  to  auto  presets;  the difference is that these are more suited for local builds
       that are suited to your personal purposes. (You may use both presets and config  files  in
       the same build.)

       The config file takes exactly the same options as the command-line interface, but with the
       additional restriction that the section name must match  the  ones  given  in  the  --help
       output.  Whitespace  is  allowed  if  and  only  if  the  same  command-line option allows
       whitespace. Finally, it is not possible to give positional arguments via this mechanism.

       Reprotest by default does not load any config file. You can tell it to load one  with  the
       --config-file  or  -f  command line options. If you give it a directory such as ., it will
       load .reprotestrc within that directory.

       A sample config file is below:

          [basics]
          verbosity = 1
          variations =
            environment
            build_path
            user_group.available+=builduser:builduser
            fileordering
            home
            kernel
            locales
            exec_path
            time
            timezone
            umask
          store_dir =
            /home/foo/build/reprotest-artifacts

          [diff]
          diffoscope_arg =
            --debug

ANALYSING DIFF OUTPUT

       Normally when diffoscope compares directories, it also compares the metadata of  files  in
       those directories - file permissions, owners, and so on.

       However  depending  on  the circumstance, this filesystem-level metadata may or may not be
       intended to be distributed to other systems. For example: (1) for  most  distros'  package
       builders,  we  don't  care  about  the metadata of the output package files; only the file
       contents will be distributed to other  systems.  On  the  other  hand,  (2)  when  running
       something  like  make install, we do care about the metadata, because this is what will be
       recreated on another system.

       In developing reprotest, our experience has been that case (1) is more common  and  so  we
       pass  --exclude-directory-metadata=yes  by default to diffoscope. If you find that you are
       using      reprotest      for      case      (2)      then      you      should       pass
       --diffoscope-args=--exclude-directory-metadata=no  to reprotest, to tell diffoscope to not
       ignore the metadata since it will be distributed and  should  therefore  be  reproducible.
       Otherwise, you may get a false-positive result.

VARIATIONS

       The  --vary  and  --variations  flags in their simple forms, are a comma-separated list of
       variation names that indicate which variations to apply. The full list of names  is  given
       in the --help text for --variations.

       In full detail, the flags are a comma-separated list of actions, as follows:

       +$variation (or $variation with no explicit operator)
       -$variation
         Enable or disable a variation

       @$variation
         Enable a variation, resetting its parameters (see below) to default values.

       $variation.$param=$value
       $variation.$param+=$value
       $variation.$param-=$value
         Set/add/remove $value as/to/from the current value of the $param parameter
         of the $variation.

       $variation.$param++
       $variation.$param--
         Increment/decrement the value of the $param parameter of the $variation.

       Most  variations  do  not  have  parameters,  and  for them only the + and - operators are
       relevant. The variations that accept parameters are:

       domain_host.use_sudo
              An integer, whether to use sudo(1) together with unshare(1) to  change  the  system
              hostname  and  domainname.  0 means don't use sudo; any non-zero value means to use
              sudo. Default is 0, however this is not recommended and make may your  build  fail,
              see "Varying the domain and host names" for details.

       environment.variables
              A  semicolon-separated ordered set, specifying environment variables that reprotest
              should  try  to  vary.  Default   is   "REPROTEST_CAPTURE_ENVIRONMENT".    Supports
              regex-based syntax e.g.

              • PID=\d{1,6}

              • HOME=(/\w{3,12}){1,4}

              • (GO|PYTHON|)PATH=(/\w{3,12}){1,4}(:(/\w{3,12}){1,4}){0,4}

              Special cases:

              • $VARNAME= (empty RHS) to tell reprotest to delete the variable

              • $VARNAME=.{0} to tell reprotest to actually set an empty value

              • \x2c and \x3b to match or generate , and ; respectively.

       user_group.available
              A semicolon-separated ordered set, specifying the available user+group combinations
              that reprotest can sudo(1) to. Default is empty, in which case the variation  is  a
              no-op,  and you'll see a warning about this. Each user+group should be given in the
              form $user:$group where either component can be omitted, or else  if  there  is  no
              colon then it is interpreted as only a $user, with no $group variation.

       time.faketimes
              A semicolon-separated ordered set, specifying possible faketime(1) time descriptors
              to use. Default is empty, in which case we randomly choose a time: either  now  (if
              the  latest  file-modtime  in  source_root is older than about half-a-year) or more
              than half-a-year in the future.

              Note that the clock  continues  to  run  during  the  build.  It  is  possible  for
              faketime(1) to freeze it, but we don't yet support that yet; it has a higher chance
              of causing your build to fail or misbehave.

       locales.locale
              A semicolon-separated list one or more locales  to  test  when  performing  locales
              variations.   If  multiple  locales  are  specified,  one will be chosen at random.
              Locales with different properties than en_US.UTF-8 are  fr_CH.UTF-8,  ru_RU.CP1251,
              kk_KZ.RK1048 or zh_CN.  Default is et_EE.UTF-8 if unspecified.

       The  difference  between  --vary and --variations is that the former appends onto previous
       values but the latter resets them. Furthermore, the last value  set  for  --variations  is
       treated as the zeroth --vary argument. For example:

          reprotest --vary=-user_group

       means  to  vary +all (the default value for --variations) and -user_group (the given value
       for --vary), whereas:

          reprotest --variations=-all,locales --variations=home,time --vary=timezone --vary=-time

       means to vary home, time (the last given value for --variations), timezone, and -time (the
       given multiple values for --vary), i.e. home and timezone.

NOTES ON VARIATIONS

       reprotest tries hard to perform variations without assuming it has full root access to the
       system. It also assumes other software may be running on the same system, so it  does  not
       perform  system-level  modifications  that  would  affect  other  processes.  Due to these
       assumptions, some variations are implemented using hacks at various levels  of  dirtiness,
       which are documented below.

       We  will  hopefully lift these assumptions for certain virtual_server contexts, in future.
       That would likely allow for smoother operation in those contexts.   The  assumptions  will
       remain for the "null" (default) virtual_server however.

   Number of CPUs
       The  control  build  uses  only  1 CPU in order to try to reduce nondeterminism that might
       exist due to multithreading or multiprocessing. If you are sure your build is not affected
       by  this  (and  good  builds  ought  not  to be), you can give --min-cpus=99999 to use all
       available cores for both builds.

   Domain or host
       Doing this without sudo may result in your build failing.

       Failure is likely if your build must  do  system-related  things  -  as  opposed  to  only
       processing  bits  and  bytes.  This  is because it runs in a separate namespace where your
       non-privileged user looks like it  is  "root",  but  this  prevents  the  filesystem  from
       recognising  files  owned  by  the  real  "root"  user,  amongst  other  things. This is a
       limitation of unshare(1) and it is not possible work  around  this  in  reprotest  without
       heavy effort.

       Therefore,  it  is  recommended  to  run this variation with use_sudo=1. To avoid password
       prompts, see the section "Avoid sudo(1) password prompts" below.

       When running inside a virtual-server:

       The  non-sudo  method  fails  with  "Operation  not  permitted",  even   if   you   edited
       /proc/sys/kernel/unprivileged_userns_clone. The cause is currently unknown.

       The  sudo  method  works  only  if you take measures to avoid sudo password prompts, since
       containers don't have a method to input this.

   User or group
       If you also vary fileordering at the same time (this is the case by  default),  each  user
       you use needs to be in the "fuse" group. Do that by running usermod -aG fuse $OTHERUSER as
       root.

       To avoid sudo(1) password prompts, see the section "Avoid sudo(1) password prompts" below.

   Time
       The "time" variation uses faketime(1) which sometimes causes  weird  and  hard-to-diagnose
       problems. In the past, this has included:

       • builds taking an infinite amount of time; though this should be fixed in recent versions
         of reprotest.

       • builds with  implausibly  huge  differences  caused  by  ./configure  scripts  producing
         different  results  with  and  without  faketime.  This  still affects bash and probably
         certain other packages using autotools.

       • builds accessing the network failing due to certificate expiration errors  and/or  other
         time-related  security errors. (Transparent builds of FOSS should not access the network
         in the first place, but it's outside of reprotest's scope to audit or prevent this.)

       If you see  a  difference  that  you  really  think  should  not  be  there,  try  passing
       --variations=-time      to      reprotest,     and/or     check     our     results     on
       https://tests.reproducible-builds.org/ which use a different (more reliable) mechanism  to
       vary the system time.

   Kernel
       The  "kernel"  variation  is  currently not working for RPM based packages and other build
       process  requiring  ldconfig.   While  building  with  this  variation  enabled,  ldconfig
       complains about FATAL: kernel too old and aborts the build.

AVOID SUDO(1) PASSWORD PROMPTS

       There  is  currently no good way to do this. The following is an EXPERIMENTAL solution and
       is brittle and unclean. You will have to decide for yourself if it's  worth  it  for  your
       use-case:

          $ reprotest --print-sudoers \
              --variations=user_group.available+=guest-builder,domain_host.use_sudo=1 \
              | sudo EDITOR=tee visudo -f /etc/sudoers.d/local-reprotest

       Make  sure  you  set  the  variations  you  actually  want  to  use. Obviously, don't pick
       privileged users for this purpose, such as root.

       (Simplifying the output using wildcards,  would  open  up  passwordless  access  to  chown
       anything  on  your  system, because wildcards here match whitespace. I don't know what the
       sudo authors were thinking.)

       No, this is not nice at all - suggestions and patches welcome.

       If you want to use this in a virtual server such as a chroot,  you'll  need  to  copy  (or
       mount or otherwise map) the resulting sudoers file into your chroot.

       For  example,  for  an  schroot,  you should (1) login to the source schroot and create an
       empty file /etc/sudoers.d/local-reprotest (this is important) and then (2) add the line:
          /etc/sudoers.d/local-reprotest  /etc/sudoers.d/local-reprotest  none bind 0 0

       to your schroot's fstab.

                                                                                               ()