Provided by: reprotest_0.7.14_all 
NAME
reprotest - Build packages and check them for reproducibility.
SYNOPSIS
reprotest --help [<virtual_server_name>]
reprotest [options] [-c <build_command>] <source_root> [<artifact_pattern>] [-- <virtual_server_args> [<virtual_server_args> ...]]
reprotest [options] [-s <source_root>] <build_command> [<artifact_pattern>] [-- <virtual_server_args> [<virtual_server_args> ...]]
DESCRIPTION
reprotest builds the same source code twice in different environments, and then checks the binaries
produced by each build for differences. If any are found, then diffoscope(1) (or if unavailable then
diff(1)) is used to display them in detail for later analysis.
See the COMMAND-LINE EXAMPLES section further below to get you started, as well as more detailed
explanations of all the command-line options. The same information is also available in
/usr/share/doc/reprotest/README.rst or similar.
source_root|build_command
The first argument is treated either as a source_root (see the -s option) or as a build-command
(see the -c option) depending on what it looks like. Most of the time, this should "just work";
but specifically: if neither -c nor -s are given, then: if this exists as a file or directory and
is not "auto", then this is treated as a source_root, else as a build_command. Otherwise, if one
of -c or -s is given, then this is treated as the other one. If both are given, then this is a
syntax error and we exit code 2.
artifact_pattern
Build artifact to test for reproducibility. May be a shell pattern such as "*.deb *.changes".
virtual_server_args
Arguments to pass to the virtual_server, the first argument being the name of the server. If this
itself contains options (of the form -xxx or --xxx), or if any of the previous arguments are
omitted, you should put a "--" between these arguments and reprotest's own options. Default:
"null", to run directly in /tmp. Choices: chroot, lxc, lxd, null, qemu, schroot, ssh
optional arguments:
--help [VIRTUAL_SERVER_NAME]
Show this help message and exit. When given an argument, show instead the help message for that
virtual server and exit.
-f CONFIG_FILE, --config-file CONFIG_FILE
File to load configuration from. (Default: None)
basic options:
--verbosity VERBOSITY
An integer. Control which messages are displayed.
-v, --verbose
Like --verbosity, but given multiple times without arguments.
--host-distro HOST_DISTRO
The distribution that will run the tests (Default: None)
-s PATH, --source-root PATH
Root of the source tree, that is copied to the virtual server and made available during the build.
If a file is given here, then its parent directory is used instead. Default: "." (current working
directory).
--source-pattern PATTERNS
Shell glob pattern to restrict the files in <source_root> that are made available during the
build. Default: empty, i.e. copy the whole <source_root> directory with no restrictions.
-c COMMANDS, --build-command COMMANDS
Build command to execute. If this is "auto" then reprotest will guess how to build the given
source_root, in which case various other options may be automatically set-if-unset. Default: auto
--store-dir DIRECTORY
Save the artifacts in this directory, which must be empty or non-existent. Otherwise, the
artifacts will be deleted and you only see their hashes (if reproducible) or the diff output (if
not). See also --no-clean-on-error.
--variations VARIATIONS
Build variations to test as a comma-separated list of variation names. Default is "+all",
equivalent to "+environment, +build_path, +kernel, +aslr, +num_cpus, +time, +user_group,
+fileordering, +domain_host, +home, +locales, +exec_path, +timezone, +umask", testing all
available variations. See the man page section on VARIATIONS for more advanced syntax options,
including tweaking how certain variations work.
--vary VARIATIONS
Like --variations, but appends to previous --vary values instead of overwriting them. The last
value set for --variations is treated implicitly as the zeroth --vary value.
--extra-build VARIATIONS
Perform another build with the given VARIATIONS (which may be empty) to be applied on top of what
was given for --variations and --vary. Each occurrence of this flag specifies another build, so
e.g. given twice this will make reprotest perform 4 builds in total.
--auto-build
Automatically perform builds to try to determine which specific variations cause
unreproducibility, potentially up to and including the ones specified by --variations and --vary.
Conflicts with --extra-build.
--env-build
Automatically perform builds to try to determine which specific environment variables cause
unreproducibility, based on a hard-coded whitelist and blacklist. You probably want to set
--vary=-all as well when setting this flag; see the man page for details. Conflicts with
--extra-build and --autobuild.
--min-cpus NUM
Minimum CPUs to use when fixing num_cpus. Default: 1.
diff options:
--diffoscope-arg ARG
Give extra arguments to diffoscope when running it. Default: ['--exclude-directory-metadata'].
Arguments are {}-formatted with: {0} the name of each experiment run, and {1} the path of the
experiment output.
--diffoscope PATH
Path to diffoscope(1). Default: diffoscope
--no-diffoscope
Don't run diffoscope; instead run diff(1). Useful if you don't want to install diffoscope and/or
just want a quick answer on whether the reproduction was successful or not, without spending time
to compute all the detailed differences.
advanced options:
--testbed-pre COMMANDS
Shell commands to run before starting the test bed, in the context of the current system
environment. This may be used to e.g. compute information needed by the build, where the
computation needs packages you don't want installed in the testbed itself.
--testbed-init COMMANDS
Shell commands to run after starting the test bed, before running anything else. Used to e.g.
install disorderfs in a chroot.
--testbed-build-pre COMMANDS
Shell commands to run before each build, even before applying variations for that build. Used to
e.g. install build-dependencies.
--auto-preset-expr PYTHON_EXPRESSION
This may be used to transform the presets returned by the auto-detection feature. The value should
be a python expression that transforms the _ variable, which is of type
reprotest.presets.ReprotestPreset. See that class's documentation for ways you can write this
expression. Default: _
--no-clean-on-error
Don't clean the virtual_server if there was an error. Useful for debugging but will leave cruft
on your system depending on the virtual_server used; we hint about some but there may be others.
--dry-run
Don't run the builds, just print what would happen.
--print-sudoers
Print a sudoers file for passwordless operation using the given --variations, useful for
user_group.available, domain_host.use_sudo.
--control-build CONTROL_BUILD
Override control build with artifacts located on this path. Allows reusing previous build as
baseline.
COMMAND-LINE EXAMPLES
The easiest way to run reprotest is via our presets:
# Build the current directory in a null server (/tmp)
$ reprotest .
$ reprotest . -vv -- null -d # for very verbose output
# Build a make-based program
$ reprotest "make clean && make" mybinary
# Build the given Debian source package in an schroot
# See https://wiki.debian.org/sbuild for instructions on setting that up.
$ reprotest reprotest_0.3.3.dsc -- schroot unstable-amd64-sbuild
Currently, we only support this for Debian packages, but are keen on adding more. If we don't have
knowledge on how to build your file or directory, you can send a patch to us on adding this intelligence
- see the reprotest.presets python module, and adapt the existing logic.
In the meantime, you can use other parts of the CLI to build arbitrary things. You'll need to give two
mandatory arguments, the build command to run and the build artifact file/pattern to test after running
the build. For example:
$ reprotest 'python3 setup.py bdist' 'dist/*.tar.gz'
This runs the command on ., the current working directory. To run it on a project located elsewhere:
$ reprotest -s ../path/to/other/project 'python3 setup.py bdist' 'dist/*.tar.gz'
$ reprotest -c 'python3 setup.py bdist' ../path/to/other/project 'dist/*.tar.gz'
These two invocations are equivalent; you can pick the most convenient one for your use-case. When using
these from a shell:
• If the build command has spaces, you will need to quote them, e.g. reprotest "dpkg-buildpackage -b
--no-sign" [..].
• If you want to use several build artifact patterns, or if you want to use shell wildcards as a
pattern, you will also need to quote them, e.g. reprotest [..] "*.tar.gz *.tar.xz".
• If your build artifacts have spaces in their names, you will need to quote these twice, e.g. '"a
file with spaces.gz"' for a single artifact or '"dir 1"/* "dir 2"/*' for multiple patterns.
To get more help for the CLI, including documentation on optional arguments and what they do, run:
$ reprotest --help
RUNNING IN A VIRTUAL SERVER
You can also run the build inside what is called a "virtual server". This could be a container, a
chroot, etc. You run them like this:
$ reprotest 'python3 setup.py bdist_wheel' 'dist/*.whl' -- qemu /path/to/qemu.img
$ reprotest 'dpkg-buildpackage -b --no-sign' '../*.deb' -- schroot unstable-amd64
There are different server types available. See --help for a list of them, which appears near the top, in
the "virtual_server_args" part of the "positional arguments" section.
For each virtual server (e.g. "schroot"), you see which extra arguments it supports:
$ reprotest --help schroot
When running builds inside a virtual server, you will probably have to give extra commands, in order to
set up your build dependencies inside the virtual server. For example, to take you through what the
"Debian directory" preset would look like, if we ran it using the full CLI:
# "Debian directory" preset
$ reprotest . -- schroot unstable-amd64-sbuild
# This is exactly equivalent to this:
$ reprotest -c auto . -- schroot unstable-amd64-sbuild
# In the non-preset full CLI, this is roughly similar to:
$ reprotest \
--testbed-init 'apt-get -y --no-install-recommends install \
disorderfs faketime locales-all sudo util-linux; \
test -c /dev/fuse || mknod -m 666 /dev/fuse c 10 229; \
test -f /etc/mtab || ln -s ../proc/self/mounts /etc/mtab' \
--testbed-build-pre 'apt-get -y --no-install-recommends build-dep ./' \
--build-command 'dpkg-buildpackage --no-sign -b' \
. \
'../*.deb' \
-- \
schroot unstable-amd64-sbuild
The --testbed-init argument is needed to set up basic tools, which reprotest needs in order to make the
variations in the first place. This should be the same regardless of what package is being built, but
might differ depending on what virtual_server is being used.
Next, we have --testbed-build-pre, then --build-command (or -c). For our Debian directory, we install
build-dependencies using apt-get, then we run the actual build command itself using dpkg-buildpackage(1).
Then, we have the source_root and the artifact_pattern. For reproducibility, we're only interested in the
binary packages.
Finally, we specify that this is to take place in the "schroot" virtual_server with arguments
"unstable-amd64-sbuild".
Of course, all of this is a burden to remember, if you must run the same thing many times. So that is why
adding new presets for new package types would be good.
Here is a more complex example. It tells reprotest to store the build products into ./artifacts to
analyse later; and also tweaks the "Debian dsc" preset so that it uses our experimental toolchain:
$ reprotest --store-dir=artifacts \
--auto-preset-expr '_.prepend.testbed_init("apt-get install -y wget; \
echo deb http://reproducible.alioth.debian.org/debian/ ./ >> /etc/apt/sources.list; \
wget -q -O- https://reproducible.alioth.debian.org/reproducible.asc | apt-key add -; \
apt-get update; apt-get upgrade -y; ")' \
./bash_4.4-4.0~reproducible1.dsc \
-- \
schroot unstable-amd64-sbuild
Alternatively, you can clone your unstable-amd64-sbuild chroot, add our repo to the cloned chroot, then
use this chroot in place of "unstable-amd64-sbuild". That would allow you to omit the long
--auto-preset-expr flag above.
CONFIG FILE
You can also give options to reprotest via a config file. This is a time-saving measure similar to auto
presets; the difference is that these are more suited for local builds that are suited to your personal
purposes. (You may use both presets and config files in the same build.)
The config file takes exactly the same options as the command-line interface, but with the additional
restriction that the section name must match the ones given in the --help output. Whitespace is allowed
if and only if the same command-line option allows whitespace. Finally, it is not possible to give
positional arguments via this mechanism.
Reprotest by default does not load any config file. You can tell it to load one with the --config-file or
-f command line options. If you give it a directory such as ., it will load .reprotestrc within that
directory.
A sample config file is below:
[basics]
verbosity = 1
variations =
environment
build_path
user_group.available+=builduser:builduser
fileordering
home
kernel
locales
exec_path
time
timezone
umask
store_dir =
/home/foo/build/reprotest-artifacts
[diff]
diffoscope_arg =
--debug
ANALYSING DIFF OUTPUT
Normally when diffoscope compares directories, it also compares the metadata of files in those
directories - file permissions, owners, and so on.
However depending on the circumstance, this filesystem-level metadata may or may not be intended to be
distributed to other systems. For example: (1) for most distros' package builders, we don't care about
the metadata of the output package files; only the file contents will be distributed to other systems. On
the other hand, (2) when running something like make install, we do care about the metadata, because this
is what will be recreated on another system.
In developing reprotest, our experience has been that case (1) is more common and so we pass
--exclude-directory-metadata by default to diffoscope. If you find that you are using reprotest for case
(2) then you should pass --diffoscope-args=--no-exclude-directory-metadata to reprotest, to tell
diffoscope to not ignore the metadata since it will be distributed and should therefore be reproducible.
Otherwise, you may get a false-positive result.
VARIATIONS
The --vary and --variations flags in their simple forms, are a comma-separated list of variation names
that indicate which variations to apply. The full list of names is given in the --help text for
--variations.
In full detail, the flags are a comma-separated list of actions, as follows:
+$variation (or $variation with no explicit operator)
-$variation
Enable or disable a variation
@$variation
Enable a variation, resetting its parameters (see below) to default values.
$variation.$param=$value
$variation.$param+=$value
$variation.$param-=$value
Set/add/remove $value as/to/from the current value of the $param parameter
of the $variation.
$variation.$param++
$variation.$param--
Increment/decrement the value of the $param parameter of the $variation.
Most variations do not have parameters, and for them only the + and - operators are relevant. The
variations that accept parameters are:
domain_host.use_sudo
An integer, whether to use sudo(1) together with unshare(1) to change the system hostname and
domainname. 0 means don't use sudo; any non-zero value means to use sudo. Default is 0, however
this is not recommended and make may your build fail, see "Varying the domain and host names" for
details.
environment.variables
A semicolon-separated ordered set, specifying environment variables that reprotest should try to
vary. Default is "REPROTEST_CAPTURE_ENVIRONMENT". Supports regex-based syntax e.g.
• PID=\d{1,6}
• HOME=(/\w{3,12}){1,4}
• (GO|PYTHON|)PATH=(/\w{3,12}){1,4}(:(/\w{3,12}){1,4}){0,4}
Special cases:
• $VARNAME= (empty RHS) to tell reprotest to delete the variable
• $VARNAME=.{0} to tell reprotest to actually set an empty value
• \x2c and \x3b to match or generate , and ; respectively.
user_group.available
A semicolon-separated ordered set, specifying the available user+group combinations that reprotest
can sudo(1) to. Default is empty, in which case the variation is a no-op, and you'll see a warning
about this. Each user+group should be given in the form $user:$group where either component can be
omitted, or else if there is no colon then it is interpreted as only a $user, with no $group
variation.
time.faketimes
A semicolon-separated ordered set, specifying possible faketime(1) time descriptors to use.
Default is empty, in which case we randomly choose a time: either now (if the latest file-modtime
in source_root is older than about half-a-year) or more than half-a-year in the future.
Note that the clock continues to run during the build. It is possible for faketime(1) to freeze
it, but we don't yet support that yet; it has a higher chance of causing your build to fail or
misbehave.
The difference between --vary and --variations is that the former appends onto previous values but the
latter resets them. Furthermore, the last value set for --variations is treated as the zeroth --vary
argument. For example:
reprotest --vary=-user_group
means to vary +all (the default value for --variations) and -user_group (the given value for --vary),
whereas:
reprotest --variations=-all,locales --variations=home,time --vary=timezone --vary=-time
means to vary home, time (the last given value for --variations), timezone, and -time (the given multiple
values for --vary), i.e. home and timezone.
NOTES ON VARIATIONS
reprotest tries hard to perform variations without assuming it has full root access to the system. It
also assumes other software may be running on the same system, so it does not perform system-level
modifications that would affect other processes. Due to these assumptions, some variations are
implemented using hacks at various levels of dirtiness, which are documented below.
We will hopefully lift these assumptions for certain virtual_server contexts, in future. That would
likely allow for smoother operation in those contexts. The assumptions will remain for the "null"
(default) virtual_server however.
Number of CPUs
The control build uses only 1 CPU in order to try to reduce nondeterminism that might exist due to
multithreading or multiprocessing. If you are sure your build is not affected by this (and good builds
ought not to be), you can give --min-cpus=99999 to use all available cores for both builds.
Domain or host
Doing this without sudo may result in your build failing.
Failure is likely if your build must do system-related things - as opposed to only processing bits and
bytes. This is because it runs in a separate namespace where your non-privileged user looks like it is
"root", but this prevents the filesystem from recognising files owned by the real "root" user, amongst
other things. This is a limitation of unshare(1) and it is not possible work around this in reprotest
without heavy effort.
Therefore, it is recommended to run this variation with use_sudo=1. To avoid password prompts, see the
section "Avoid sudo(1) password prompts" below.
When running inside a virtual-server:
The non-sudo method fails with "Operation not permitted", even if you edited
/proc/sys/kernel/unprivileged_userns_clone. The cause is currently unknown.
The sudo method works only if you take measures to avoid sudo password prompts, since containers don't
have a method to input this.
User or group
If you also vary fileordering at the same time (this is the case by default), each user you use needs to
be in the "fuse" group. Do that by running usermod -aG fuse $OTHERUSER as root.
To avoid sudo(1) password prompts, see the section "Avoid sudo(1) password prompts" below.
Time
The "time" variation uses faketime(1) which sometimes causes weird and hard-to-diagnose problems. In the
past, this has included:
• builds taking an infinite amount of time; though this should be fixed in recent versions of reprotest.
• builds with implausibly huge differences caused by ./configure scripts producing different results with
and without faketime. This still affects bash and probably certain other packages using autotools.
• builds accessing the network failing due to certificate expiration errors and/or other time-related
security errors. (Transparent builds of FOSS should not access the network in the first place, but it's
outside of reprotest's scope to audit or prevent this.)
If you see a difference that you really think should not be there, try passing --variations=-time to
reprotest, and/or check our results on https://tests.reproducible-builds.org/ which use a different (more
reliable) mechanism to vary the system time.
AVOID SUDO(1) PASSWORD PROMPTS
There is currently no good way to do this. The following is an EXPERIMENTAL solution and is brittle and
unclean. You will have to decide for yourself if it's worth it for your use-case:
$ reprotest --print-sudoers \
--variations=user_group.available+=guest-builder,domain_host.use_sudo=1 \
| sudo EDITOR=tee visudo -f /etc/sudoers.d/local-reprotest
Make sure you set the variations you actually want to use. Obviously, don't pick privileged users for
this purpose, such as root.
(Simplifying the output using wildcards, would open up passwordless access to chown anything on your
system, because wildcards here match whitespace. I don't know what the sudo authors were thinking.)
No, this is not nice at all - suggestions and patches welcome.
If you want to use this in a virtual server such as a chroot, you'll need to copy (or mount or otherwise
map) the resulting sudoers file into your chroot.
For example, for an schroot, you should (1) login to the source schroot and create an empty file
/etc/sudoers.d/local-reprotest (this is important) and then (2) add the line:
/etc/sudoers.d/local-reprotest /etc/sudoers.d/local-reprotest none bind 0 0
to your schroot's fstab.
reprotest 0.7.14 February 2020 REPROTEST(1)