Ubuntu Manpage: adt-run - test an installed binary package using the package's tests

NAME

       adt-run - test an installed binary package using the package's tests

SYNOPSYS

       adt-run options...  --- virt-server [virt-server-arg...]

DESCRIPTION

       adt-run is the program for invoking the autopkgtest package testing machinery.

       autopkgtest  is  a  facility  for  testing  binary  packages, as installed on a system (such as a testbed
       system).  The tests are those supplied in the source package.

       adt-run runs each test supplied by a particular package and reports the results.  It drives the specified
       virtualisation  regime as appropriate, and parses the test description metadata, and arranges for data to
       be copied to and from the testbed as required.

       adt-run should be invoked (unless options to the contrary are supplied) in the top level directory of the
       built source tree, on the host.  The package should be installed on the testbed.

       See /usr/share/doc/autopkgtest/README.running-tests.gz for an introduction about how to use adt-run.

PROCESSING INSTRUCTIONS

       --source dsc
              Run  tests  from  source  package dsc. By default the package will also be built and the resulting
              binaries will be used to satisfy test dependencies; to disable that,  specify  the  -B/--no-built-
              binaries option before.

              The ordering is significant: each --source option should precede options whose dependencies are to
              be satisfied by the binaries it produces.

       --unbuilt-tree directory
              Specifies that tests from the unbuilt source tree directory should be run.  This is  very  similar
              to  specifying  --source  except  that  a  directory  tree (which should be pristine) is supplied,
              instead of a source package.

       --built-tree directory
              Specifies that tests from the built source tree directory  should  be  run.  Note  that  all  test
              dependencies  are  then satisfied by archive packages, unless you explicitly specify locally built
              .debs with --binary.

       --apt-source srcpkgname
              Downloads srcpkgname with apt-get source in the testbed and run its  tests.  This  is  similar  to
              specifying --source but avoids copying the source from the host to the testbed. Resulting binaries
              will not be used to satisfy dependencies, as usually in this mode you want to test binaries from a
              real archive.

       --binary deb
              Specifies  that deb should be used for tests of all following source packages.  By default it will
              be used to satisfy dependencies, both during building and testing.

              The ordering is significant, as for --source. In particular, if a subsequent source  package  will
              build a binary of the same name, that will be used from then on, and deb will be ignored.

       filename
              Bare  filename arguments are processed as if --built-tree, --source, --unbuilt-tree, --apt-source,
              or --binary was specified; the nature of the argument is guessed from the form  of  the  filename.
              In  the  case  of --built-tree, either the option must be specified, or the filename must end in a
              slash; two slashes at the end are taken to mean --unbuilt-tree.

PROCESSING OPTIONS

       These affect processing instructions.  Unless stated otherwise, they affect all subsequent options.

       --sources-tests|--sources-no-tests Specifies that the tests in  subsequent  --source  and  --unbuilt-tree
       arguments should (or should not) be run.

       --built-binaries-filter=pattern,pattern,...
              Specifies  that  only  binaries  whose package names match one of the specified patterns should be
              used; others will be ignored.  This option  applies  to  subsequent  --source  and  --unbuilt-tree
              arguments.

       -B | --no-built-binaries
              Specifies  that  all  built  binaries  from  source  packages  should  be  ignored completely, and
              dependencies are satisfied with packages from the archive. Note that packages still get built if a
              test  requires  build-needed.  Equivalent to --built-binaries-filter=_ (since no package name ever
              contains _).

       --binaries=ignore | --binaries=auto | --binaries=install
              Specifies that binary package (in subsequently specified --binary  arguments,  or  resulting  from
              subsequently  specified  --source  or  --unbuilt-tree  arguments  and  not filtered out) should be
              ignored,  used  only  to  satisfy  dependencies,  or  installed   unconditionally,   respectively.
              Equivalent to specifying both --binaries-forbuilds and --binaries-fortests.

       --binaries-forbuilds=...
              Like  --binaries= but only changes the handling during package building: packages will be ignored,
              used for dependencies, or unconditionally installed, when a source package is built.

       --binaries-fortests=...
              Like --binaries= but only changes the handling during testing: packages will be ignored, used  for
              dependencies  (including  as the package under test), or unconditionally installed, when tests are
              run (as a result of --source, --built-tree or --unbuilt-tree).

OTHER OPTIONS

-u user | --user=user
Run builds and tests as user on the testbed. This needs root on the testbed; if root on the
testbed is not available then builds and tests run as whatever user is provided.

--gain-root=gain-root
Prefixes debian/rules binary with gain-root. The default is not to use anything, except that if
--user is supplied or root on the testbed is not available the default is fakeroot.

-o dir | --output-dir=dir
Specifies that test artifacts (stderr and stdout from the tests, the log file, built binary
packages etc.) should be placed in the given directory. dir will be created if necessary, and
emptied of all of its contents before adt-run starts.

-l logfile | --log-file=logfile
Specifies that the trace log should be written to logfile instead of to output-dir.

--summary=summary
Specifies that a summary of the outcome should be written to summary. The events in the summary
are written to the log in any case.

--timeout-which=seconds
Use a different timeout for operations on or with the testbed. There are four timeouts affected
by four values of which: short: supposedly short operations like setting up the testbed's apt and
checking the state (default: 100s); install: installation of packages including dependencies
(default: 3ks); test: test runs (default: 10ks); and build: builds (default: 100ks). The value
must be specified as an integer number of seconds.

--timeout-factor=double
Multiply all of the default timeouts by the specified factor (see --timeout-which above). Only
the defaults are affected; explicit timeout settings are used exactly as specified.

--debug|-d
Include additional debugging information in the trace log. Each additional -d increases the
debugging level; the current maximum is -ddd. If you like to see what's going on, -d or -dd is
recommended.

--shell-fail|-s
Run an interactive shell in the testbed after every failed test.

--shell
Run an interactive shell in the testbed after every test.

--gnupg-home=dir
Uses dir as the GNUPGHOME for local apt archive signing. The specified directory should not
contain keyrings containing other unrelated keys, since adt-run does not specify to gpg which keys
to use. The default is $HOME/.autopkgtest.

--gnupg-home=fresh
Use a fresh temporary directory and generate fresh keys each run. This can be very slow and
depends on the availability of sufficient quantities of high-quality entropy.

-q | --quiet
Do not send a copy of adt-run's trace logstream to stderr. This option does not affect the copy
sent to logfile or output-dir. Note that without the trace logstream it can be very hard to
diagnose problems.

--set-lang=langval
When running commands on the testbed, sets the LANG environment variable to langval. The default
in adt-run is to set it to C.

--leave-lang
Suppresses the setting by adt-run of LANG on the testbed. This results in tests and builds using
the testbed's own normal LANG value setting.

--setup-commands=commands
Run commands after opening the testbed. This can be used e. g. to enable additional apt sources,
run apt-get update or similar. If commands is an existing file name, the commands are read from
that; otherwise it is a string with the actual commands that gets run as-is. This option can be
specified multiple times.

--apt-upgrade | -U
Run apt-get update and apt-get dist-upgrade -y in the testbed before running the tests.

--apt-pocket=pocket
Add apt sources for release-pocket. This finds the first deb line in /etc/apt/sources.list which
does not already specify a pocket and adds a deb and deb-src line with that pocket to
/etc/apt/sources.list.d/pocket.list. Note that this does not imply calling apt-get update.

VIRTUALIZATION SERVER

       --- virt-server virt-server-arg...
              Specifies the virtualisation regime server, as a command and  arguments  to  invoke.   virt-server
              must  be  an existing autopkgtest virtualization server such as adt-virt-schroot or adt-virt-qemu.
              You can leave out the adt-virt- prefix and just specify the last part, e. g.  schroot.

              All the remaining arguments and options after --- are passed to the virtualisation server program.
              See the manpages of the individual servers for how to use them.

OUTPUT FORMAT

       During a normal test run, one line is printed for each test.  This consists of a short string identifying
       the test, some horizontal whitespace, and either PASS or FAIL reason or SKIP reason where  the  pass/fail
       indication is separated by any reason by some horizontal whitespace.

       The  string  to  identify  the  test  consists  of  a  short  alphanumeric  string invented by adt-run to
       distinguish different command-line arguments, the argid, followed by a hyphen and the test name.

       Sometimes a SKIP will be reported when the name of the test is not known or not applicable: for  example,
       when  there  are  no  tests  in  the  package,  or  a  there is a test stanza which contains features not
       understood by this version of adt-run.  In this case * will appear where the name of the test should be.

       If adt-run detects that erroneous package(s) are involved, it will print the  two  lines  blame:  blamed-
       thing...   and  badpkg:  message.   Here  each  whitespace-separated  blamed-thing is one of arg:argument
       (representing a pathname found in  a  command  line  argument),  dsc:package  (a  source  package  name),
       deb:package  (a  binary  package  name) or possibly other strings to be determined.  This indicates which
       arguments and/or packages might have contributed to the problem;  the  ones  which  were  processed  most
       recently and which are therefore most likely to be the cause of a problem are listed last.

EXIT STATUS

       0    all tests passed
       1    unexpected failure (the python interpreter invents this exit status)
       2    at least one test skipped
       4    at least one test failed
       6    at least one test failed and at least one test skipped
       8    no tests in this package
       12   erroneous package
       16   testbed failure
       20   other unexpected failures including bad usage

BUGS

       This tool still lacks some important features and is not very well-tested.

AUTHORS AND COPYRIGHT

       This manpage is part of autopkgtest, a tool for testing Debian binary packages.  autopkgtest is Copyright
       (C) 2006-2007 Canonical Ltd and others.

       See /usr/share/doc/autopkgtest/CREDITS for the list of contributors and full copying conditions.