Ubuntu Manpage: adt-run - test an installed binary package using the package's tests

NAME

       adt-run - test an installed binary package using the package's tests

SYNOPSYS

       adt-run options...  --- virt-server [virt-server-arg...]

DESCRIPTION

       adt-run is the program for invoking the autopkgtest package testing machinery.

       autopkgtest is a facility for testing binary packages, as installed on a system (such as a
       testbed system).  The tests are those supplied in the source package.

       adt-run runs each test supplied by a particular  package  and  reports  the  results.   It
       drives the specified virtualisation regime as appropriate, and parses the test description
       metadata, and arranges for data to be copied to and from the testbed as required.

       adt-run should be invoked (unless options to the contrary are supplied) in the  top  level
       directory  of  the built source tree, on the host.  The package should be installed on the
       testbed.

       See /usr/share/doc/autopkgtest/README.running-tests.gz for an introduction  about  how  to
       use adt-run.

PROCESSING INSTRUCTIONS

       --source dsc
              Run  tests  from  source package dsc. By default the package will also be built and
              the resulting binaries will be used to satisfy test dependencies; to disable  that,
              specify the -B/--no-built-binaries option before.

              The  ordering  is  significant:  each  --source option should precede options whose
              dependencies are to be satisfied by the binaries it produces.

       --unbuilt-tree directory
              Specifies that tests from the unbuilt source tree directory should be run.  This is
              very  similar  to specifying --source except that a directory tree (which should be
              pristine) is supplied, instead of a source package.

       --built-tree directory
              Specifies that tests from the built source tree directory should be run. Note  that
              all test dependencies are then satisfied by archive packages, unless you explicitly
              specify locally built .debs with --binary.

       --apt-source srcpkgname
              Downloads srcpkgname with apt-get source in the testbed and run its tests. This  is
              similar  to  specifying --source but avoids copying the source from the host to the
              testbed. Resulting binaries will not be used to satisfy dependencies, as usually in
              this mode you want to test binaries from a real archive.

       --binary deb
              Specifies  that  deb should be used for tests of all following source packages.  By
              default it will be used to satisfy dependencies, both during building and testing.

              The ordering is significant, as for --source. In particular, if a subsequent source
              package  will  build a binary of the same name, that will be used from then on, and
              deb will be ignored.

       filename
              Bare filename arguments are processed as if --built-tree, --source, --unbuilt-tree,
              --apt-source, or --binary was specified; the nature of the argument is guessed from
              the form of the filename.  In the case of --built-tree, either the option  must  be
              specified, or the filename must end in a slash; two slashes at the end are taken to
              mean --unbuilt-tree.

PROCESSING OPTIONS

       These affect processing instructions.  Unless stated otherwise, they affect all subsequent
       options.

       --sources-tests|--sources-no-tests  Specifies  that  the  tests in subsequent --source and
       --unbuilt-tree arguments should (or should not) be run.

       --built-binaries-filter=pattern,pattern,...
              Specifies that only binaries  whose  package  names  match  one  of  the  specified
              patterns should be used; others will be ignored.  This option applies to subsequent
              --source and --unbuilt-tree arguments.

       -B | --no-built-binaries
              Specifies  that  all  built  binaries  from  source  packages  should  be   ignored
              completely,  and  dependencies  are  satisfied with packages from the archive. Note
              that packages still get built  if  a  test  requires  build-needed.  Equivalent  to
              --built-binaries-filter=_ (since no package name ever contains _).

       --binaries=ignore | --binaries=auto | --binaries=install
              Specifies  that  binary  package  (in subsequently specified --binary arguments, or
              resulting from subsequently specified --source or --unbuilt-tree arguments and  not
              filtered  out)  should  be ignored, used only to satisfy dependencies, or installed
              unconditionally, respectively.  Equivalent to specifying both  --binaries-forbuilds
              and --binaries-fortests.

       --binaries-forbuilds=...
              Like  --binaries=  but  only changes the handling during package building: packages
              will be ignored, used for dependencies, or unconditionally installed, when a source
              package is built.

       --binaries-fortests=...
              Like  --binaries=  but  only  changes the handling during testing: packages will be
              ignored,  used  for  dependencies  (including  as  the  package  under  test),   or
              unconditionally  installed,  when  tests are run (as a result of --source, --built-
              tree or --unbuilt-tree).

OTHER OPTIONS

-u user | --user=user
Run builds and tests as user on the testbed. This needs root on the testbed; if
root on the testbed is not available then builds and tests run as whatever user is
provided.

--gain-root=gain-root
Prefixes debian/rules binary with gain-root. The default is not to use anything,
except that if --user is supplied or root on the testbed is not available the
default is fakeroot.

-o dir | --output-dir=dir
Specifies that test artifacts (stderr and stdout from the tests, the log file,
built binary packages etc.) should be placed in the given directory. dir will be
created if necessary, and emptied of all of its contents before adt-run starts.

-l logfile | --log-file=logfile
Specifies that the trace log should be written to logfile instead of to output-dir.

--summary=summary
Specifies that a summary of the outcome should be written to summary. The events
in the summary are written to the log in any case.

--timeout-which=seconds
Use a different timeout for operations on or with the testbed. There are four
timeouts affected by four values of which: short: supposedly short operations like
setting up the testbed's apt and checking the state (default: 100s); install:
installation of packages including dependencies (default: 3ks); test: test runs
(default: 10ks); and build: builds (default: 100ks). The value must be specified
as an integer number of seconds.

--timeout-factor=double
Multiply all of the default timeouts by the specified factor (see --timeout-which
above). Only the defaults are affected; explicit timeout settings are used exactly
as specified.

--debug|-d
Include additional debugging information in the trace log. Each additional -d
increases the debugging level; the current maximum is -ddd. If you like to see
what's going on, -d or -dd is recommended.

--shell-fail|-s
Run an interactive shell in the testbed after every failed test.

--shell
Run an interactive shell in the testbed after every test.

--gnupg-home=dir
Uses dir as the GNUPGHOME for local apt archive signing. The specified directory
should not contain keyrings containing other unrelated keys, since adt-run does not
specify to gpg which keys to use. The default is $HOME/.autopkgtest.

--gnupg-home=fresh
Use a fresh temporary directory and generate fresh keys each run. This can be very
slow and depends on the availability of sufficient quantities of high-quality
entropy.

-q | --quiet
Do not send a copy of adt-run's trace logstream to stderr. This option does not
affect the copy sent to logfile or output-dir. Note that without the trace
logstream it can be very hard to diagnose problems.

--set-lang=langval
When running commands on the testbed, sets the LANG environment variable to
langval. The default in adt-run is to set it to C.

--leave-lang
Suppresses the setting by adt-run of LANG on the testbed. This results in tests
and builds using the testbed's own normal LANG value setting.

--setup-commands=commands
Run commands after opening the testbed. This can be used e. g. to enable additional
apt sources, run apt-get update or similar. If commands is an existing file name,
the commands are read from that; otherwise it is a string with the actual commands
that gets run as-is. This option can be specified multiple times.

--apt-upgrade | -U
Run apt-get update and apt-get dist-upgrade -y in the testbed before running the
tests.

--apt-pocket=pocket
Add apt sources for release-pocket. This finds the first deb line in
/etc/apt/sources.list which does not already specify a pocket and adds a deb and
deb-src line with that pocket to /etc/apt/sources.list.d/pocket.list. Note that
this does not imply calling apt-get update.

VIRTUALIZATION SERVER

       --- virt-server virt-server-arg...
              Specifies  the  virtualisation regime server, as a command and arguments to invoke.
              virt-server must be an existing autopkgtest virtualization server such as adt-virt-
              schroot  or adt-virt-qemu.  You can leave out the adt-virt- prefix and just specify
              the last part, e. g.  schroot.

              All the remaining arguments and options after --- are passed to the  virtualisation
              server program. See the manpages of the individual servers for how to use them.

OUTPUT FORMAT

       During  a  normal  test  run, one line is printed for each test.  This consists of a short
       string identifying the test, some horizontal whitespace, and either PASS or FAIL reason or
       SKIP  reason  where the pass/fail indication is separated by any reason by some horizontal
       whitespace.

       The string to identify the test consists of a short alphanumeric string invented  by  adt-
       run  to  distinguish different command-line arguments, the argid, followed by a hyphen and
       the test name.

       Sometimes a SKIP will be reported  when  the  name  of  the  test  is  not  known  or  not
       applicable:  for  example,  when  there  are no tests in the package, or a there is a test
       stanza which contains features not understood by this version of adt-run.  In this case  *
       will appear where the name of the test should be.

       If  adt-run  detects  that  erroneous package(s) are involved, it will print the two lines
       blame: blamed-thing...  and badpkg: message.  Here each whitespace-separated  blamed-thing
       is  one  of  arg:argument  (representing  a  pathname  found  in a command line argument),
       dsc:package (a source package name), deb:package (a binary package name) or possibly other
       strings  to  be  determined.   This  indicates  which arguments and/or packages might have
       contributed to the problem; the ones which were processed  most  recently  and  which  are
       therefore most likely to be the cause of a problem are listed last.

EXIT STATUS

       0    all tests passed
       1    unexpected failure (the python interpreter invents this exit status)
       2    at least one test skipped
       4    at least one test failed
       6    at least one test failed and at least one test skipped
       8    no tests in this package
       12   erroneous package
       16   testbed failure
       20   other unexpected failures including bad usage

BUGS

       This tool still lacks some important features and is not very well-tested.

AUTHORS AND COPYRIGHT

       This  manpage  is  part  of  autopkgtest,  a  tool  for  testing  Debian  binary packages.
       autopkgtest is Copyright (C) 2006-2007 Canonical Ltd and others.

       See /usr/share/doc/autopkgtest/CREDITS for the  list  of  contributors  and  full  copying
       conditions.