Provided by: autopkgtest_5.10ubuntu2_all bug

NAME

       autopkgtest - test an installed binary package using the source package's tests

SYNOPSIS

       autopkgtest [options...]  [testbinary...]  testsrc -- virt-server [virt-server-arg...]

DESCRIPTION

       autopkgtest  runs  tests  on  binary  Debian  or  Click packages, as installed on a system
       (called "testbed").  The tests are supplied in the source package.

       autopkgtest runs each test supplied by a particular package and reports the  results.   It
       drives  the  specified  virtualisation  regime as appropriate, parses the test description
       metadata, and arranges for data to be copied to and from the testbed as required.

       See /usr/share/doc/autopkgtest/README.running-tests.rst.gz for an introduction  about  how
       to use autopkgtest.

TESTING A DEBIAN PACKAGE

       Positional  (non-option) arguments specify exactly one source package (containing the test
       code) and optionally some binary packages to test.

       testsrc can be one of:

       .dsc file
              Run tests from Debian .dsc source package. By default  the  package  will  also  be
              built  and  the  resulting  binaries  will be used to satisfy test dependencies; to
              disable that, specify the -B/--no-built-binaries option.

       source package directory
              Run tests from a Debian source tree directory. If that is an unbuilt tree, this  is
              very  similar  to specifying a .dsc. If that is a built tree, all test dependencies
              get satisfied by archive packages, unless  you  explicitly  specify  locally  built
              .debs as well.

              Attention: If you just specify a bare directory name which is a legal Debian source
              package name, it will be interpreted as the  latter  (see  below).  In  this  case,
              prefix the directory name with ./.

       current directory
              If  no source package is specified on the command line and the current directory is
              a Debian source package, this will be tested.

       source package name
              Downloads the given source package name with apt-get source in the testbed and  run
              its  tests. This is similar to specifying a .dsc but avoids copying the source from
              the host to the testbed. Possibly built binaries  (if  the  test  specifies  needs-
              build)  will  not be used to satisfy dependencies, as usually in this mode you want
              to test binaries from a real archive.

       git URL or URL#branch
              Git-clones the given URL (which must contain an unbuilt  Debian  source  tree)  and
              runs  the  tests  from  that.   If branch is given, this branch will be checked out
              instead of the default (usually "master"). This can also be a more general  refspec
              such as #refs/pull/123/head" for a GitHub pull request.

              This  is  very similar to cloning manually and specifying the checkout directory as
              test; i. e.  this is commonly used with --no-built-binaries.  The git package  will
              be installed if necessary.

       .changes file
              Run  tests from the .dsc source package in the given .changes file. If the .changes
              contains .deb packages, they will be used  for  the  test.   Acts  as  if  you  had
              specified  the  .debs  and  .dsc from the .changes file as explicit arguments. Note
              that if the .changes contains only debs, the corresponding .dsc still needs  to  be
              specified alongside, or the current directory must be the source package.

       All  other  positional  arguments must be .deb binary packages. They will be used for both
       build and test dependencies of the source package. If any binary package  is  given,  then
       --no-built-binaries is implied.

TESTING A CLICK PACKAGE

       This  needs  exactly  one  .click  package. If its manifest specifies an x-source URL, the
       corresponding source package will be downloaded and tests run from there;  then  a  source
       directory does not need to be given. Otherwise specifying a source directory as positional
       argument is mandatory.

       For testing a local  .click  package,  give  its  path  to  autopkgtest.   For  testing  a
       preinstalled  click,  use  --installed-click  com.example.myclick (this cannot be given as
       positional argument as this cannot be disambiguated from Debian source package names).

       Examples with explicit click source directory:

              autopkgtest src/myclick_0.1_all.click com.example.myclick src/myclick/ -- [...]

              autopkgtest --installed-click com.example.myclick src/myclick/ -- [...]

       Examples without click source directory; click package needs  to  have  x-source  manifest
       entry:

              autopkgtest src/myclick_0.1_all.click -- [...]

              autopkgtest --installed-click com.example.myclick -- [...]

TEST OPTIONS

       -B | --no-built-binaries
              Binaries from unbuilt source packages (see above) will not be built or ignored, and
              dependencies are satisfied with packages from the archive.  Note  that  the  source
              package still gets built if a test requires build-needed.

       --override-control=PATH
              Read  the  test  metadata  from  PATH  instead  of debian/tests/control (for Debian
              sources) or the Click manifest.

       --test-name=TEST
              Run only the given test name (from test control file).   This  replaces  --testname
              which is deprecated.

LOGGING OPTIONS

       If you don't specify any option, autopkgtest only writes its output/results to stderr.

       -o dir | --output-dir=dir
              Specifies  that  test  artifacts  (stderr  and stdout from the tests, the log file,
              built binary packages etc.) should be placed in the given directory.  dir must  not
              exist yet or be empty, otherwise autopkgtest will refuse to use it.

       -l logfile | --log-file=logfile
              Specifies that the trace log should be written to logfile instead of to output-dir.

       --summary=summary
              Specifies  that  a summary of the outcome should be written to summary.  The events
              in the summary are written to the log in any case.

       -q | --quiet
              Do not send a copy of autopkgtest's trace logstream to stderr.   This  option  does
              not  affect  the  copy  sent to logfile or output-dir.  Note that without the trace
              logstream it can be very hard to diagnose problems.

TEST BED SETUP OPTIONS

       --setup-commands=commands
              Run commands after opening the testbed. This can be used to do anything that  isn't
              supported by an existing autopkgtest command. If commands is an existing file name,
              the commands are read from that; otherwise it is a string with the actual  commands
              that  gets  run  as-is.  File  names without directory will be searched in both the
              current directory and in /usr/share/autopkgtest/setup-commands/ so you do not  need
              to give the full path for setup scripts shipped with autopkgtest.

              Normally,  if  the  setup commands fail, autopkgtest will consider this a transient
              testbed error (exit code 16). However, if the setup commands exit  with  code  100,
              autopkgtest  will  consider  this an "erroneous package" (exit code 12) instead, so
              this can be used to e. g. detect upgrade errors to a new  version.  Note  that  apt
              exits with exit code 100 in these cases.

              This option can be specified multiple times.

              If --user is given or the test bed provides a suggested-normal-user capability, the
              $AUTOPKGTEST_NORMAL_USER environment variable will be set to that user.

              If  the  setup  commands  affect  anything  in  boot  directories  (like  /boot  or
              /lib/systemd/system)  and  the  testbed  supports  rebooting,  the  testbed will be
              rebooted after the setup commands. This  can  be  suppressed  by  creating  a  file
              /run/autopkgtest_no_reboot.stamp.

       --setup-commands-boot=commands
              Run commands after the --setup-commands, and after every reboot. For example, these
              commands could be used to add files in a tmpfs.

              These commands never cause the testbed to be rebooted (because that could  lead  to
              an infinite loop). Otherwise, they are just like the --setup-commands.

              This option can be specified multiple times.

       --add-apt-source='deb http://MIRROR SUITE COMPONENT...'
              Add  the  given apt source to /etc/apt/sources.list.d and update it, before running
              any --setup-commands.

              This option can be specified multiple times.

       --add-apt-release='RELEASE'
              Add the given apt RELEASE to /etc/apt/sources.list.d and update it, before  running
              any  --setup-commands.   The  mirror and components to use are copied from the very
              first existing APT sources.list entry. Both binary ("deb") and  source  ("deb-src")
              entries are added.

              This option can be specified multiple times.

       --apt-upgrade | -U
              Run  apt-get  update  and apt-get dist-upgrade -y in the testbed before running the
              tests.  Any --add-apt-source or --apt-pocket options take  effect  first,  so  this
              will upgrade packages from those sources if appropriate.

       --apt-default-release=SUITE
              Set's APT::Default-Release value to the provided value. For apt pinning (related to
              --apt-pocket, and --pin-packages) to work properly,  APT::Default-Release  must  be
              set to the release that should provide the packages that are not pinned. For Debian
              and Ubuntu, this is  normally  automatically  detected  from  the  first  entry  in
              /etc/apt/sources.list.

       --apt-pocket=pocket[=pkgname,src:srcname,...]
              Add   apt   sources   for   release-pocket.  This  finds  the  first  deb  line  in
              /etc/apt/sources.list which does not already specify a pocket and adds  a  deb  and
              deb-src  line  with  that pocket to /etc/apt/sources.list.d/pocket.list.  This also
              calls apt-get update for the new pocket (but not for anything else).

              If a package list is given after =, set up apt pinning to use only  those  packages
              from  pocket.   An entry "src:srcname" expands to all binary packages built by that
              source.  This can be used for minimizing dependencies taken  from  pocket  so  that
              package  updates  in  that  pocket  can be tested independently from each other for
              better isolation.  Attention: This does not currently resolve some situations where
              dependencies  of  the  given  packages can only be resolved in the given pocket. In
              that case the apt pinning will be removed and package installation will be  retried
              with the entirety of pocket.

       --copy=HOSTPATH:TESTBEDPATH
              Copy  file  or  directory from host into testbed after opening. This happens before
              --setup-commands thus you can use these files in the setup commands.

       --env=VAR=value
              Set arbitrary environment variable in the build and test. Can be specified multiple
              times.

       --pin-packages=RELEASE=PACKAGE[,PACKAGE2]
              Set up apt pinning to use only those packages from RELEASE.  An entry "src:srcname"
              expands to all binary packages  built  by  that  source.   This  can  be  used  for
              minimizing  dependencies taken from RELEASE so that package updates in that release
              can be tested independently from each other for better isolation.

       --no-apt-fallback
              Disable the apt-get fallback which is used with --apt-pocket or  --pin-packages  in
              case installation of dependencies fails due to strict pinning.

USER/PRIVILEGE HANDLING OPTIONS

       -u user | --user=user
              Run  builds  and  tests as user on the testbed.  This needs root on the testbed; if
              root on the testbed is not available then builds and tests run as whatever user  is
              provided.

       --gain-root=gain-root
              Prefixes  debian/rules  binary with gain-root.  The default is not to use anything,
              except that if --user is supplied or root on  the  testbed  is  not  available  the
              default is fakeroot.

DEBUGGING OPTIONS

       --debug|-d
              Include  additional  debugging  information  in  the trace log.  Each additional -d
              increases the debugging level; the current maximum is -ddd.  If  you  like  to  see
              what's going on, -d or -dd is recommended.

       --shell-fail|-s
              Run  an  interactive shell in the testbed after a failed build, test, or dependency
              installation.

       --shell
              Run an interactive shell in the testbed after every test.

TIMEOUT OPTIONS

       --timeout-which=seconds
              Use a different timeout for operations on or with  the  testbed.   There  are  five
              timeouts  affected by five values of which: short: supposedly short operations like
              setting up the testbed's apt and  checking  the  state  (default:  100s);  install:
              installation  of packages including dependencies (default: 3,000s); test: test runs
              (default: 10,000s); copy: copy files/directories between host and testbed (default:
              300s);  and  build:  builds (default: 100,000s).  The value must be specified as an
              integer number of seconds.

       --timeout-factor=double
              Multiply all of the default timeouts by the specified factor  (see  --timeout-which
              above).  Only the defaults are affected; explicit timeout settings are used exactly
              as specified.

LOCALE OPTIONS

       --set-lang=langval
              When running commands on  the  testbed,  sets  the  LANG  environment  variable  to
              langval.  The default in autopkgtest is to set it to C.UTF-8.

OTHER OPTIONS

       --no-auto-control
              Disable  automatic  test generation with autodep8, even if it is installed. In that
              case, packages without tests will exit with code 8 ("No  tests  in  this  package")
              just like without autodep8.

       --build-parallel=N
              Set  parallel=N  DEB_BUILD_OPTION  for  building  packages.  By default this is the
              number of available processors. This is mostly useful in containers where  you  can
              restrict the available RAM, but not restrict the number of CPUs.

       -h|--help
              Show command line help and exit.

VIRTUALIZATION SERVER

       -- virt-server virt-server-arg...
              Specifies  the  virtualisation regime server, as a command and arguments to invoke.
              virt-server must be an existing autopkgtest virtualization server such  as  schroot
              or qemu.

              All  the  remaining arguments and options after -- are passed to the virtualisation
              server program. See the manpages of the individual servers for how to use them.

OUTPUT FORMAT

       During a normal test run, one line is printed for each test.  This  consists  of  a  short
       string  identifying  the  test, some horizontal whitespace, and one of PASS, PASS details,
       FAIL reason, SKIP reason, or FLAKY reason where the pass/fail indication is  separated  by
       any reason or details by some horizontal whitespace.

       The  string  to  identify  the  test  consists  of a short alphanumeric string invented by
       autopkgtest to distinguish different command-line arguments,  the  argid,  followed  by  a
       hyphen and the test name.

       SKIP  indicates  that  a  test was not run, or that the test code was started but detected
       that the test could not complete,  for  instance  because  a  required  resource  was  not
       available.

       FLAKY indicates that a test would ordinarily have failed, but because this particular test
       is known to be unreliable, the failure was ignored.

       Sometimes a SKIP will be reported  when  the  name  of  the  test  is  not  known  or  not
       applicable:  for  example,  when  there  are no tests in the package, or a there is a test
       stanza which contains features not understood by this version  of  autopkgtest.   In  this
       case * will appear where the name of the test should be.

       If autopkgtest detects that erroneous package(s) are involved, it will print the two lines
       blame: blamed-thing...  and badpkg: message.  Here each whitespace-separated  blamed-thing
       is  one  of  arg:argument  (representing  a  pathname  found  in a command line argument),
       dsc:package (a source package name), deb:package (a binary package name) or possibly other
       strings  to  be  determined.   This  indicates  which arguments and/or packages might have
       contributed to the problem; the ones which were processed  most  recently  and  which  are
       therefore most likely to be the cause of a problem are listed last.

CONFIGURATION FILES

       If  you  use  lots of options or nontrivial virt server arguments, you can put any part of
       the command line into a text file, with one line per option. E. g. you can create  a  file
       sid.cfg with contents like

              -s
              --output-dir=/tmp/testout
              --apt-upgrade
              --
              schroot
              sid

       and then run

              autopkgtest foo_1_amd64.changes @sid.cfg

       The  contents  of  the  configuration  file will be expanded in-place as if you would have
       given its contents on the command line. Please ensure that you don't place spaces  between
       short options and their values, they would become a part of the argument value.

EXIT STATUS

       0    all tests passed
       2    at least one test was skipped (or at least one flaky test failed)
       4    at least one test failed
       6    at least one test failed and at least one test skipped
       8    no tests in this package, or all non-superficial tests were skipped
       12   erroneous package
       14   erroneous package and at least one test skipped
       16   testbed failure
       20   other unexpected failures including bad usage

SEE ALSO

       /usr/share/doc/autopkgtest/README.running-tests.rst.gz
       /usr/share/doc/autopkgtest/README.package-tests.rst.gz

AUTHORS AND COPYRIGHT

       This  manpage  is  part  of  autopkgtest,  a  tool  for  testing  Debian  binary packages.
       autopkgtest is Copyright (C) 2006-2014 Canonical Ltd.

       See /usr/share/doc/autopkgtest/CREDITS for the  list  of  contributors  and  full  copying
       conditions.