Provided by: autopkgtest_5.5ubuntu1_all bug

NAME

       autopkgtest - test an installed binary package using the source package's tests

SYNOPSIS

       autopkgtest [options...]  [testbinary...]  testsrc -- virt-server [virt-server-arg...]

DESCRIPTION

       autopkgtest  runs  tests  on  binary  Debian  or  Click packages, as installed on a system
       (called "testbed").  The tests are supplied in the source package.

       autopkgtest runs each test supplied by a particular package and reports the  results.   It
       drives  the  specified  virtualisation  regime as appropriate, parses the test description
       metadata, and arranges for data to be copied to and from the testbed as required.

       See /usr/share/doc/autopkgtest/README.running-tests.rst.gz for an introduction  about  how
       to use autopkgtest.

TESTING A DEBIAN PACKAGE

       Positional  (non-option) arguments specify exactly one source package (containing the test
       code) and optionally some binary packages to test.

       testsrc can be one of:

       .dsc file
              Run tests from Debian .dsc source package. By default  the  package  will  also  be
              built  and  the  resulting  binaries  will be used to satisfy test dependencies; to
              disable that, specify the -B/--no-built-binaries option.

       source package directory
              Run tests from a Debian source tree directory. If that is an unbuilt tree, this  is
              very  similar  to specifying a .dsc. If that is a built tree, all test dependencies
              get satisfied by archive packages, unless  you  explicitly  specify  locally  built
              .debs as well.

              Attention: If you just specify a bare directory name which is a legal Debian source
              package name, it will be interpreted as the  latter  (see  below).  In  this  case,
              prefix the directory name with ./.

       current directory
              If  no source package is specified on the command line and the current directory is
              a Debian source package, this will be tested.

       source package name
              Downloads the given source package name with apt-get source in the testbed and  run
              its  tests. This is similar to specifying a .dsc but avoids copying the source from
              the host to the testbed. Possibly built binaries  (if  the  test  specifies  needs-
              build)  will  not be used to satisfy dependencies, as usually in this mode you want
              to test binaries from a real archive.

       git URL or URL#branch
              Git-clones the given URL (which must contain an unbuilt  Debian  source  tree)  and
              runs  the  tests  from  that.   If branch is given, this branch will be checked out
              instead of the default (usually "master"). This can also be a more general  refspec
              such as #refs/pull/123/head" for a GitHub pull request.

              This  is  very similar to cloning manually and specifying the checkout directory as
              test; i. e.  this is commonly used with --no-built-binaries.  The git package  will
              be installed if necessary.

       .changes file
              Run  tests from the .dsc source package in the given .changes file. If the .changes
              contains .deb packages, they will be used  for  the  test.   Acts  as  if  you  had
              specified  the  .debs  and  .dsc from the .changes file as explicit arguments. Note
              that if the .changes contains only debs, the corresponding .dsc still needs  to  be
              specified alongside, or the current directory must be the source package.

       All  other  positional  arguments must be .deb binary packages. They will be used for both
       build and test dependencies of the source package. If any binary package  is  given,  then
       --no-built-binaries is implied.

TESTING A CLICK PACKAGE

       This  needs  exactly  one  .click  package. If its manifest specifies an x-source URL, the
       corresponding source package will be downloaded and tests run from there;  then  a  source
       directory does not need to be given. Otherwise specifying a source directory as positional
       argument is mandatory.

       For testing a local  .click  package,  give  its  path  to  autopkgtest.   For  testing  a
       preinstalled  click,  use  --installed-click  com.example.myclick (this cannot be given as
       positional argument as this cannot be disambiguated from Debian source package names).

       Examples with explicit click source directory:

              autopkgtest src/myclick_0.1_all.click com.example.myclick src/myclick/ -- [...]

              autopkgtest --installed-click com.example.myclick src/myclick/ -- [...]

       Examples without click source directory; click package needs  to  have  x-source  manifest
       entry:

              autopkgtest src/myclick_0.1_all.click -- [...]

              autopkgtest --installed-click com.example.myclick -- [...]

TEST OPTIONS

       -B | --no-built-binaries
              Binaries from unbuilt source packages (see above) will not be built or ignored, and
              dependencies are satisfied with packages from the archive.  Note  that  the  source
              package still gets built if a test requires build-needed.

       --override-control=PATH
              Read  the  test  metadata  from  PATH  instead  of debian/tests/control (for Debian
              sources) or the Click manifest.

       --test-name=TEST
              Run only the given test name (from test control file).   This  replaces  --testname
              which is deprecated.

LOGGING OPTIONS

       If you don't specify any option, autopkgtest only writes its output/results to stderr.

       -o dir | --output-dir=dir
              Specifies  that  test  artifacts  (stderr  and stdout from the tests, the log file,
              built binary packages etc.) should be placed in the given directory.  dir must  not
              exist yet or be empty, otherwise autopkgtest will refuse to use it.

       -l logfile | --log-file=logfile
              Specifies that the trace log should be written to logfile instead of to output-dir.

       --summary=summary
              Specifies  that  a summary of the outcome should be written to summary.  The events
              in the summary are written to the log in any case.

       -q | --quiet
              Do not send a copy of autopkgtest's trace logstream to stderr.   This  option  does
              not  affect  the  copy  sent to logfile or output-dir.  Note that without the trace
              logstream it can be very hard to diagnose problems.

TEST BED SETUP OPTIONS

       --setup-commands=commands
              Run commands after opening the testbed. This can be used to do anything that  isn't
              supported by an existing autopkgtest command. If commands is an existing file name,
              the commands are read from that; otherwise it is a string with the actual  commands
              that  gets  run  as-is.  File  names without directory will be searched in both the
              current directory and in /usr/share/autopkgtest/setup-commands/ so you do not  need
              to give the full path for setup scripts shipped with autopkgtest.

              Normally,  if  the  setup commands fail, autopkgtest will consider this a transient
              testbed error (exit code 16). However, if the setup commands exit  with  code  100,
              autopkgtest  will  consider  this an "erroneous package" (exit code 12) instead, so
              this can be used to e. g. detect upgrade errors to a new  version.  Note  that  apt
              exits with exit code 100 in these cases.

              This option can be specified multiple times.

              If --user is given or the test bed provides a suggested-normal-user capability, the
              $AUTOPKGTEST_NORMAL_USER environment variable will be set to that user.

              If  the  setup  commands  affect  anything  in  boot  directories  (like  /boot  or
              /lib/systemd/system)  and  the  testbed  supports  rebooting,  the  testbed will be
              rebooted after the setup commands. This  can  be  suppressed  by  creating  a  file
              /run/autopkgtest_no_reboot.stamp.

       --setup-commands-boot=commands
              Run commands after the --setup-commands, and after every reboot. For example, these
              commands could be used to add files in a tmpfs.

              These commands never cause the testbed to be rebooted (because that could  lead  to
              an infinite loop). Otherwise, they are just like the --setup-commands.

              This option can be specified multiple times.

       --add-apt-source='deb http://MIRROR SUITE COMPONENT...'
              Add  the  given apt source to /etc/apt/sources.list.d and update it, before running
              any --setup-commands.

              This option can be specified multiple times.

       --add-apt-release='RELEASE'
              Add the given apt RELEASE to /etc/apt/sources.list.d and update it, before  running
              any  --setup-commands.   The  mirror and components to use are copied from the very
              first existing APT sources.list entry. Both binary ("deb") and  source  ("deb-src")
              entries are added.

              This option can be specified multiple times.

       --apt-upgrade | -U
              Run  apt-get  update  and apt-get dist-upgrade -y in the testbed before running the
              tests.  Any --add-apt-source or --apt-pocket options take  effect  first,  so  this
              will upgrade packages from those sources if appropriate.

       --apt-default-release=SUITE
              Set's APT::Default-Release value to the provided value. For apt pinning (related to
              --apt-pocket, and --pin-packages) to work properly,  APT::Default-Release  must  be
              set to the release that should provide the packages that are not pinned. For Debian
              and Ubuntu, this is  normally  automatically  detected  from  the  first  entry  in
              /etc/apt/sources.list.

       --apt-pocket=pocket[=pkgname,src:srcname,...]
              Add   apt   sources   for   release-pocket.  This  finds  the  first  deb  line  in
              /etc/apt/sources.list which does not already specify a pocket and adds  a  deb  and
              deb-src  line  with  that pocket to /etc/apt/sources.list.d/pocket.list.  This also
              calls apt-get update for the new pocket (but not for anything else).

              If a package list is given after =, set up apt pinning to use only  those  packages
              from  pocket.   An entry "src:srcname" expands to all binary packages built by that
              source.  This can be used for minimizing dependencies taken  from  pocket  so  that
              package  updates  in  that  pocket  can be tested independently from each other for
              better isolation.  Attention: This does not currently resolve some situations where
              dependencies  of  the  given  packages can only be resolved in the given pocket. In
              that case the apt pinning will be removed and package installation will be  retried
              with the entirety of pocket.

       --copy=HOSTPATH:TESTBEDPATH
              Copy  file  or  directory from host into testbed after opening. This happens before
              --setup-commands thus you can use these files in the setup commands.

       --env=VAR=value
              Set arbitrary environment variable in the build and test. Can be specified multiple
              times.

       --pin-packages=RELEASE=PACKAGE[,PACKAGE2]
              Set up apt pinning to use only those packages from RELEASE.  An entry "src:srcname"
              expands to all binary packages  built  by  that  source.   This  can  be  used  for
              minimizing  dependencies taken from RELEASE so that package updates in that release
              can be tested independently from each other for better isolation.

USER/PRIVILEGE HANDLING OPTIONS

       -u user | --user=user
              Run builds and tests as user on the testbed.  This needs root on  the  testbed;  if
              root  on the testbed is not available then builds and tests run as whatever user is
              provided.

       --gain-root=gain-root
              Prefixes debian/rules binary with gain-root.  The default is not to  use  anything,
              except  that  if  --user  is  supplied  or root on the testbed is not available the
              default is fakeroot.

DEBUGGING OPTIONS

       --debug|-d
              Include additional debugging information in the  trace  log.   Each  additional  -d
              increases  the  debugging  level;  the current maximum is -ddd.  If you like to see
              what's going on, -d or -dd is recommended.

       --shell-fail|-s
              Run an interactive shell in the testbed after a failed build, test,  or  dependency
              installation.

       --shell
              Run an interactive shell in the testbed after every test.

TIMEOUT OPTIONS

       --timeout-which=seconds
              Use  a  different  timeout  for  operations on or with the testbed.  There are five
              timeouts affected by five values of which: short: supposedly short operations  like
              setting  up  the  testbed's  apt  and  checking the state (default: 100s); install:
              installation of packages including dependencies (default: 3,000s); test: test  runs
              (default: 10,000s); copy: copy files/directories between host and testbed (default:
              300s); and build: builds (default: 100,000s).  The value must be  specified  as  an
              integer number of seconds.

       --timeout-factor=double
              Multiply  all  of the default timeouts by the specified factor (see --timeout-which
              above).  Only the defaults are affected; explicit timeout settings are used exactly
              as specified.

LOCALE OPTIONS

       --set-lang=langval
              When  running  commands  on  the  testbed,  sets  the  LANG environment variable to
              langval.  The default in autopkgtest is to set it to C.UTF-8.

OTHER OPTIONS

       --no-auto-control
              Disable automatic test generation with autodep8, even if it is installed.  In  that
              case,  packages  without  tests  will exit with code 8 ("No tests in this package")
              just like without autodep8.

       --build-parallel=N
              Set parallel=N DEB_BUILD_OPTION for building  packages.  By  default  this  is  the
              number  of  available processors. This is mostly useful in containers where you can
              restrict the available RAM, but not restrict the number of CPUs.

       -h|--help
              Show command line help and exit.

VIRTUALIZATION SERVER

       -- virt-server virt-server-arg...
              Specifies the virtualisation regime server, as a command and arguments  to  invoke.
              virt-server  must  be an existing autopkgtest virtualization server such as schroot
              or qemu.

              All the remaining arguments and options after -- are passed to  the  virtualisation
              server program. See the manpages of the individual servers for how to use them.

OUTPUT FORMAT

       During  a  normal  test  run, one line is printed for each test.  This consists of a short
       string identifying the test, some horizontal whitespace, and one  of  PASS,  FAIL  reason,
       SKIP  reason, or FLAKY reason where the pass/fail indication is separated by any reason by
       some horizontal whitespace.

       The string to identify the test consists  of  a  short  alphanumeric  string  invented  by
       autopkgtest  to  distinguish  different  command-line  arguments, the argid, followed by a
       hyphen and the test name.

       SKIP indicates that a test was not run, or that the test code  was  started  but  detected
       that  the  test  could  not  complete,  for  instance  because a required resource was not
       available.

       FLAKY indicates that a test would ordinarily have failed, but because this particular test
       is known to be unreliable, the failure was ignored.

       Sometimes  a  SKIP  will  be  reported  when  the  name  of  the  test is not known or not
       applicable: for example, when there are no tests in the package, or  a  there  is  a  test
       stanza  which  contains  features  not understood by this version of autopkgtest.  In this
       case * will appear where the name of the test should be.

       If autopkgtest detects that erroneous package(s) are involved, it will print the two lines
       blame:  blamed-thing...  and badpkg: message.  Here each whitespace-separated blamed-thing
       is one of arg:argument (representing  a  pathname  found  in  a  command  line  argument),
       dsc:package (a source package name), deb:package (a binary package name) or possibly other
       strings to be determined.  This indicates  which  arguments  and/or  packages  might  have
       contributed  to  the  problem;  the  ones which were processed most recently and which are
       therefore most likely to be the cause of a problem are listed last.

CONFIGURATION FILES

       If you use lots of options or nontrivial virt server arguments, you can put  any  part  of
       the  command  line into a text file, with one line per option. E. g. you can create a file
       sid.cfg with contents like

              -s
              --output-dir=/tmp/testout
              --apt-upgrade
              --
              schroot
              sid

       and then run

              autopkgtest foo_1_amd64.changes @sid.cfg

       The contents of the configuration file will be expanded in-place  as  if  you  would  have
       given  its contents on the command line. Please ensure that you don't place spaces between
       short options and their values, they would become a part of the argument value.

EXIT STATUS

       0    all tests passed
       2    at least one test was skipped (or at least one flaky test failed)
       4    at least one test failed
       6    at least one test failed and at least one test skipped
       8    no tests in this package
       12   erroneous package
       14   erroneous package and at least one test skipped
       16   testbed failure
       20   other unexpected failures including bad usage

SEE ALSO

       /usr/share/doc/autopkgtest/README.running-tests.rst.gz
       /usr/share/doc/autopkgtest/README.package-tests.rst.gz

AUTHORS AND COPYRIGHT

       This manpage  is  part  of  autopkgtest,  a  tool  for  testing  Debian  binary  packages.
       autopkgtest is Copyright (C) 2006-2014 Canonical Ltd.

       See  /usr/share/doc/autopkgtest/CREDITS  for  the  list  of  contributors and full copying
       conditions.