Provided by: autopkgtest_5.25ubuntu1_all bug

NAME

       autopkgtest - test an installed binary package using the source package's tests

SYNOPSIS

       autopkgtest [options...]  [testbinary...]  testsrc -- virt-server [virt-server-arg...]

DESCRIPTION

       autopkgtest  runs  tests  on  binary  Debian  packages,  as  installed on a system (called
       "testbed").  The tests are supplied in the source package.

       autopkgtest runs each test supplied by a particular package and reports the  results.   It
       drives  the  specified  virtualisation  regime as appropriate, parses the test description
       metadata, and arranges for data to be copied to and from the testbed as required.

       See /usr/share/doc/autopkgtest/README.running-tests.rst.gz for an introduction  about  how
       to use autopkgtest.

TESTING A DEBIAN PACKAGE

       Positional  (non-option) arguments specify exactly one source package (containing the test
       code) and optionally some binary packages to test.

       testsrc can be one of:

       .dsc file
              Run tests from Debian .dsc source package. By default  the  package  will  also  be
              built  and  the  resulting  binaries  will be used to satisfy test dependencies; to
              disable that, specify the -B/--no-built-binaries option.

       source package directory
              Run tests from a Debian source tree directory. If that is an unbuilt tree, this  is
              very  similar  to specifying a .dsc. If that is a built tree, all test dependencies
              get satisfied by archive packages, unless  you  explicitly  specify  locally  built
              .debs as well.

              Attention: If you just specify a bare directory name which is a legal Debian source
              package name, it will be interpreted as the  latter  (see  below).  In  this  case,
              prefix the directory name with ./.

       current directory
              If  no source package is specified on the command line and the current directory is
              a Debian source package, this will be tested.

       source package name
              Downloads the given source package name with apt-get source in the testbed and  run
              its  tests. This is similar to specifying a .dsc but avoids copying the source from
              the host to the testbed. Possibly built binaries  (if  the  test  specifies  build-
              needed)  will not be used to satisfy dependencies, as usually in this mode you want
              to test binaries from a real archive.

       git URL or URL#branch
              Git-clones the given URL (which must contain an unbuilt  Debian  source  tree)  and
              runs  the  tests  from  that.   If branch is given, this branch will be checked out
              instead of the default (usually "master"). This can also be a more general  refspec
              such as #refs/pull/123/head" for a GitHub pull request.

              This  is  very similar to cloning manually and specifying the checkout directory as
              test; i. e.  this is commonly used with --no-built-binaries.  The git package  will
              be installed if necessary.

       .changes file
              Run  tests from the .dsc source package in the given .changes file. If the .changes
              contains .deb packages, they will be used  for  the  test.   Acts  as  if  you  had
              specified  the  .debs  and  .dsc from the .changes file as explicit arguments. Note
              that if the .changes contains only debs, the corresponding .dsc still needs  to  be
              specified alongside, or the current directory must be the source package.

       All  other  positional  arguments must be .deb binary packages. They will be used for both
       build and test dependencies of the source package. If any binary package  is  given,  then
       --no-built-binaries is implied.

TEST OPTIONS

       -B | --no-built-binaries
              Binaries from unbuilt source packages (see above) will not be built or ignored, and
              dependencies are satisfied with packages from the archive.  Note  that  the  source
              package still gets built if a test requires build-needed.

       --override-control=PATH
              Read the test metadata from PATH instead of debian/tests/control.

       --test-name=TEST
              Run only the given test name (from test control file).  If this option is used more
              than once, all the named  tests  are  run.   This  replaces  --testname,  which  is
              deprecated.

       --skip-test=TEST
              Skip  the  given  test  name (from test control file).  If this option is used more
              than once, all the named tests are skipped.

LOGGING OPTIONS

       If you don't specify any option, autopkgtest only writes its output/results to stderr.

       -o dir | --output-dir=dir
              Specifies that test artifacts (stderr and stdout from  the  tests,  the  log  file,
              built  binary packages etc.) should be placed in the given directory.  dir must not
              exist yet or be empty, otherwise autopkgtest will refuse to use it.

       -l logfile | --log-file=logfile
              Specifies that the trace log should be written to logfile instead of to output-dir.

       --summary=summary
              Specifies that a summary of the outcome should be written to summary.   The  events
              in the summary are written to the log in any case.

       -q | --quiet
              Do  not  send  a copy of autopkgtest's trace logstream to stderr.  This option does
              not affect the copy sent to logfile or output-dir.  Note  that  without  the  trace
              logstream it can be very hard to diagnose problems.

TEST BED SETUP OPTIONS

       --setup-commands=commands
              Run  commands after opening the testbed. This can be used to do anything that isn't
              supported by an existing autopkgtest command. If commands is an existing file name,
              the  commands are read from that; otherwise it is a string with the actual commands
              that gets run as-is. File names without directory will  be  searched  in  both  the
              current  directory and in /usr/share/autopkgtest/setup-commands/ so you do not need
              to give the full path for setup scripts shipped with autopkgtest.

              Normally, if the setup commands fail, autopkgtest will consider  this  a  transient
              testbed  error  (exit  code 16). However, if the setup commands exit with code 100,
              autopkgtest will consider this an "erroneous package" (exit code  12)  instead,  so
              this  can  be  used  to e. g. detect upgrade errors to a new version. Note that apt
              exits with exit code 100 in these cases.

              This option can be specified multiple times.

              If --user is given or the test bed provides a suggested-normal-user capability, the
              $AUTOPKGTEST_NORMAL_USER environment variable will be set to that user.

              If  the  setup  commands  affect  anything  in  boot  directories  (like  /boot  or
              /lib/systemd/system) and the  testbed  supports  rebooting,  the  testbed  will  be
              rebooted  after  the  setup  commands.  This  can  be suppressed by creating a file
              /run/autopkgtest_no_reboot.stamp.

       --setup-commands-boot=commands
              Run commands after the --setup-commands, and after every reboot. For example, these
              commands could be used to add files in a tmpfs.

              These  commands  never cause the testbed to be rebooted (because that could lead to
              an infinite loop). Otherwise, they are just like the --setup-commands.

              This option can be specified multiple times.

       --add-apt-source='deb http://MIRROR SUITE COMPONENT...'
              Add the given apt source to /etc/apt/sources.list.d and update it,  before  running
              any --setup-commands.

              This option can be specified multiple times.

       --add-apt-release='RELEASE'
              Add  the given apt RELEASE to /etc/apt/sources.list.d and update it, before running
              any --setup-commands.  The mirror and components to use are copied  from  the  very
              first  existing  APT sources.list entry. Both binary ("deb") and source ("deb-src")
              entries are added.

              This option can be specified multiple times.

       --apt-upgrade | -U
              Run apt-get update and apt-get dist-upgrade -y in the testbed  before  running  the
              tests.   Any  --add-apt-source  or  --apt-pocket options take effect first, so this
              will upgrade packages from those sources if appropriate.

       --apt-default-release=SUITE
              Set's APT::Default-Release value to the provided value. For apt pinning (related to
              --apt-pocket,  and  --pin-packages)  to work properly, APT::Default-Release must be
              set to the release that should provide the packages that are not pinned. For Debian
              and  Ubuntu,  this  is  normally  automatically  detected  from  the first entry in
              /etc/apt/sources.list.

       --apt-pocket=pocket[=pkgname,src:srcname,...]
              Add  apt  sources  for  release-pocket.  This  finds  the   first   deb   line   in
              /etc/apt/sources.list  which  does  not already specify a pocket and adds a deb and
              deb-src line with that pocket to  /etc/apt/sources.list.d/pocket.list.   This  also
              calls apt-get update for the new pocket (but not for anything else).

              If  a  package list is given after =, set up apt pinning to use only those packages
              from pocket.  An entry "src:srcname" expands to all binary packages built  by  that
              source.   This  can  be  used for minimizing dependencies taken from pocket so that
              package updates in that pocket can be tested  independently  from  each  other  for
              better isolation.  Attention: This does not currently resolve some situations where
              dependencies of the given packages can only be resolved in  the  given  pocket.  In
              that  case the apt pinning will be removed and package installation will be retried
              with the entirety of pocket.

       --copy=HOSTPATH:TESTBEDPATH
              Copy file or directory from host into testbed after opening.  This  happens  before
              --setup-commands thus you can use these files in the setup commands.

       --env=VAR=value
              Set arbitrary environment variable in the build and test. Can be specified multiple
              times.

       --pin-packages=RELEASE=PACKAGE[,PACKAGE2]
              Set up apt pinning to use only those packages from RELEASE.  An entry "src:srcname"
              expands  to  all  binary  packages  built  by  that  source.   This can be used for
              minimizing dependencies taken from RELEASE so that package updates in that  release
              can be tested independently from each other for better isolation.

       --no-apt-fallback
              Disable  the  apt-get fallback which is used with --apt-pocket or --pin-packages in
              case installation of dependencies fails due to strict pinning.

       --ignore-restrictions=RESTRICTION,RESTRICTION...
              If a test would normally be skipped because it has Restrictions:  RESTRICTION,  run
              it anyway. Can be specified multiple times.

              For example, you might ignore the restriction isolation-machine when using the null
              virtualization server if  you  know  that  autopkgtest  itself  is  running  on  an
              expendable  virtual  machine.  These options also work for unknown restrictions, so
              they can be used when experimenting with new restrictions.

USER/PRIVILEGE HANDLING OPTIONS

       -u user | --user=user
              Run builds and tests as user on the testbed.  This needs root on  the  testbed;  if
              root  on the testbed is not available then builds and tests run as whatever user is
              provided.

       --gain-root=gain-root
              Prefixes debian/rules binary with gain-root.  The default is not to  use  anything,
              except  that  if  --user  is  supplied  or root on the testbed is not available the
              default is fakeroot.

DEBUGGING OPTIONS

       --debug|-d
              Include additional debugging information in the  trace  log.   Each  additional  -d
              increases  the  debugging  level;  the current maximum is -ddd.  If you like to see
              what's going on, -d or -dd is recommended.

       --shell-fail|-s
              Run an interactive shell in the testbed after a failed build, test,  or  dependency
              installation.

       --shell
              Run an interactive shell in the testbed after every test.

TIMEOUT OPTIONS

       --timeout-which=seconds
              Use  a  different  timeout  for  operations on or with the testbed.  There are five
              timeouts affected by five values of which: short: supposedly short operations  like
              setting  up  the  testbed's  apt  and  checking the state (default: 100s); install:
              installation of packages including dependencies (default: 3,000s); test: test  runs
              (default: 10,000s); copy: copy files/directories between host and testbed (default:
              300s); and build: builds (default: 100,000s).  The value must be  specified  as  an
              integer number of seconds.

       --timeout-factor=double
              Multiply  all  of the default timeouts by the specified factor (see --timeout-which
              above).  Only the defaults are affected; explicit timeout settings are used exactly
              as specified.

LOCALE OPTIONS

       --set-lang=langval
              When  running  commands  on  the  testbed,  sets  the  LANG environment variable to
              langval.  The default in autopkgtest is to set it to C.UTF-8.

OTHER OPTIONS

       --no-auto-control
              Disable automatic test generation with autodep8, even if it is installed.  In  that
              case,  packages  without  tests  will exit with code 8 ("No tests in this package")
              just like without autodep8.

       --build-parallel=N
              Set parallel=N DEB_BUILD_OPTION for building  packages.  By  default  this  is  the
              number  of  available processors. This is mostly useful in containers where you can
              restrict the available RAM, but not restrict the number of CPUs.

       --needs-internet=run|try|skip
              Define how to handle the needs-internet restriction. With "try" tests  with  needs-
              internet  restrictions  will be run, but if they fail they will be treated as flaky
              tests. With "skip" these tests will skipped immediately and will not be  run.  With
              "run" the restriction is basically ignored, this is the default.

       -V|--validate
              Validate the test control file and exit without running any tests.

       -h|--help
              Show command line help and exit.

VIRTUALIZATION SERVER

       -- virt-server virt-server-arg...
              Specifies  the  virtualisation regime server, as a command and arguments to invoke.
              virt-server must be an existing autopkgtest virtualization server such  as  schroot
              or qemu.

              All  the  remaining arguments and options after -- are passed to the virtualisation
              server program. See the manpages of the individual servers for how to use them.

OUTPUT FORMAT

       During a normal test run, one line is printed for each test.  This  consists  of  a  short
       string  identifying  the  test, some horizontal whitespace, and one of PASS, PASS details,
       FAIL reason, SKIP reason, or FLAKY reason where the pass/fail indication is  separated  by
       any reason or details by some horizontal whitespace.

       The  string  to  identify  the  test  consists  of a short alphanumeric string invented by
       autopkgtest to distinguish different command-line arguments,  the  argid,  followed  by  a
       hyphen and the test name.

       SKIP  indicates  that  a  test was not run, or that the test code was started but detected
       that the test could not complete,  for  instance  because  a  required  resource  was  not
       available.

       FLAKY indicates that a test would ordinarily have failed, but because this particular test
       is known to be unreliable, the failure was ignored.

       Sometimes a SKIP will be reported  when  the  name  of  the  test  is  not  known  or  not
       applicable:  for  example,  when  there  are no tests in the package, or a there is a test
       stanza which contains features not understood by this version  of  autopkgtest.   In  this
       case * will appear where the name of the test should be.

       If autopkgtest detects that erroneous package(s) are involved, it will print the two lines
       blame: blamed-thing...  and badpkg: message.  Here each whitespace-separated  blamed-thing
       is  one  of  arg:argument  (representing  a  pathname  found  in a command line argument),
       dsc:package (a source package name), deb:package (a binary package name) or possibly other
       strings  to  be  determined.   This  indicates  which arguments and/or packages might have
       contributed to the problem; the ones which were processed  most  recently  and  which  are
       therefore most likely to be the cause of a problem are listed last.

CONFIGURATION FILES

       If  you  use  lots of options or nontrivial virt server arguments, you can put any part of
       the command line into a text file, with one line per option. E. g. you can create  a  file
       sid.cfg with contents like

              -s
              --output-dir=/tmp/testout
              --apt-upgrade
              --
              schroot
              sid

       and then run

              autopkgtest foo_1_amd64.changes @sid.cfg

       The  contents  of  the  configuration  file will be expanded in-place as if you would have
       given its contents on the command line. Please ensure that you don't place spaces  between
       short options and their values, they would become a part of the argument value.

EXIT STATUS

       0    all tests passed
       2    at least one test was skipped (or at least one flaky test failed)
       4    at least one test failed
       6    at least one test failed and at least one test skipped
       8    no tests in this package, or all non-superficial tests were skipped
       12   erroneous package
       14   erroneous package and at least one test skipped
       16   testbed failure
       20   other unexpected failures including bad usage

SEE ALSO

       /usr/share/doc/autopkgtest/README.running-tests.rst.gz
       /usr/share/doc/autopkgtest/README.package-tests.rst.gz

AUTHORS AND COPYRIGHT

       This  manpage  is  part  of  autopkgtest,  a  tool  for  testing  Debian  binary packages.
       autopkgtest is Copyright (C) 2006-2014 Canonical Ltd.

       See /usr/share/doc/autopkgtest/CREDITS for the  list  of  contributors  and  full  copying
       conditions.