Provided by: charliecloud-tests_0.26-1_amd64 bug

NAME

       ch-test - Run some or all of the Charliecloud test suite

SYNOPSIS

          $ ch-test [PHASE] [--scope SCOPE] [--pack-fmt FMT] [ARGS]

DESCRIPTION

       Charliecloud  comes  with a comprehensive test suite that exercises the container workflow
       itself as well as a few example applications.  ch-test coordinates running the test suite.

       While the CLI has lots of options, the defaults are reasonable, and bare ch-test will give
       useful  results  in a few minutes on single-node, internet-connected systems with a few GB
       available in /var/tmp.

       The test suite requires a few GB (standard scope) or tens of GB (full  scope)  of  storage
       for test fixtures:

       • Builder storage (e.g., layer cache). This goes wherever the builder puts it.

       • Packed images directory: image tarballs or SquashFS files.

       • Unpacked images directory. Images are unpacked into and then run from here.

       • Filesystem  permissions directories. These are used to test that the kernel is enforcing
         permissions correctly. Note that this exercises the kernel, not Charliecloud, and can be
         omitted from routine Charliecloud testing.

       The  first  three  are  created  when  needed  if  they  don’t exist, while the filesystem
       permissions fixtures must be created manually,  in  order  to  accommodate  configurations
       where sudo is not available via the same login path used for running tests.

       The  packed  and  unpacked  image  directories  specified  for  testing are volatile.  The
       contents of these directories are deleted before the build and run phases, respectively.

       In all four cases, when creating directories, only the final path  component  is  created.
       Parent  directories  must  already  exist, i.e., ch-test uses the behavior of mkdir rather
       than mkdir -p.

       Some of the tests exercise parallel functionality. If ch-test is run  on  a  single  node,
       multiple cores will be used; if in a Slurm allocation, multiple nodes too.

       The  subset  of  tests  to  run mostly splits along two key dimensions. The phase is which
       parts of the workflow to run. Different parts of the workflow can be tested  on  different
       systems  by  copying  the necessary artifacts between them, e.g. by building images on one
       system and running them on another. The scope allows trading off thoroughness versus time.

       PHASE must be one of the following:

          build  Image building and associated functionality, with the selected builder.

          run    Running containers and associated functionality. This requires a  packed  images
                 directory  produced  by  a  successful build phase, which can be copied from the
                 build system if it’s not also the run system.

          examples
                 Example applications. Requires  an  unpacked  images  directory  produced  by  a
                 successful run phase.

          all    Execute phases build, run, and examples, in that order.

          mk-perm-dirs
                 Create the filesystem permissions directories. Requires --perm-dirs.

          clean  Delete  automatically-generated  test  files,  and  packed  and  unpacked  image
                 directories.

          rm-perm-dirs
                 Remove the filesystem permissions directories. Requires --perm-dirs.

          -f, --file FILE[:TEST]
                 Run the tests in the given file only, which can  be  an  arbitrary  .bats  file,
                 except  for  test.bats  under examples, where you must specify the corresponding
                 Dockerfile or Build file instead. This is somewhat brittle  and  typically  used
                 for  development  or  debugging.  For  example,  it  does  not check whether the
                 pre-requisites of whatever is in the file are satisfied. Often running build and
                 run first is sufficient, but this varies.

                 If  TEST  is  also  given, then run only tests with name containing that string,
                 skipping the others. The separator is a literal colon. If  the  string  contains
                 shell metacharacters such as space, you’ll need to quote the argument to protect
                 it from the shell.

       Scope is specified with:

          -s, --scope SCOPE
                 SCOPE must be one of the following:

                 • quick: Most important subset of workflow. Handy for development.

                 • standard: All tested workflow functionality and a selection of more  important
                   examples. (Default.)

                 • full: All available tests, including all examples.

       Image format is specified with:

          --pack-fmt FMT
                 FMT must be one of the following:

                 • squash-mount  or  🐘:  SquashFS  archive,  run directly from the archive using
                   ch-run’s internal SquashFUSE functionality. In this mode, tests  that  require
                   writing to the image are skipped.

                 • tar-unpack or 📠: Tarball, and the images are unpacked before running.

                 • squash-unpack or 🎃: SquashFS, and the images are unpacked before running.

                 Default:  $CH_TEST_PACK_FMT if set. Otherwise, if mksquashfs(1) is available and
                 ch-run was built with libsquashfuse support, then squash-mount, else tar-unpack.

       Additional arguments:

          -b, --builder BUILDER
                 Image builder to use. See ch-build(1) for how the default is selected.

          --dry-run
                 Print summary of what would be tested and then exit.

          -h, --help
                 Print usage and then exit.

          --img-dir DIR
                 Set unpacked images directory to DIR. In a multi-node allocation, this directory
                 may  not  be  shared  between  nodes. Default: $CH_TEST_IMGDIR if set; otherwise
                 /var/tmp/img.

          --lustre DIR
                 Use DIR for run-phase Lustre tests. Default: CH_TEST_LUSTREDIR if set; otherwise
                 skip them.

                 The  tests  will  create,  populate,  and  delete  a new subdirectory under DIR,
                 leaving everything else in DIR untouched.

          --pack-dir DIR
                 Set packed images directory to DIR. Default: $CH_TEST_TARDIR if  set;  otherwise
                 /var/tmp/pack.

          --pedantic (yes|no)
                 Some  tests  require configurations that are very specific (e.g., being a member
                 of at least two groups) or unusual (e.g., sudo to a  non-root  group).  If  yes,
                 then  fail  if  the requirement is not met; if no, then skip. The default is yes
                 for CI environments or people listed in README.md, no otherwise.

                 If yes and sudo seems to be available, implies --sudo.

          --perm-dir DIR
                 Add DIR to filesystem permission fixture directories; can be specified  multiple
                 times.  We recommend one such directory per mounted filesystem type whose kernel
                 module you do not trust; e.g., you probably don’t need to test your tmpfses, but
                 out-of-tree filesystems very likely need this.

                 Implies  --sudo. Default: CH_TEST_PERMDIRS if set; otherwise skip the filesystem
                 permissions tests.

          --sudo Enable things that require sudo, such as certain privilege escalation tests  and
                 creating/removing  the  filesystem  permissions  fixtures. Requires generic sudo
                 capabilities. Note that the Docker builder uses sudo docker  even  without  this
                 option.

EXIT STATUS

       Zero  if  all tests passed; non-zero if any failed. For setup and teardown phases, zero if
       everything was created or deleted correctly, non-zero otherwise.

BUGS

       Bats will wait until all descendant processes finish before exiting, so if you get into  a
       failure mode where a test sequence doesn’t clean up all its processes, ch-test will hang.

EXAMPLES

       Many  systems can simply use the defaults. To run the build, run, and examples phases on a
       single system, without the filesystem permissions tests:

          $ ch-test
          ch-test version 0.12

          ch-run: 0.12 /usr/local/bin/ch-run
          bats:   0.4.0 /usr/bin/bats
          tests:  /usr/local/libexec/charliecloud/test

          phase:                build run examples
          scope:                standard (default)
          builder:              docker (default)
          use generic sudo:     no (default)
          unpacked images dir:  /var/tmp/img (default)
          packed images dir:    /var/tmp/tar (default)
          fs permissions dirs:  skip (default)

          checking namespaces ...
          ok

          checking builder ...
          found: /usr/bin/docker 19.03.2

          bats build.bats build_auto.bats build_post.bats
           ✓ documentation seems sane
           ✓ version number seems sane
          [...]
          All tests passed.

       The next example is for a more complex setup like you might find in HPC centers:

          • Non-default fixture directories.

          • Non-default scope.

          • Different build and run systems.

          • Run the filesystem permissions tests.

       Output has been omitted.

          (mybox)$ ssh hpc-admin
          (hpc-admin)$ ch-test mk-perm-dirs --perm-dir /scratch/$USER/perms \
                                            --perm-dir /home/$USER/perms
          (hpc-admin)$ exit
          (mybox)$ ch-test build --scope full
          (mybox)$ scp -r /var/tmp/pack hpc:/scratch/$USER/pack
          (mybox)$ ssh hpc
          (hpc)$ salloc -N2
          (cn001)$ export CH_TEST_TARDIR=/scratch/$USER/pack
          (cn001)$ export CH_TEST_IMGDIR=/local/tmp
          (cn001)$ export CH_TEST_PERMDIRS="/scratch/$USER/perms /home/$USER/perms"
          (cn001)$ export CH_TEST_SCOPE=full
          (cn001)$ ch-test run
          (cn001)$ ch-test examples

REPORTING BUGS

       If Charliecloud was obtained from your Linux distribution,  use  your  distribution’s  bug
       reporting procedures.

       Otherwise, report bugs to: https://github.com/hpc/charliecloud/issues

SEE ALSO

       charliecloud(7)

       Full documentation at: <https://hpc.github.io/charliecloud>

COPYRIGHT

       2014–2021, Triad National Security, LLC