NAME

       tesh -- testing shell

SYNOPSIS

       tesh [options]... testsuite

DESCRIPTION

       Tesh is the testing shell, a specialized shell for running tests. It provides the
       specified input to the tested commands, and check that they produce the expected output
       and return the expected value.

OPTIONS

         --help              : display help
         --cd some/directory : ask tesh to switch the working directory before
                               launching the tests
         --setenv var=value  : set a specific environment variable
         --cfg arg           : add parameter --cfg=arg to each command line
         --log arg           : add parameter --log=arg to each command line
         --ignore-jenkins    : ignore all cruft generated on SimGrid
                               continuous integration servers

TEST SUITE FILE SYTAX

       A test suite is composed of one or several command blocks separated by empty lines, each
       of them being composed of a command to run, its input text and the expected output.

       The first char of each line specifies the type of line according to the following list.
       The second char of each line is ignored.

        `$' command to run in foreground
        `&' command to run in background

        `<' input to pass to the command
        `>' output expected from the command

        `!' metacommand, which can be one of:
            `timeout' <integer>|no
            `expect signal' <signal name>[|<signal name>]*
            `expect return' <integer>
            `output' <ignore|display>
            `output sort' [integer]
            `setenv <key>=<val>'
            `ignore' <regexp>

        `p' an informative message to print
        `#' a comment

       If the expected output do not match the produced output, or if the command did not end as
       expected, Tesh provides an error message (see the OUTPUT section below) and stops.

   Command blocks examples
       In a given command block, you can declare the command, its input and its expected output
       in the order that you see fit.

           $ cat
           < TOTO
           > TOTO

           > TOTO
           $ cat
           < TOTO

           > TOTO
           < TOTO
           $ cat

       You can group several commands together, provided that they don't have any input nor
       output.

           $ mkdir testdir
           $ cd testdir

   Enforcing the command return code
       By default, Tesh enforces that the tested command returns 0. If not, it fails with an
       appropriate message and returns code+40 itself.

       You specify that a given command block is expected to return another code as follows:

           # This command MUST return 42
           ! expect return 42
           $ sh -e "exit 42"

       The expect return construct applies only to the next command block.

   Commands that are expected to raise signals
       By default, Tesh detects when the command is killed by a signal (such as SEGV on
       segfaults). This is usually unexpected and unfortunate. But if not, you can specify that a
       given command block is expected to fail with a signal as follows:

           # This command MUST raise a segfault
           ! expect signal SIGSEGV
           $ ./some_failing_code

       The expect signal construct applies only to the next command block.

   Timeouts
       By default, no command is allowed to run more than 10 seconds. You can change this value
       as follows:

           # Allow some more time to the command
           ! timeout 60
           $ ./some_longer_command

       You can also disable the timeout completely by passing "no" as a value:

           # This command will never timeout
           ! timeout no
           $ ./some_very_long_but_safe_command

   Setting environment variables
       You can modify the environment of the tested commands as follows:

           ! setenv PATH=/bin
           $ my_command

       You can also set an envirmnent variable from the command line:

           tesh --setenv bindir=/opt/bin/

       And then use it within the tesh file:

           $ ${bindir}/myprogram

       Tesh also supports perl default value for undefined variables:

           $ ${bindir:=/usr/bin}/myprogram

   Not enforcing the expected output
       By default, the commands output is matched against the one expected, and an error is
       raised on discrepancy. Metacommands to change this:

       output ignore
           The output is completely discarded.

       output display
           The output is displayed, but no error is issued if it differs from the expected
           output.

       output sort
           The output and the expected output are sorted before comparison (see next section).

   Sorting output
       If the order of the command output changes between runs, you want to sort it before
       enforcing that it is exactly what you expect. In SimGrid for example, this happens when
       parallel execution is activated: User processes are run in parallel at each timestamp, and
       the output is not reproducible anymore. Until you sort the lines.

       You can sort the command output as follows:

           ! output sort
           $ ./some_multithreaded_command

       Sorting lines this ways often makes the tesh output very intricate, complicating the error
       analysis: the process logical order is defeated by the lexicographical sort.

       The solution is to prefix each line of your output with temporal information so that lines
       can be grouped by timestamps. The lexicographical sort then only applies to lines that
       occurred at the same timestamp. Here is a SimGrid example:

           # Sort only lines depending on the first 19 chars
           ! output sort 19
           $ ./some_simgrid_simulator --log=root.fmt:[%10.6r]%e(%i:%a@%h)%e%m%n

       This approach may seem surprising at the first glance but it does its job:

       Every timestamps remain separated, as it should;
       In each timestamp, the output order of processes become reproducible: that's the
       lexicographical order of their name;
       For each process, the order of its execution is preserved: its messages within a given
       timestamp are not reordered.

       That way, tesh can do its job (no false positive, no false negative) despite the
       unpredictable order of executions of processes within a timestamp, and reported errors
       remain easy to analyze (execution of a given process preserved).

       This example is very SimGrid oriented, but the feature could even be usable by others, who
       knows?

   Ignoring some output
       Some outputted lines can be ignored by setting the ignore command followed by a regular
       expression:

           ! ignore .*0x[0-9A-F]+\.
           $  printf 'word\nMemory address: 0x42AA42.\nanotherword\n'
           > word
           > anotherword

   Colored and formatted text
       Tesh removes ANSI/VT100 control sequences from outputted text to make easier the writing
       of tests.

           $ printf "I \033[0;31mlove\033[0m tesh\n"
           > I love tesh

BUILTIN COMMANDS

   mkfile: creating a file
       This command creates a file of the name provided as argument, and adds the content it gets
       as input.

         $ mkfile myFile
         > some content
         > to the file

       It is not possible to use the cat command, as one would expect, because stream
       redirections are currently not implemented in Tesh.

BUGS, LIMITATIONS AND POSSIBLE IMPROVEMENTS

       The main limitation is the lack of stream redirections in the commands (">", "<" and "|"
       shell constructs and friends). The mkfile builtin command makes this situation bearable.

       It would be nice if we could replace the tesh file completely with command line flags when
       the output is not to be verified.