Provided by: falconpl-dev_0.9.6.9-git20120606-2_amd64 bug

NAME

       faltest - Falcon unit test interface.

SYNOPSIS

       faltest [-d unit_test_dir] [options] [unit_list] module_file.fam

DESCRIPTION

       The faltest command line tool is a powerful interface to the Falcon unit testing system.

       The  basic  working  principle  of  faltest  is  that  of taking all the .fal script files
       contained in a directory, compile and execute them, eventually keeping  track  of  errors,
       elapsed  times  and execution failures. After running all the scripts, faltest may print a
       report on what happened if requested to do so.

       A list of one or more unit test may be indicated in the faltest  command  line  after  the
       options. Also, the executed tests can be limited to named subsets.

       The  unit  test  directory  is  added  to the module load path, so load directives will be
       resolved searching the required scripts in the test path.

UNIT TEST SCRIPTS

       Scripts being part of unit test have  to  start  with  a  common  header  indicating  some
       information about them. The header is actually a formatted Falcon comment which is read by
       the faltest utility.

       This is a typical header:

       /**********************************************
       * Falcon test suite
       *
       * ID: 10a
       * Category: cat-name
       * Subcategory: subcat-name
       * Short: The short name
       * Description:
       *   Long description of this test
       *   Spanning on many lines...
       * [/Description]
       **********************************************/

       The header has a free form; faltest  recognizes  the  following  fields,  being  inside  a
       comment and eventually preceded by a "*".

       ID:
              The  only  mandatory field, it declares the ID under which this unit test is known.
              It will be used in listing the tests and in  selecting  them  as  argument  of  the
              faltest  command  line.  The id must be an integer number, eventually followed by a
              single lowercase letter. Similar tests should be filed under the  same  ID  with  a
              different specification letter.

              Scripts not having this field will be ignored by faltest.

       Category:
              The  name  of  the  category  under which this test is filed.  Faltest can select a
              subset of scripts to be executed to a certain category.

       Subcategory:
              The name of the subcategory under which this test is filed.  Faltest can  select  a
              subset of scripts to be executed to a certain subcategory.

       Short:
              Short description (or symbolic name) for this unit test.

       Description:
              A  longer  description explaining what this test is supposed to do. The description
              always spans on more lines, and is closed by a [/Description] tag.

THE TESTSUITE MODULE

       Falcon system provides a loadable  module  called  "testsuite".  The  module  is  actually
       embedded  in faltest , and is provided to all the scripts it runs. The module provides the
       following functions:

       success()
              The script is terminated and recorded as a success.

       failure( reason )
              The script  is  terminated  and  recorded  as  a  failure.  An  optional  parameter
              containing  a  description  of the failure condition may be optionally provided; it
              will be written as part of the report and may be used to track which  part  of  the
              test wasn't working.

       testReflect( item )
              Returns the passed item. This is used to test for engine responsiveness to external
              calls and item copy through external functions.

       alive( percent )
              In tests running for some human sensible  time,  this  function  should  be  called
              periodically to inform the system and the user that the test is proceeding.

              An  optional "percent" parameter can be provided. It will be interpreted as a value
              between 1 and 100 representing the amount of test that has  been  performed  up  to
              this moment.

       timings( total_time, performed_ops )
              In  case  the  execution  time  is  relevant for the test, like in benchmarks, this
              function can be used to communicate  back  to  faltest  the  time  elapsed  in  the
              operations  being  tested  and the number of operations performed. Those parameters
              will be recorded and eventually saved in the report file, to be used as  benchmarks
              against falcon engine modifications.

       timeFactor()
              Lengthy  tests are often performed by looping over the operation to be tested for a
              certain time. Benchmarks and performance tests should be written so that  they  can
              normally  complete  in  a reasonable time, between one and ten seconds. In case the
              user wants the tests to perform longer, in order to obtain better statistical data,
              it  can  pass  the -f (time factor) option to faltest command line. The time factor
              will be a number greater than 1, and should be  used  by  tests  that  may  perform
              lengthy  operation  to  customize  the  number  of  performed tests.  This function
              returns as an integer value of the -f option given to faltest or 1 if the -f option
              was not given.

OPTIONS

       -c <cat>
              Select this category and ignore the rest.

       -C <subcat>
              Select this subcategory and ignore the rest.

       -d     Directory  from  where  to  load  the  unit tests. If not specified, it will be the
              current directory.

       -f <n> Set time factor to N. Some scripts may use this information to perform  more  loops
              or lengthy operations.

       -h     Show version and a short help.

       -l     List the selected tests and exit. Combine with -v to have the path of the tests, as
              -l only lists the script ID, its short name and the category/subcategory pair.

       -m     Do not compile in memory. Use disk temporary files.

       -M     Checks for memory leaks. This option makes faltest to report unclaimed memory after
              each  script  execution,  and  a final report at the end of the tests. The check is
              extended to all the engine operations, so errors in the engine are detected too.

       -o <file>
              Write final report to the given output file.

       -s     Perform module serialization test. Other than compiling the  file,  the  module  is
              also  saved  and then restored before being executed.  This allows one to check for
              errors in module serialization (that is, loading of .fam files). The  operation  is
              performed in memory, unless the option -m is also specified.

       -S     Compile via assembly. This test the correct behavior of the assembler generator and
              compiler instead of the binary module generator.

       -t     Records and display timings. The statistics of compilation, linking  and  execution
              overall times are recorded and written as part of the report.

       -T     Records  timings() calls from scripts. This allows the scripts to declare their own
              performance ratings, and collects the results in the final report.

       -v     Be verbose. Normally, execution and failures are sparsely reported. This is because
              the  normal  execution  mode  is meant for automated runs. Tests can be executed by
              automated utilities and errors can be reported to system  administrator  by  simple
              checks on the output data.

              A  developer  willing  to  fix  a  broken test must run that test alone with the -v
              enabled. A more complete error report (including compilation or  execution  errors,
              if  they  were  the  cause for the failure) will be then visualized. The -v options
              also allows one to see the path of the original script, which is  otherwise  hidden
              (masked by the testsuite ID).

       -V     Prints version number and exits.

SAMPLE

       This is a simple and complete example from the Falcon benchmark suite.

       /*******************************************
       * Falcon direct benchmarks
       *
       * ID: 2a
       * Category: benchmark
       * Subcategory: calls
       * Short: Benchmark on function calls
       * Description:
       *    Performing repeated function calls and returns.
       *    This test calls a function without parameters.
       *
       * [/Description]
       ******************************************/

       loops = 1000000 * timeFactor()
       each = int(loops/10)

       function toBeCalled()
       end

       // getting time
       time = seconds()
       for i = 1 to loops

          // perform the call
          toBeCalled()

          if i % each == 0
             alive( i/loops*100 )
          end
       end
       // taking end time
       time = seconds() - time

       // subtract alive time
       timeAlive = seconds()
       for i = 1 to loops
          if i % each == 0
             alive( i/loops*100 )
          end
       end
       timeAlive = seconds() - timeAlive
       time -= timeAlive

       timings( time, loops )

       /* end of file */

FILES

       /usr/lib/libfalcon_engine.so
              Default location of the Falcon Engine loadable module.

AUTHOR

       Giancarlo Niccolai <gc@falconpl.org>

SEE ALSO

       falcon(1)

LICENSE

       This  document  is  released  under the "GNU Free Documentation License, version 1.2".  On
       Debian systems, the complete text of the Free Documentation License, version 1.2,  can  be
       found in /usr/share/common-licenses/.