Provided by: erlang-manpages_16.b.3-dfsg-1ubuntu2_all bug

NAME

       common_test - A framework for automated testing of arbitrary target nodes

DESCRIPTION

       The  Common Test framework is an environment for implementing and performing automatic and
       semi-automatic execution of test cases. Common Test uses the OTP Test Server as engine for
       test case execution and logging.

       In brief, Common Test supports:

         * Automated execution of test suites (sets of test cases).

         * Logging of the events during execution.

         * HTML presentation of test suite results.

         * HTML presentation of test suite code.

         * Support functions for test suite authors.

         * Step by step execution of test cases.

       The  following  sections  describe  the mandatory and optional test suite functions Common
       Test will call during test execution. For more details see Common Test User's Guide.

TEST CASE CALLBACK FUNCTIONS

       The following functions define the callback interface for a test suite.

EXPORTS

       Module:all() -> Tests | {skip,Reason}

              Types:

                 Tests  =  [TestCase  |  {group,GroupName}   |   {group,GroupName,Properties}   |
                 {group,GroupName,Properties,SubGroups}]
                 TestCase = atom()
                 GroupName = atom()
                 Properties = [parallel | sequence | Shuffle | {RepeatType,N}] | default
                 SubGroups = [{GroupName,Properties} | {GroupName,Properties,SubGroups}]
                 Shuffle = shuffle | {shuffle,Seed}
                 Seed = {integer(),integer(),integer()}
                 RepeatType   =   repeat   |   repeat_until_all_ok   |   repeat_until_all_fail  |
                 repeat_until_any_ok | repeat_until_any_fail
                 N = integer() | forever
                 Reason = term()

              MANDATORY

              This function must return the list of all test cases and test case  groups  in  the
              test  suite  module that are to be executed. This list also specifies the order the
              cases and groups will be executed by Common Test. A test case is represented by  an
              atom,  the  name  of  the test case function. A test case group is represented by a
              group tuple, where GroupName, an atom,  is  the  name  of  the  group  (defined  in
              groups/0).  Execution  properties  for groups may also be specified, both for a top
              level group and for any of its sub-groups.  Group  execution  properties  specified
              here,  will override properties in the group definition (see groups/0). (With value
              default, the group definition properties will be used).

              If {skip,Reason} is returned, all test cases in the module will be skipped, and the
              Reason will be printed on the HTML result page.

              For details on groups, see Test case groups in the User's Guide.

       Module:groups() -> GroupDefs

              Types:

                 GroupDefs = [Group]
                 Group = {GroupName,Properties,GroupsAndTestCases}
                 GroupName = atom()
                 Properties = [parallel | sequence | Shuffle | {RepeatType,N}]
                 GroupsAndTestCases = [Group | {group,GroupName} | TestCase]
                 TestCase = atom()
                 Shuffle = shuffle | {shuffle,Seed}
                 Seed = {integer(),integer(),integer()}
                 RepeatType   =   repeat   |   repeat_until_all_ok   |   repeat_until_all_fail  |
                 repeat_until_any_ok | repeat_until_any_fail
                 N = integer() | forever

              OPTIONAL

              Function for defining test case groups. Please see Test case groups in  the  User's
              Guide for details.

       Module:suite() -> [Info]

              Types:

                  Info   =  {timetrap,Time}  |  {require,Required}  |  {require,Name,Required}  |
                 {userdata,UserData}  |  {silent_connections,Conns}  |   {stylesheet,CSSFile}   |
                 {ct_hooks, CTHs}
                  Time = TimeVal | TimeFunc
                  TimeVal    =   MilliSec   |   {seconds,integer()}   |   {minutes,integer()}   |
                 {hours,integer()}
                  TimeFunc = {Mod,Func,Args} | Fun
                  MilliSec = integer()
                  Mod = atom()
                  Func = atom()
                  Args = list()
                  Fun = fun()
                  Required = Key | {Key,SubKeys} | {Key,SubKey} | {Key,SubKey,SubKeys}
                  Key = atom()
                  SubKeys = SubKey | [SubKey]
                  SubKey = atom()
                  Name = atom()
                  UserData = term()
                  Conns = [atom()]
                  CSSFile = string()
                  CTHs = [CTHModule |
                  {CTHModule, CTHInitArgs} |
                  {CTHModule, CTHInitArgs, CTHPriority}]
                  CTHModule = atom()
                  CTHInitArgs = term()

              OPTIONAL

              This is the test suite info function. It is supposed to return  a  list  of  tagged
              tuples  that specify various properties related to the execution of this test suite
              (common for all test cases in the suite).

              The timetrap tag sets the maximum  time  each  test  case  is  allowed  to  execute
              (including  init_per_testcase/2  and  end_per_testcase/2).  If the timetrap time is
              exceeded, the test case fails with reason timetrap_timeout. A TimeFunc function can
              be  used  to  set  a  new  timetrap  by returning a TimeVal. It may also be used to
              trigger a timetrap timeout by, at some  point,  returning  a  value  other  than  a
              TimeVal. (See the User's Guide for details).

              The  require  tag specifies configuration variables that are required by test cases
              (and/or configuration functions)  in  the  suite.  If  the  required  configuration
              variables  are  not  found  in  any  of the configuration files, all test cases are
              skipped. For more information about the 'require' functionality, see the  reference
              manual for the function ct:require/1/2.

              With  userdata, it is possible for the user to specify arbitrary test suite related
              information which can be read by calling ct:userdata/2.

              The ct_hooks tag specifies which Common Test Hooks are to be run together with this
              suite.

              Other tuples than the ones defined will simply be ignored.

              For  more  information  about  the  test  suite  info function, see Test suite info
              function in the User's Guide.

       Module:init_per_suite(Config)       ->       NewConfig       |       {skip,Reason}       |
       {skip_and_save,Reason,SaveConfig}

              Types:

                  Config = NewConfig = SaveConfig = [{Key,Value}]
                  Key = atom()
                  Value = term()
                  Reason = term()

              OPTIONAL

              This  configuration  function  is  called  as  the  first function in the suite. It
              typically contains initializations which are common  for  all  test  cases  in  the
              suite, and which shall only be done once. The Config parameter is the configuration
              data which can be modified here. Whatever is returned from this function  is  given
              as  Config  to  all  configuration  functions  and  test  cases  in  the  suite. If
              {skip,Reason} is returned, all test cases in the suite will be skipped  and  Reason
              printed in the overview log for the suite.

              For  information  on save_config and skip_and_save, please see Dependencies between
              Test Cases and Suites in the User's Guide.

       Module:end_per_suite(Config) -> void() | {save_config,SaveConfig}

              Types:

                  Config = SaveConfig = [{Key,Value}]
                  Key = atom()
                  Value = term()

              OPTIONAL

              This function is called as the last test case in the suite. It is meant to be  used
              for  cleaning up after init_per_suite/1. For information on save_config, please see
              Dependencies between Test Cases and Suites in the User's Guide.

       Module:group(GroupName) -> [Info]

              Types:

                  Info  =  {timetrap,Time}  |  {require,Required}  |  {require,Name,Required}   |
                 {userdata,UserData}   |   {silent_connections,Conns}  |  {stylesheet,CSSFile}  |
                 {ct_hooks, CTHs}
                  Time = TimeVal | TimeFunc
                  TimeVal   =   MilliSec   |   {seconds,integer()}   |   {minutes,integer()}    |
                 {hours,integer()}
                  TimeFunc = {Mod,Func,Args} | Fun
                  MilliSec = integer()
                  Mod = atom()
                  Func = atom()
                  Args = list()
                  Fun = fun()
                  Required = Key | {Key,SubKeys} | {Key,Subkey} | {Key,Subkey,SubKeys}
                  Key = atom()
                  SubKeys = SubKey | [SubKey]
                  SubKey = atom()
                  Name = atom()
                  UserData = term()
                  Conns = [atom()]
                  CSSFile = string()
                  CTHs = [CTHModule |
                  {CTHModule, CTHInitArgs} |
                  {CTHModule, CTHInitArgs, CTHPriority}]
                  CTHModule = atom()
                  CTHInitArgs = term()

              OPTIONAL

              This  is  the  test  case  group  info function. It is supposed to return a list of
              tagged tuples that specify various properties related to the execution  of  a  test
              case group (i.e. its test cases and sub-groups). Properties set by group/1 override
              properties with the same key that have been previously set by suite/0.

              The timetrap tag sets the maximum  time  each  test  case  is  allowed  to  execute
              (including  init_per_testcase/2  and  end_per_testcase/2).  If the timetrap time is
              exceeded, the test case fails with reason timetrap_timeout. A TimeFunc function can
              be  used  to  set  a  new  timetrap  by returning a TimeVal. It may also be used to
              trigger a timetrap timeout by, at some  point,  returning  a  value  other  than  a
              TimeVal. (See the User's Guide for details).

              The  require  tag specifies configuration variables that are required by test cases
              (and/or configuration functions)  in  the  suite.  If  the  required  configuration
              variables  are  not found in any of the configuration files, all test cases in this
              group are skipped. For more information about the 'require' functionality, see  the
              reference manual for the function ct:require/1/2.

              With  userdata,  it  is  possible for the user to specify arbitrary test case group
              related information which can be read by calling ct:userdata/2.

              The ct_hooks tag specifies which Common Test Hooks are to be run together with this
              suite.

              Other tuples than the ones defined will simply be ignored.

              For  more  information about the test case group info function, see Test case group
              info function in the User's Guide.

       Module:init_per_group(GroupName, Config) -> NewConfig | {skip,Reason}

              Types:

                  GroupName = atom()
                  Config = NewConfig = [{Key,Value}]
                  Key = atom()
                  Value = term()
                  Reason = term()

              OPTIONAL

              This configuration function is called before execution of a  test  case  group.  It
              typically  contains  initializations  which  are common for all test cases and sub-
              groups in the group, and which shall only be performed once. GroupName is the  name
              of  the  group,  as  specified  in  the group definition (see groups/0). The Config
              parameter is the configuration data which can be modified here. The return value of
              this  function is given as Config to all test cases and sub-groups in the group. If
              {skip,Reason} is returned, all test cases in the group will be skipped  and  Reason
              printed in the overview log for the group.

              For  information about test case groups, please see Test case groups chapter in the
              User's Guide.

       Module:end_per_group(GroupName, Config) -> void() | {return_group_result,Status}

              Types:

                  GroupName = atom()
                  Config = [{Key,Value}]
                  Key = atom()
                  Value = term()
                  Status = ok | skipped | failed

              OPTIONAL

              This function is called after the execution of a test case group is finished. It is
              meant   to   be   used   for  cleaning  up  after  init_per_group/2.  By  means  of
              {return_group_result,Status}, it is possible to return a status value for a  nested
              sub-group.  The  status  can  be  retrieved in end_per_group/2 for the group on the
              level above. The status will also be used by Common Test for deciding if  execution
              of a group should proceed in case the property sequence or repeat_until_* is set.

              For more information about test case groups, please see Test case groups chapter in
              the User's Guide.

       Module:init_per_testcase(TestCase, Config) -> NewConfig | {fail,Reason} | {skip,Reason}

              Types:

                  TestCase = atom()
                  Config = NewConfig = [{Key,Value}]
                  Key = atom()
                  Value = term()
                  Reason = term()

              OPTIONAL

              This function is called before each test case. The TestCase argument is the name of
              the test case, and Config (list of key-value tuples) is the configuration data that
              can be modified here. The NewConfig list returned from this function  is  given  as
              Config  to  the test case. If {fail,Reason} is returned, the test case is marked as
              failed without being executed. If {skip,Reason} is returned, the test case will  be
              skipped and Reason printed in the overview log for the suite.

       Module:end_per_testcase(TestCase,     Config)     ->     void()    |    {fail,Reason}    |
       {save_config,SaveConfig}

              Types:

                  TestCase = atom()
                  Config = SaveConfig = [{Key,Value}]
                  Key = atom()
                  Value = term()
                  Reason = term()

              OPTIONAL

              This function is called after each test case, and can be used  to  clean  up  after
              init_per_testcase/2  and the test case. Any return value (besides {fail,Reason} and
              {save_config,SaveConfig}) is ignored. By returning {fail,Reason}, TestCase will  be
              marked  as  failed  (even  though  it  was actually successful in the sense that it
              returned a value instead of terminating). For information  on  save_config,  please
              see Dependencies between Test Cases and Suites in the User's Guide

       Module:Testcase() -> [Info]

              Types:

                  Info   =  {timetrap,Time}  |  {require,Required}  |  {require,Name,Required}  |
                 {userdata,UserData} | {silent_connections,Conns}
                  Time = TimeVal | TimeFunc
                  TimeVal   =   MilliSec   |   {seconds,integer()}   |   {minutes,integer()}    |
                 {hours,integer()}
                  TimeFunc = {Mod,Func,Args} | Fun
                  MilliSec = integer()
                  Mod = atom()
                  Func = atom()
                  Args = list()
                  Fun = fun()
                  Required = Key | {Key,SubKeys} | {Key,Subkey} | {Key,Subkey,SubKeys}
                  Key = atom()
                  SubKeys = SubKey | [SubKey]
                  SubKey = atom()
                  Name = atom()
                  UserData = term()
                  Conns = [atom()]

              OPTIONAL

              This  is  the  test  case  info function. It is supposed to return a list of tagged
              tuples that specify various properties related to the execution of this  particular
              test  case.  Properties  set  by  Testcase/0  override  properties  that  have been
              previously set for the test case by group/1 or suite/0.

              The timetrap tag sets the maximum time the test case is allowed to execute. If  the
              timetrap  time  is  exceeded,  the  test  case  fails with reason timetrap_timeout.
              init_per_testcase/2 and end_per_testcase/2 are included in  the  timetrap  time.  A
              TimeFunc  function can be used to set a new timetrap by returning a TimeVal. It may
              also be used to trigger a timetrap timeout by, at some  point,  returning  a  value
              other than a TimeVal. (See the User's Guide for details).

              The  require  tag  specifies  configuration variables that are required by the test
              case (and/or init/end_per_testcase/2). If the required configuration variables  are
              not  found  in  any  of the configuration files, the test case is skipped. For more
              information about the 'require' functionality, see the  reference  manual  for  the
              function ct:require/1/2.

              If  timetrap and/or require is not set, the default values specified by suite/0 (or
              group/1) will be used.

              With userdata, it is possible for the user to specify arbitrary test  case  related
              information which can be read by calling ct:userdata/3.

              Other tuples than the ones defined will simply be ignored.

              For more information about the test case info function, see Test case info function
              in the User's Guide.

       Module:Testcase(Config)   ->   void()    |    {skip,Reason}    |    {comment,Comment}    |
       {save_config,SaveConfig} | {skip_and_save,Reason,SaveConfig} | exit()

              Types:

                  Config = SaveConfig = [{Key,Value}]
                  Key = atom()
                  Value = term()
                  Reason = term()
                  Comment = string()

              MANDATORY

              This  is  the  implementation  of a test case. Here you must call the functions you
              want to test, and do whatever you need to check the  result.  If  something  fails,
              make  sure  the  function  causes  a runtime error, or call ct:fail/1/2 (which also
              causes the test case process to terminate).

              Elements from the Config list can e.g. be read with proplists:get_value/2  (or  the
              macro ?config defined in ct.hrl).

              You  can  return  {skip,Reason}  if  you decide not to run the test case after all.
              Reason will then be printed in 'Comment' field on the HTML result page.

              You can return {comment,Comment} if you wish  to  print  some  information  in  the
              'Comment' field on the HTML result page.

              If the function returns anything else, the test case is considered successful. (The
              return value always gets printed in the test case log file).

              For more information about test case implementation, please see Test cases  in  the
              User's Guide.

              For  information  on save_config and skip_and_save, please see Dependencies between
              Test Cases and Suites in the User's Guide.