Provided by: erlang-manpages_16.b.3-dfsg-1ubuntu2.2_all 
NAME
common_test - A framework for automated testing of arbitrary target nodes
DESCRIPTION
The Common Test framework is an environment for implementing and performing automatic and
semi-automatic execution of test cases. Common Test uses the OTP Test Server as engine for
test case execution and logging.
In brief, Common Test supports:
* Automated execution of test suites (sets of test cases).
* Logging of the events during execution.
* HTML presentation of test suite results.
* HTML presentation of test suite code.
* Support functions for test suite authors.
* Step by step execution of test cases.
The following sections describe the mandatory and optional test suite functions Common
Test will call during test execution. For more details see Common Test User's Guide.
TEST CASE CALLBACK FUNCTIONS
The following functions define the callback interface for a test suite.
EXPORTS
Module:all() -> Tests | {skip,Reason}
Types:
Tests = [TestCase | {group,GroupName} | {group,GroupName,Properties} |
{group,GroupName,Properties,SubGroups}]
TestCase = atom()
GroupName = atom()
Properties = [parallel | sequence | Shuffle | {RepeatType,N}] | default
SubGroups = [{GroupName,Properties} | {GroupName,Properties,SubGroups}]
Shuffle = shuffle | {shuffle,Seed}
Seed = {integer(),integer(),integer()}
RepeatType = repeat | repeat_until_all_ok | repeat_until_all_fail |
repeat_until_any_ok | repeat_until_any_fail
N = integer() | forever
Reason = term()
MANDATORY
This function must return the list of all test cases and test case groups in the
test suite module that are to be executed. This list also specifies the order the
cases and groups will be executed by Common Test. A test case is represented by an
atom, the name of the test case function. A test case group is represented by a
group tuple, where GroupName, an atom, is the name of the group (defined in
groups/0). Execution properties for groups may also be specified, both for a top
level group and for any of its sub-groups. Group execution properties specified
here, will override properties in the group definition (see groups/0). (With value
default, the group definition properties will be used).
If {skip,Reason} is returned, all test cases in the module will be skipped, and the
Reason will be printed on the HTML result page.
For details on groups, see Test case groups in the User's Guide.
Module:groups() -> GroupDefs
Types:
GroupDefs = [Group]
Group = {GroupName,Properties,GroupsAndTestCases}
GroupName = atom()
Properties = [parallel | sequence | Shuffle | {RepeatType,N}]
GroupsAndTestCases = [Group | {group,GroupName} | TestCase]
TestCase = atom()
Shuffle = shuffle | {shuffle,Seed}
Seed = {integer(),integer(),integer()}
RepeatType = repeat | repeat_until_all_ok | repeat_until_all_fail |
repeat_until_any_ok | repeat_until_any_fail
N = integer() | forever
OPTIONAL
Function for defining test case groups. Please see Test case groups in the User's
Guide for details.
Module:suite() -> [Info]
Types:
Info = {timetrap,Time} | {require,Required} | {require,Name,Required} |
{userdata,UserData} | {silent_connections,Conns} | {stylesheet,CSSFile} |
{ct_hooks, CTHs}
Time = TimeVal | TimeFunc
TimeVal = MilliSec | {seconds,integer()} | {minutes,integer()} |
{hours,integer()}
TimeFunc = {Mod,Func,Args} | Fun
MilliSec = integer()
Mod = atom()
Func = atom()
Args = list()
Fun = fun()
Required = Key | {Key,SubKeys} | {Key,SubKey} | {Key,SubKey,SubKeys}
Key = atom()
SubKeys = SubKey | [SubKey]
SubKey = atom()
Name = atom()
UserData = term()
Conns = [atom()]
CSSFile = string()
CTHs = [CTHModule |
{CTHModule, CTHInitArgs} |
{CTHModule, CTHInitArgs, CTHPriority}]
CTHModule = atom()
CTHInitArgs = term()
OPTIONAL
This is the test suite info function. It is supposed to return a list of tagged
tuples that specify various properties related to the execution of this test suite
(common for all test cases in the suite).
The timetrap tag sets the maximum time each test case is allowed to execute
(including init_per_testcase/2 and end_per_testcase/2). If the timetrap time is
exceeded, the test case fails with reason timetrap_timeout. A TimeFunc function can
be used to set a new timetrap by returning a TimeVal. It may also be used to
trigger a timetrap timeout by, at some point, returning a value other than a
TimeVal. (See the User's Guide for details).
The require tag specifies configuration variables that are required by test cases
(and/or configuration functions) in the suite. If the required configuration
variables are not found in any of the configuration files, all test cases are
skipped. For more information about the 'require' functionality, see the reference
manual for the function ct:require/1/2.
With userdata, it is possible for the user to specify arbitrary test suite related
information which can be read by calling ct:userdata/2.
The ct_hooks tag specifies which Common Test Hooks are to be run together with this
suite.
Other tuples than the ones defined will simply be ignored.
For more information about the test suite info function, see Test suite info
function in the User's Guide.
Module:init_per_suite(Config) -> NewConfig | {skip,Reason} |
{skip_and_save,Reason,SaveConfig}
Types:
Config = NewConfig = SaveConfig = [{Key,Value}]
Key = atom()
Value = term()
Reason = term()
OPTIONAL
This configuration function is called as the first function in the suite. It
typically contains initializations which are common for all test cases in the
suite, and which shall only be done once. The Config parameter is the configuration
data which can be modified here. Whatever is returned from this function is given
as Config to all configuration functions and test cases in the suite. If
{skip,Reason} is returned, all test cases in the suite will be skipped and Reason
printed in the overview log for the suite.
For information on save_config and skip_and_save, please see Dependencies between
Test Cases and Suites in the User's Guide.
Module:end_per_suite(Config) -> void() | {save_config,SaveConfig}
Types:
Config = SaveConfig = [{Key,Value}]
Key = atom()
Value = term()
OPTIONAL
This function is called as the last test case in the suite. It is meant to be used
for cleaning up after init_per_suite/1. For information on save_config, please see
Dependencies between Test Cases and Suites in the User's Guide.
Module:group(GroupName) -> [Info]
Types:
Info = {timetrap,Time} | {require,Required} | {require,Name,Required} |
{userdata,UserData} | {silent_connections,Conns} | {stylesheet,CSSFile} |
{ct_hooks, CTHs}
Time = TimeVal | TimeFunc
TimeVal = MilliSec | {seconds,integer()} | {minutes,integer()} |
{hours,integer()}
TimeFunc = {Mod,Func,Args} | Fun
MilliSec = integer()
Mod = atom()
Func = atom()
Args = list()
Fun = fun()
Required = Key | {Key,SubKeys} | {Key,Subkey} | {Key,Subkey,SubKeys}
Key = atom()
SubKeys = SubKey | [SubKey]
SubKey = atom()
Name = atom()
UserData = term()
Conns = [atom()]
CSSFile = string()
CTHs = [CTHModule |
{CTHModule, CTHInitArgs} |
{CTHModule, CTHInitArgs, CTHPriority}]
CTHModule = atom()
CTHInitArgs = term()
OPTIONAL
This is the test case group info function. It is supposed to return a list of
tagged tuples that specify various properties related to the execution of a test
case group (i.e. its test cases and sub-groups). Properties set by group/1 override
properties with the same key that have been previously set by suite/0.
The timetrap tag sets the maximum time each test case is allowed to execute
(including init_per_testcase/2 and end_per_testcase/2). If the timetrap time is
exceeded, the test case fails with reason timetrap_timeout. A TimeFunc function can
be used to set a new timetrap by returning a TimeVal. It may also be used to
trigger a timetrap timeout by, at some point, returning a value other than a
TimeVal. (See the User's Guide for details).
The require tag specifies configuration variables that are required by test cases
(and/or configuration functions) in the suite. If the required configuration
variables are not found in any of the configuration files, all test cases in this
group are skipped. For more information about the 'require' functionality, see the
reference manual for the function ct:require/1/2.
With userdata, it is possible for the user to specify arbitrary test case group
related information which can be read by calling ct:userdata/2.
The ct_hooks tag specifies which Common Test Hooks are to be run together with this
suite.
Other tuples than the ones defined will simply be ignored.
For more information about the test case group info function, see Test case group
info function in the User's Guide.
Module:init_per_group(GroupName, Config) -> NewConfig | {skip,Reason}
Types:
GroupName = atom()
Config = NewConfig = [{Key,Value}]
Key = atom()
Value = term()
Reason = term()
OPTIONAL
This configuration function is called before execution of a test case group. It
typically contains initializations which are common for all test cases and sub-
groups in the group, and which shall only be performed once. GroupName is the name
of the group, as specified in the group definition (see groups/0). The Config
parameter is the configuration data which can be modified here. The return value of
this function is given as Config to all test cases and sub-groups in the group. If
{skip,Reason} is returned, all test cases in the group will be skipped and Reason
printed in the overview log for the group.
For information about test case groups, please see Test case groups chapter in the
User's Guide.
Module:end_per_group(GroupName, Config) -> void() | {return_group_result,Status}
Types:
GroupName = atom()
Config = [{Key,Value}]
Key = atom()
Value = term()
Status = ok | skipped | failed
OPTIONAL
This function is called after the execution of a test case group is finished. It is
meant to be used for cleaning up after init_per_group/2. By means of
{return_group_result,Status}, it is possible to return a status value for a nested
sub-group. The status can be retrieved in end_per_group/2 for the group on the
level above. The status will also be used by Common Test for deciding if execution
of a group should proceed in case the property sequence or repeat_until_* is set.
For more information about test case groups, please see Test case groups chapter in
the User's Guide.
Module:init_per_testcase(TestCase, Config) -> NewConfig | {fail,Reason} | {skip,Reason}
Types:
TestCase = atom()
Config = NewConfig = [{Key,Value}]
Key = atom()
Value = term()
Reason = term()
OPTIONAL
This function is called before each test case. The TestCase argument is the name of
the test case, and Config (list of key-value tuples) is the configuration data that
can be modified here. The NewConfig list returned from this function is given as
Config to the test case. If {fail,Reason} is returned, the test case is marked as
failed without being executed. If {skip,Reason} is returned, the test case will be
skipped and Reason printed in the overview log for the suite.
Module:end_per_testcase(TestCase, Config) -> void() | {fail,Reason} |
{save_config,SaveConfig}
Types:
TestCase = atom()
Config = SaveConfig = [{Key,Value}]
Key = atom()
Value = term()
Reason = term()
OPTIONAL
This function is called after each test case, and can be used to clean up after
init_per_testcase/2 and the test case. Any return value (besides {fail,Reason} and
{save_config,SaveConfig}) is ignored. By returning {fail,Reason}, TestCase will be
marked as failed (even though it was actually successful in the sense that it
returned a value instead of terminating). For information on save_config, please
see Dependencies between Test Cases and Suites in the User's Guide
Module:Testcase() -> [Info]
Types:
Info = {timetrap,Time} | {require,Required} | {require,Name,Required} |
{userdata,UserData} | {silent_connections,Conns}
Time = TimeVal | TimeFunc
TimeVal = MilliSec | {seconds,integer()} | {minutes,integer()} |
{hours,integer()}
TimeFunc = {Mod,Func,Args} | Fun
MilliSec = integer()
Mod = atom()
Func = atom()
Args = list()
Fun = fun()
Required = Key | {Key,SubKeys} | {Key,Subkey} | {Key,Subkey,SubKeys}
Key = atom()
SubKeys = SubKey | [SubKey]
SubKey = atom()
Name = atom()
UserData = term()
Conns = [atom()]
OPTIONAL
This is the test case info function. It is supposed to return a list of tagged
tuples that specify various properties related to the execution of this particular
test case. Properties set by Testcase/0 override properties that have been
previously set for the test case by group/1 or suite/0.
The timetrap tag sets the maximum time the test case is allowed to execute. If the
timetrap time is exceeded, the test case fails with reason timetrap_timeout.
init_per_testcase/2 and end_per_testcase/2 are included in the timetrap time. A
TimeFunc function can be used to set a new timetrap by returning a TimeVal. It may
also be used to trigger a timetrap timeout by, at some point, returning a value
other than a TimeVal. (See the User's Guide for details).
The require tag specifies configuration variables that are required by the test
case (and/or init/end_per_testcase/2). If the required configuration variables are
not found in any of the configuration files, the test case is skipped. For more
information about the 'require' functionality, see the reference manual for the
function ct:require/1/2.
If timetrap and/or require is not set, the default values specified by suite/0 (or
group/1) will be used.
With userdata, it is possible for the user to specify arbitrary test case related
information which can be read by calling ct:userdata/3.
Other tuples than the ones defined will simply be ignored.
For more information about the test case info function, see Test case info function
in the User's Guide.
Module:Testcase(Config) -> void() | {skip,Reason} | {comment,Comment} |
{save_config,SaveConfig} | {skip_and_save,Reason,SaveConfig} | exit()
Types:
Config = SaveConfig = [{Key,Value}]
Key = atom()
Value = term()
Reason = term()
Comment = string()
MANDATORY
This is the implementation of a test case. Here you must call the functions you
want to test, and do whatever you need to check the result. If something fails,
make sure the function causes a runtime error, or call ct:fail/1/2 (which also
causes the test case process to terminate).
Elements from the Config list can e.g. be read with proplists:get_value/2 (or the
macro ?config defined in ct.hrl).
You can return {skip,Reason} if you decide not to run the test case after all.
Reason will then be printed in 'Comment' field on the HTML result page.
You can return {comment,Comment} if you wish to print some information in the
'Comment' field on the HTML result page.
If the function returns anything else, the test case is considered successful. (The
return value always gets printed in the test case log file).
For more information about test case implementation, please see Test cases in the
User's Guide.
For information on save_config and skip_and_save, please see Dependencies between
Test Cases and Suites in the User's Guide.