Provided by:
erlang-manpages_13.b.2.1-1_all 
NAME
common_test - A framework for automated testing of arbitrary target
nodes
DESCRIPTION
The Common Test framework is an environment for implementing and
performing automatic and semi-automatic execution of test cases. Common
Test uses the OTP Test Server as engine for test case execution and
logging.
In brief, Common Test supports:
* Automated execution of test suites (sets of test cases).
* Logging of the events during execution.
* HTML presentation of test suite results.
* HTML presentation of test suite code.
* Support functions for test suite authors.
* Step by step execution of test cases.
The following sections describe the mandatory and optional test suite
functions Common Test will call during test execution. For more details
see Common Test User’s Guide.
TEST CASE CALLBACK FUNCTIONS
The following functions define the callback interface for a test suite.
EXPORTS
Module:all() -> TestCases | {skip,Reason}
Types TestCases = [atom() | {group, GroupName}]
Reason = term()
GroupName = atom()
MANDATORY
This function must return the list of all test cases and test
case groups in the test suite module that are to be executed.
This list also specifies the order the cases and groups will be
executed by Common Test. A test case is represented by an atom,
the name of the test case function. A test case group is
represented by a {group, GroupName} tuple, where GroupName, an
atom, is the name of the group (defined with groups/0).
If {skip, Reason} is returned, all test cases in the module will
be skipped, and the Reason will be printed on the HTML result
page.
For details on groups, see Test case groups in the User’s Guide.
Module:groups() -> GroupDefs
Types GroupDefs = [Group]
Group = {GroupName, Properties, GroupsAndTestCases}
GroupName = atom()
Properties = [parallel | sequence | Shuffle |
{RepeatType, N}]
GroupsAndTestCases = [Group | {group, GroupName} |
TestCase]
TestCase = atom()
Shuffle = shuffle | {shuffle, Seed}
Seed = {integer(), integer(), integer()}
RepeatType = repeat | repeat_until_all_ok |
repeat_until_all_fail | repeat_until_any_ok |
repeat_until_any_fail
N = integer() | forever
OPTIONAL
See Test case groups in the User’s Guide for details.
Module:suite() -> [Info]
Types Info = {timetrap, Time} | {require, Required} |
{require, Name, Required} | {userdata, UserData} |
{silent_connections, Conns} | {stylesheet, CSSFile}
Time = MilliSec | {seconds, integer()} | {minutes,
integer()} | {hours, integer()}
MilliSec = integer()
Required = Key | {Key, SubKeys}
Key = atom()
SubKeys = SubKey | [SubKey]
SubKey = atom()
Name = atom()
UserData = term()
Conns = [atom()]
CSSFile = string()
OPTIONAL
This is the test suite info function. It is supposed to return a
list of tagged tuples that specify various properties regarding
the execution of this test suite (common for all test cases in
the suite).
The timetrap tag sets the maximum time each test case is allowed
to take (including init_per_testcase/2 and end_per_testcase/2).
If the timetrap time is exceeded, the test case fails with
reason timetrap_timeout.
The require tag specifies configuration variables that are
required by test cases in the suite. If the required
configuration variables are not found in any of the
configuration files, all test cases are skipped. For more
information about the ’require’ functionality, see the reference
manual for the function ct:require/[1, 2].
With userdata, it is possible for the user to specify arbitrary
test suite related information which can be read by calling
ct:userdata/2.
Other tuples than the ones defined will simply be ignored.
For more information about the test suite info function, see
Test suite info function in the User’s Guide.
Module:init_per_suite(Config) -> NewConfig | {skip,Reason} |
{skip_and_save,Reason,SaveConfig}
Types Config = NewConfig = SaveConfig = [{Key, Value}]
Key = atom()
Value = term()
Reason = term()
OPTIONAL
This function is called as the first function in the suite. It
typically contains initialization which is common for all test
cases in the suite, and which shall only be done once. The
Config parameter is the configuration which can be modified
here. Whatever is returned from this function is given as Config
to all configuration functions and test cases in the suite. If
{skip, Reason} is returned, all test cases in the suite will be
skipped and Reason printed in the overview log for the suite.
For information on save_config and skip_and_save, please see
Dependencies between Test Cases and Suites in the User’s Guide.
Module:end_per_suite(Config) -> void() | {save_config,SaveConfig}
Types Config = SaveConfig = [{Key, Value}]
Key = atom()
Value = term()
OPTIONAL
This function is called as the last test case in the suite. It
is meant to be used for cleaning up after init_per_suite/1. For
information on save_config, please see Dependencies between Test
Cases and Suites in the User’s Guide.
Module:init_per_group(GroupName, Config) -> NewConfig | {skip,Reason}
Types GroupName = atom()
Config = NewConfig = [{Key, Value}]
Key = atom()
Value = term()
Reason = term()
OPTIONAL
This function is called before execution of a test case group.
It typically contains initialization which is common for all
test cases in the group, and which shall only be performed once.
GroupName is the name of the group, as specified in the group
definition (see groups/0). The Config parameter is the
configuration which can be modified here. Whatever is returned
from this function is given as Config to all test cases in the
group. If {skip, Reason} is returned, all test cases in the
group will be skipped and Reason printed in the overview log for
the group.
For information about test case groups, please see Test case
groups chapter in the User’s Guide.
Module:end_per_group(GroupName, Config) -> void() |
{return_group_result,Status}
Types GroupName = atom()
Config = [{Key, Value}]
Key = atom()
Value = term()
Status = ok | skipped | failed
OPTIONAL
This function is called after the execution of a test case group
is finished. It is meant to be used for cleaning up after
init_per_group/2. By means of {return_group_result, Status}, it
is possible to return a status value for a nested sub-group. The
status can be retrieved in end_per_group/2 for the group on the
level above. The status will also be used by Common Test for
deciding if execution of a group should proceed in case the
property sequence or repeat_until_* is set.
For more information about test case groups, please see Test
case groups chapter in the User’s Guide.
Module:init_per_testcase(TestCase, Config) -> NewConfig | {skip,Reason}
Types TestCase = atom()
Config = NewConfig = [{Key, Value}]
Key = atom()
Value = term()
Reason = term()
OPTIONAL
This function is called before each test case. The TestCase
argument is the name of the test case, and Config is the
configuration which can be modified here. Whatever is returned
from this function is given as Config to the test case. If
{skip, Reason} is returned, the test case will be skipped and
Reason printed in the overview log for the suite.
Module:end_per_testcase(TestCase, Config) -> void() |
{save_config,SaveConfig}
Types TestCase = atom()
Config = SaveConfig = [{Key, Value}]
Key = atom()
Value = term()
OPTIONAL
This function is called after each test case, and can be used to
clean up after init_per_testcase/2 and the test case. Any return
value (besides {save_config, SaveConfig}) is ignored. For
information on save_config, please see Dependencies between Test
Cases and Suites in the User’s Guide
Module:testcase() -> [Info]
Types Info = {timetrap, Time} | {require, Required} |
{require, Name, Required} | {userdata, UserData} |
{silent_connections, Conns}
Time = MilliSec | {seconds, integer()} | {minutes,
integer()} | {hours, integer()}
MilliSec = integer()
Required = Key | {Key, SubKeys}
Key = atom()
SubKeys = SubKey | [SubKey]
SubKey = atom()
Name = atom()
UserData = term()
Conns = [atom()]
OPTIONAL
This is the test case info function. It is supposed to return a
list of tagged tuples that specify various properties regarding
the execution of this particular test case.
The timetrap tag sets the maximum time the test case is allowed
to take. If the timetrap time is exceeded, the test case fails
with reason timetrap_timeout. init_per_testcase/2 and
end_per_testcase/2 are included in the timetrap time.
The require tag specifies configuration variables that are
required by the test case. If the required configuration
variables are not found in any of the configuration files, the
test case is skipped. For more information about the ’require’
functionality, see the reference manual for the function
ct:require/[1, 2].
If timetrap and/or require is not set, the default values
specified in the suite/0 return list will be used.
With userdata, it is possible for the user to specify arbitrary
test case related information which can be read by calling
ct:userdata/3.
Other tuples than the ones defined will simply be ignored.
For more information about the test case info function, see Test
case info function in the User’s Guide.
Module:testcase(Config) -> void() | {skip,Reason} | {comment,Comment} |
{save_config,SaveConfig} | {skip_and_save,Reason,SaveConfig} | exit()
Types Config = SaveConfig = [{Key, Value}]
Key = atom()
Value = term()
Reason = term()
Comment = string()
MANDATORY
This is the implementation of a test case. Here you must call
the functions you want to test, and do whatever you need to
check the result. If someting fails, make sure the function
causes a runtime error, or call ct:fail/[0, 1] (which also
causes the test case process to crash).
Elements from the Config parameter can be read with the ?config
macro. The config macro is defined in ct.hrl
You can return {skip, Reason} if you decide not to run the test
case after all. Reason will then be printed in ’Comment’ field
on the HTML result page.
You can return {comment, Comment} if you wish to print some
information in the ’Comment’ field on the HTML result page.
If the function returns anything else, the test case is
considered successful. (The return value always gets printed in
the test case log file).
For more information about test case implementation, please see
Test cases in the User’s Guide.
For information on save_config and skip_and_save, please see
Dependencies between Test Cases and Suites in the User’s Guide.