Ubuntu Manpage: Devel::Cover - Code coverage metrics for Perl

Provided by: libdevel-cover-perl_1.21-1ubuntu1_amd64

NAME

       Devel::Cover - Code coverage metrics for Perl

VERSION

       version 1.21

SYNOPSIS

       To get coverage for an uninstalled module:

        cover -test

       or

        cover -delete
        HARNESS_PERL_SWITCHES=-MDevel::Cover make test
        cover

       To get coverage for an uninstalled module which uses Module::Build (0.26 or later):

        ./Build testcover

       If the module does not use the t/*.t framework:

        PERL5OPT=-MDevel::Cover make test

       If you want to get coverage for a program:

        perl -MDevel::Cover yourprog args
        cover

       To alter default values:

        perl -MDevel::Cover=-db,cover_db,-coverage,statement,time yourprog args

DESCRIPTION

This module provides code coverage metrics for Perl. Code coverage metrics describe how
thoroughly tests exercise code. By using Devel::Cover you can discover areas of code not
exercised by your tests and determine which tests to create to increase coverage. Code
coverage can be considered an indirect measure of quality.

Although it is still being developed, Devel::Cover is now quite stable and provides many
of the features to be expected in a useful coverage tool.

Statement, branch, condition, subroutine, and pod coverage information is reported.
Statement and subroutine coverage data should be accurate. Branch and condition coverage
data should be mostly accurate too, although not always what one might initially expect.
Pod coverage comes from Pod::Coverage. If Pod::Coverage::CountParents is available it
will be used instead. Coverage data for other criteria are not yet collected.

The cover program can be used to generate coverage reports. Devel::Cover ships with a
number of different reports including various types of HTML output, textual reports, a
report to display missing coverage in the same format as compilation errors and a report
to display coverage information within the Vim editor.

It is possible to add annotations to reports, for example you can add a column to an HTML
report showing who last changed a line, as determined by git blame. Some annotation
modules are shipped with Devel::Cover and you can easily create your own.

The gcov2perl program can be used to convert gcov files to "Devel::Cover" databases. This
allows you to display your C or XS code coverage together with your Perl coverage, or to
use any of the Devel::Cover reports to display your C coverage data.

Code coverage data are collected by replacing perl ops with functions which count how many
times the ops are executed. These data are then mapped back to reality using the B
compiler modules. There is also a statement profiling facility which should not be relied
on. For proper profiling use Devel::NYTProf. Previous versions of Devel::Cover collected
coverage data by replacing perl's runops function. It is still possible to switch to that
mode of operation, but this now gets little testing and will probably be removed soon.
You probably don't care about any of this.

The most appropriate mailing list on which to discuss this module would be perl-qa. See
<http://lists.perl.org/list/perl-qa.html>.

The Devel::Cover repository can be found at <http://github.com/pjcj/Devel--Cover>. This
is also where problems should be reported.

REQUIREMENTS AND RECOMMENDED MODULES

REQUIREMENTS
• Perl 5.6.1 or greater. Perl 5.8.8 or greater is recommended.

Perl 5.7 is unsupported. Perl 5.8.8 or greater is recommended. Perl 5.8.7 has
problems and may crash. Whilst Perl 5.6 should mostly work you will probably miss out
on coverage information which would be available using a more modern version and will
likely run into bugs in perl. Devel::Cover support for unsupported Perl versions may
be removed at any time, but I try to keep older versions running provided this does
not cause undue difficulty i other areas.

Different versions of perl may give slightly different results due to changes in the
op tree.

• The ability to compile XS extensions.

This means a working C compiler and make program at least. If you built perl from
source you will have these already and they will be used automatically. If your perl
was built in some other way, for example you may have installed it using your
Operating System's packaging mechanism, you will need to ensure that the appropriate
tools are installed.

• Storable and Digest::MD5

Both are in the core in Perl 5.8.0 and above.

OPTIONAL MODULES
• Template, and either PPI::HTML or Perl::Tidy

Needed if you want syntax highlighted HTML reports.

• Pod::Coverage (0.06 or above) or Pod::Coverage::CountParents

One is needed if you want Pod coverage. If Pod::Coverage::CountParents is installed,
it is preferred.

• Test::More

Required if you want to run Devel::Cover's own tests.

• Test::Differences

Needed if the tests fail and you would like nice output telling you why.

• Template and Parallel::Iterator

Needed if you want to run cpancover.

• JSON, JSON::PP or JSON::XS

JSON is used to store the coverage database if it is available.

Use with mod_perl
By adding "use Devel::Cover;" to your mod_perl startup script, you should be able to
collect coverage information when running under mod_perl. You can also add any options
you need at this point. I would suggest adding this as early as possible in your startup
script in order to collect as much coverage information as possible.

Alternatively, add -MDevel::Cover to the parameters for mod_perl. In this example,
Devel::Cover will be operating in silent mode.

PerlSwitches -MDevel::Cover=-silent,1

OPTIONS

-blib - "use blib" and ignore files matching \bt/ (default true
if blib directory exists, false otherwise).
-coverage criterion - Turn on coverage for the specified criterion. Criteria
include statement, branch, condition, path, subroutine,
pod, time, all and none (default all available).
-db cover_db - Store results in coverage db (default ./cover_db).
-dir path - Directory in which coverage will be collected (default
cwd).
-ignore RE - Set REs of files to ignore (default "/Devel/Cover\b").
+ignore RE - Append to REs of files to ignore.
-inc path - Set prefixes of files to include (default @INC).
+inc path - Append to prefixes of files to include.
-merge val - Merge databases, for multiple test benches (default on).
-select RE - Set REs of files to select (default none).
+select RE - Append to REs of files to select.
-silent val - Don't print informational messages (default off).
-subs_only val - Only cover code in subroutine bodies (default off).
-replace_ops val - Use op replacing rather than runops (default on).
-summary val - Print summary information if val is true (default on).

More on Coverage Options
You can specify options to some coverage criteria. At the moment only pod coverage takes
any options. These are the parameters which are passed into the Pod::Coverage
constructor. The extra options are separated by dashes, and you may specify as many as
you wish. For example, to specify that all subroutines containing xx are private, call
Devel::Cover with the option -coverage,pod-also_private-xx.

SELECTING FILES TO COVER

       You may select the files for which you want to collect coverage data using the select,
       ignore and inc options.  The system uses the following procedure to decide whether a file
       will be included in coverage reports:

       •   If the file matches a RE given as a select option, it will be included.

       •   Otherwise, if it matches a RE given as an ignore option, it won't be included.

       •   Otherwise, if it is in one of the inc directories, it won't be included.

       •   Otherwise, it will be included.

       You may add to the REs to select by using +select, or you may reset the selections using
       -select.  The same principle applies to the REs to ignore.

       The inc directories are initially populated with the contents of perl's @INC array.  You
       may reset these directories using -inc, or add to them using +inc.

       Although these options take regular expressions, you should not enclose the RE within //
       or any other quoting characters.

       The options -coverage, [+-]select, [+-]ignore and [+-]inc can be specified multiple times,
       but they can also take multiple comma separated arguments.  In any case you should not add
       a space after the comma, unless you want the argument to start with that literal space.

UNCOVERABLE CRITERIA

Sometimes you have code which is uncoverable for some reason. Perhaps it is an else
clause that cannot be reached, or a check for an error condition that should never happen.
You can tell Devel::Cover that certain criteria are uncoverable and then they are not
counted as errors when they are not exercised. In fact, they are counted as errors if
they are exercised.

This feature should only be used as something of a last resort. Ideally you would find
some way of exercising all your code. But if you have analysed your code and determined
that you are not going to be able to exercise it, it may be better to record that fact in
some formal fashion and stop Devel::Cover complaining about it, so that real problems are
not lost in the noise.

If you have uncoverable criteria I suggest not using the default HTML report (with uses
html_minimal at the moment) because this sometimes shows uncoverable points as uncovered.
Instead, you should use the html_basic report for HTML output which should behave
correctly in this regard.

There are two ways to specify a construct as uncoverable, one invasive and one non-
invasive.

Invasive specification
You can use special comments in your code to specify uncoverable criteria. Comments are
of the form:

# uncoverable <criterion> [details]

The keyword "uncoverable" must be the first text in the comment. It should be followed by
the name of the coverage criterion which is uncoverable. There may then be further
information depending on the nature of the uncoverable construct.

Statements

The "uncoverable" comment should appear on either the same line as the statement, or on
the line before it:

$impossible++; # uncoverable statement
# uncoverable statement
it_has_all_gone_horribly_wrong();

If there are multiple statements (or any other criterion) on a line you can specify which
statement is uncoverable by using the "count" attribute, count:n, which indicates that the
uncoverable statement is the nth statement on the line.

# uncoverable statement count:1
# uncoverable statement count:2
cannot_run_this(); or_this();

Branches

The "uncoverable" comment should specify whether the "true" or "false" branch is
uncoverable.

# uncoverable branch true
if (pi == 3)

Both branches may be uncoverable:

# uncoverable branch true
# uncoverable branch false
if (impossible_thing_happened_one_way()) {
handle_it_one_way(); # uncoverable statement
} else {
handle_it_another_way(); # uncoverable statement
}

Conditions

Because of the way in which Perl short-circuits boolean operations, there are three ways
in which such conditionals can be uncoverable. In the case of " $x && $y" for example,
the left operator may never be true, the right operator may never be true, and the whole
operation may never be false. These conditions may be modelled thus:

# uncoverable branch true
# uncoverable condition left
# uncoverable condition false
if ($x && !$y) {
$x++; # uncoverable statement
}

# uncoverable branch true
# uncoverable condition right
# uncoverable condition false
if (!$x && $y) {
}

"Or" conditionals are handled in a similar fashion (TODO - provide some examples) but
"xor" conditionals are not properly handled yet.

Subroutines

A subroutine should be marked as uncoverable at the point where the first statement is
marked as uncoverable. Ideally all other criteria in the subroutine would be marked as
uncoverable automatically, but that isn't the case at the moment.

sub z {
# uncoverable subroutine
$y++; # uncoverable statement
}

Non-invasive specification
If you can't, or don't want to add coverage comments to your code, you can specify the
uncoverable information in a separate file. By default this file is .uncoverable but you
can override that.

The interface to managing this file is the cover program, and the options are:

-uncoverable_file
-add_uncoverable_point
-delete_uncoverable_point
-clean_uncoverable_points

Of these, only the first two are implemented at the moment. The parameter for
-add_uncoverable_point is a string composed of up to seven space separated elements:
"$file $criterion $line $count $type $class $note".

TODO - more information and examples.

ENVIRONMENT

   User variables
       The -silent option is turned on when Devel::Cover is invoked via $HARNESS_PERL_SWITCHES or
       $PERL5OPT.  Devel::Cover tries to do the right thing when $MOD_PERL is set.
       $DEVEL_COVER_OPTIONS is appended to any options passed into Devel::Cover.

   Developer variables
       When running Devel::Cover's own test suite, $DEVEL_COVER_DEBUG turns on debugging
       information, $DEVEL_COVER_GOLDEN_VERSION overrides Devel::Cover's own idea of which golden
       results it should test against, and $DEVEL_COVER_NO_COVERAGE runs the tests without
       collecting coverage.  $DEVEL_COVER_DB_FORMAT may be set to "Sereal", "JSON" or "Storable"
       to override the default choice of DB format (Sereal, then JSON if either are available,
       otherwise Storable).  $DEVEL_COVER_IO_OPTIONS provides fine-grained control over the DB
       format.  For example, setting it to "pretty" when the format is JSON will store the DB in
       a readable JSON format.  $DEVEL_COVER_CPUS overrides the automated detection of the number
       of CPUs to use in parallel testing.

ACKNOWLEDGEMENTS

       Some code and ideas cribbed from:

       •   Devel::OpProf

       •   B::Concise

       •   B::Deparse

LIMITATIONS

       There are things that Devel::Cover can't cover.

   Absence of shared dependencies
       Perl keeps track of which modules have been loaded (to avoid reloading them).  Because of
       this, it isn't possible to get coverage for a path where a runtime import fails if the
       module being imported is one that Devel::Cover uses internally.  For example, suppose your
       program has this function:

        sub foo {
            eval { require Storable };
            if ($@) {
                carp "Can't find Storable";
                return;
            }
            # ...
        }

       You might write a test for the failure mode as

        BEGIN { @INC = () }
        foo();
        # check for error message

       Because Devel::Cover uses Storable internally, the import will succeed (and the test will
       fail) under a coverage run.

       Modules used by Devel::Cover while gathering coverage:

       •   B

       •   B::Debug

       •   B::Deparse

       •   Carp

       •   Cwd

       •   Digest::MD5

       •   File::Path

       •   File::Spec

       •   Storable or JSON

   Redefined subroutines
       If you redefine a subroutine you may find that the original subroutine is not reported on.
       This is because I haven't yet found a way to locate the original CV.  Hints, tips or
       patches to resolve this will be gladly accepted.

       The module Test::TestCoverage uses this technique and so should not be used in conjunction
       with Devel::Cover.

BUGS

       Almost certainly.

       See the BUGS file, the TODO file and the bug trackers at
       <https://github.com/pjcj/Devel--Cover/issues?sort=created&direction=desc&state=open> and
       <https://rt.cpan.org/Public/Dist/Display.html?Name=Devel-Cover>

       Please report new bugs on Github.

LICENCE

       Copyright 2001-2015, Paul Johnson (paul@pjcj.net)

       This software is free.  It is licensed under the same terms as Perl itself.

       The latest version of this software should be available on CPAN and from my homepage:
       http://www.pjcj.net/.