Provided by: libregexp-debugger-perl_0.002006-1_all 
NAME
Regexp::Debugger - Visually debug regexes in-place
VERSION
This document describes Regexp::Debugger version 0.002006
SYNOPSIS
use Regexp::Debugger;
DESCRIPTION
When you load this module, any regex in the same lexical scope will be visually (and
interactively) debugged as it matches.
INTERFACE
The module itself provides no API. You load it and the debugger is automatically
activated in that lexical scope.
The debugger offers the following commands:
"?" : Print a help message listing these commands
"s" : Step forward (stepping into any named subpattern calls)
"n" : Step forward (stepping over any named subpattern calls)
"-" : Step backward (stepping into any named subpattern calls)
"p" : Step backward (stepping over any named subpattern calls)
"r" : Continue forward until the end of the current (sub)pattern
"m" : Continue forward to the next regex component that matches
"M" : Continue forward to the next regex component that matches
in the current (sub)pattern (i.e. silently stepping over
any named subpattern calls)
"f" : Continue forward to the next regex component that fails
to match something
"F" : Continue forward to the next regex component that fails
to match something in the current (sub)pattern
(i.e. silently stepping over any named subpattern calls)
"c" : Continue forward until the entire regex matches
or completely backtracks
"C" : Continue forward until the entire regex matches
or completely backtracks, silently stepping over
any named subroutine calls
"R" : Rewind to the start of the entire match
"<RETURN>/<ENTER>"
: Repeat the previous command
"v" : Switch to regex/string visualization mode
"h" : Switch to heatmapped visualization mode
"e" : Switch to the event log
"j" : Switch to the underlying JSON data
"d" : Describe each component of the regex in detail
"V"
"H"
"E"
"J"
"D" : Take a snapshot of the corresponding display mode.
When prompted for a filename:
"<RET>"
...prints the snapshot to the terminal
"<TAB>"
...prints the snapshot to a file named "./rxrx_DISPLAY_MODE_TIMESTAMP"
Anything else
...prints the snapshot to that file
"q" : Quit the debugger and finish matching this regex
without any further visualization. The program
continues to execute and other regexes may
still be debugged.
"x" : Exit the debugger and the entire program immediately.
CONFIGURATION
You can configure the debugger by setting up a .rxrx file in in the current directory or
in your home directory. This configuration consists of key:value pairs (everything else in
the file is silently ignored).
Display mode configuration
If the "'display'" key is specified, the debugger starts in that mode. The four available
modes are:
# Show dynamic visualization of matching (the default)...
display : visual
# Show dynamic heatmap visualization of matching...
display : heatmap
# Show multi-line matching event log...
display : events
# Show JSON encoding of matching process...
display : JSON
Whitespace display configuration
Normally, the debugger compacts whitespaces in the regex down to a single space character,
but you can configure that with the "show_ws" key:
# Compact whitespace and comments to a single space (the default)...
show_ws : compact
# Compact whitespace, but show comments, newlines (\n), and tabs (\t)...
show_ws : visible
# Don't compact whitespace, and show newlines and tabs as \n and \t...
show_ws : original
Colour configuration
The following keys reconfigure the colours with which the debugger displays various
information:
Colours for debugging information
• "try_col"
The colour in which attempts to match part of the regex are reported
• "match_col"
The colour in which successful matches of part of the regex are reported
• "fail_col"
The colour in which unsuccessful matches of part of the regex are reported
• "ws_col"
The colour in which special characters (such as "\n", "\t", "\e", etc.) are reported
(as single letters: 'n', 't', 'e', etc.)
• "info_col"
The colour in which other information is reported
Colours for regex descriptions
• "desc_regex_col"
The colour in which components of the regex are displayed
• "desc_text_col"
The colour in which descriptions of regex components are displayed
• "desc_sep_col"
The colour in which separators between component descriptions are displayed.
Colours for heatmaps
Any key that starts with "heatmap"... is treated as a specifier for an equal part of the
total range of each heatmap.
These names are sorted (numerically, if possible; otherwise alphabetically) and the
corresponding values are then used to display equal percentiles from the heatmap.
For example (using numeric sorting):
heatmap_0_colour : cyan on_black # 0-33rd percentile
heatmap_50_colour : yellow on_black # 34-66th percentile
heatmap_100_colour : red on_black # 67-100th percentile
Or, equivalently (using alphabetic sorting):
heatmap_infrequent : cyan on_black # 0-33rd percentile
heatmap_more_frequent : yellow on_black # 34-66th percentile
heatmap_very_frequent : red on_black # 67-100th percentile
Colour specifications
The colour values that may be used in any of the above colour specifications are any
combination of the following (i.e. the colour specifiers supported by the Term::ANSIColor
module):
clear reset bold dark
faint underline underscore blink
reverse concealed
black red green yellow
blue magenta cyan white
bright_black bright_red bright_green bright_yellow
bright_blue bright_magenta bright_cyan bright_white
on_black on_red on_green on_yellow
on_blue on_magenta on_cyan on_white
on_bright_black on_bright_red on_bright_green on_bright_yellow
on_bright_blue on_bright_magenta on_bright_cyan on_bright_white
The default colour configurations are:
try_col : bold magenta on_black
match_col : bold cyan on_black
fail_col : yellow on_red
ws_col : bold blue underline
info_col : white on_black
desc_regex_col : white on_black
desc_text_col : cyan on_black
desc_sep_col : blue on_black underline
heatmap__20th_percentile : white on_black
heatmap__40th_percentile : cyan on_blue
heatmap__60th_percentile : blue on_cyan
heatmap__80th_percentile : red on_yellow
heatmap_100th_percentile : yellow on_red
Output configuration
Normally Regexp::Debugger sends its visualizations to the terminal and expects input from
the same device.
However, you can configure the module to output its information (in standard JSON format)
to a nominated file instead, using the 'save_to' option:
save_to : filename_to_save_data_to.json
Data saved in this way may be re-animated using the "rxrx" utility, or by calling
"Regexp::Debugger::rxrx()" directly. (See: "COMMAND-LINE DEBUGGING" for details).
Configuration API
You can also configure the debugger on a program-by-program basis, by passing any of the
above key/value pairs when the module is loaded.
For example:
use Regexp::Debugger fail => 'bold red', whitespace => 'compact';
Note that any configuration specified in the user's .rxrx file is overridden by an
explicit specification of this type.
The commonest use of this mechanism is to dump regex debugging information from an non-
interactive program:
use Regexp::Debugger save_to => 'regex_debugged.json';
Note that, when 'save_to' is specified within a program, the value supplied does not have
to be a string specifying the filename. You can also provide an actual filehandle (or
equivalent). For example:
use Regexp::Debugger save_to => IO::Socket::INET->new(
Proto => "tcp",
PeerAddr => 'localhost:666',
);
COMMAND-LINE DEBUGGING
The module provides a non-exported subroutine ("rxrx()") that implements a useful command-
line regex debugging utility.
The utility can be invoked with:
perl -MRegexp::Debugger -E 'Regexp::Debugger::rxrx\(@ARGV\)'
which is usually aliased in the shell to "rxrx" (and will be referred to by that name
hereafter).
Regex debugging REPL
When called without any arguments, "rxrx" initiates a simple REPL that allows the user to
type in regexes and strings and debug matches between them:
• Any line starting with a "/" is treated as a new regex to match with. The closing "/"
may be omitted. If the closing "/" is supplied, any one or more of the following flags
may be specified immediately after it: "x", "i", "m", "s", "a", "u", "d", "l".
• Any line starting with a "+/" is treated as the first line of a new multi- line regex
to match with. Subsequent lines are added to the regex until the closing "/" is
encountered. Any one or more of the following flags may be specified immediately after
the closing "/": "x", "i", "m", "s", "a", "u", "d", "l".
• Any line starting with a "'" or """ is treated as a new string to match against. The
corresponding closing delimiter may be omitted.
• Any line beginning with "m" causes the REPL to match the current regex against the
current string, visualizing the match in the usual way.
• Any line beginning with "g" causes the REPL to exhaustively match the current regex
against the current string (i.e. as if the regex had a /g flag), visualizing all the
matches in the usual way.
• Any line beginning with "d" causes the REPL to display a detailed decomposition and
explanation of the current regex.
• Any line beginning with "q" or "x" causes the REPL to quit and exit.
• Any line beginning with "?" invokes the help listing for the REPL.
If the IO::Prompter module (version 0.004 or later) is available, the input process
remembers its history, which you can recall by typing "CTRL-R". Repeated "CTRL-R"'s step
successively backwards through earlier inputs. "CTRL-N" steps successfully forward again.
You can then use "CTRL-B"/"CTRL-F"/"CTRL-A"/"CTRL-E" to move the cursor around the line of
recalled input, to delete or insert characters. This is useful for modifying and retrying
a recently entered regex or string.
Debugging regexes from a dumped session
When called with a filename, "rxrx" first checks whether the file contains a JSON dump of
a previous debugging, in which case it replays the visualization of that regex match
interactively.
This is useful for debugging non-interactive programs where the 'save_to' option was used
(see "Output configuration" and "Configuration API").
In this mode, all the features of the interactive debugger (as listed under "INTERFACE")
are fully available: you can step forwards and backwards through the match, skip to the
successful submatch or a breakpoint, swap visualization modes, and take snapshots.
Wrap-around regex debugging
When called with the name of a file that does not contain a JSON dump, "rxrx" attempts to
execute the file as a Perl program, with Regexp::Debugger enabled at the top level. In
other words:
rxrx prog.pl
is a convenient shorthand for:
perl -MRegexp::Debugger prog.pl
LIMITATIONS
"/x"-mode comments
Due to limitations in the Perl "overload::constant()" mechanism, the current
implementation cannot always distinguish whether a regex has an external /x modifier (and
hence, what whitespace and comment characters mean). Whitespace is handled correctly in
almost all cases, but comments are sometimes not.
When processing a "# comment to end of line" within a regex, the module currently assumes
a "/x" is in effect at start of the regex (unless that assumption causes the regex to fail
to compile). This will sometimes cause erroneous behaviour if an unescaped "#" is used in
a non-"/x" regex.
Unfortunately, this limitation is unlikely to be fully removed in a future release, unless
an additional flag-detection mechanism is added to "overload::constant()".
Note, however, that this limitation does not affect the handling of comments in "(?x:...)"
blocks or of literal "#" in "(?-x:...)" blocks within a regex. These are always correctly
handled, which means that explicitly using either of these blocks is a reliable
workaround. Alternatively, there is no problem if you always use the "/x" modifier on
every debugged regex (for example, via "use re '/x'"). Nor if you explicitly escape every
literal "#" (i.e. write it as "\#").
As regards whitespace, the one case where the current implementation does not always
correctly infer behaviour is where whitespace is used to separate a repetition qualifier
from the atom it qualifies in a non-"/x" regex, such as:
/ x + /
Because the module defaults to assuming that regexes always have "/x" applied, this is
always interpreted as:
/\ x+\ /x
rather than what it really is, namely:
/\ x\ +\ /
The most reliable workaround for the time being is either to always use "/x" on any regex,
or never to separate repetition qualifiers from their preceding atoms.
Multiple 'save_to' with the same target
At present, making the same file the target of two successive "save_to" requests causes
the second JSON data structure to overwrite the first.
This limitation will be removed in a subsequent release (but this will certainly involve a
small change to the structure of the JSON data that is written, even when only one
"save_to" is specified).
Variable interpolations
The module handles the interpolation of strings correctly, expanding them in-place before
debugging begins.
However, it currently does not correctly handle the interpolation of "qr"'d regexes. That
is, this:
use Regexp::Debugger;
my $ident = qr{ [^\W\d]\w* }x; # a qr'd regex...
$str =~ m{ ($ident) : (.*) }xms; # ...interpolated into another regex
does not work correctly...and usually will not even compile.
It is expected that this limitation will be removed in a future release, however it may
only be possible to fix the problem for more recent versions of Perl (i.e. 5.14 and later)
in which the regex engine is re-entrant.
DIAGNOSTICS
"Odd number of configuration args after "use Regexp::Debugger""
The module expects configuration arguments (see "Configuration API") to be passed as
"key => value" pairs. You passed something else.
"Unknown 'show_ws' option: %s"
The only valid options for the 'show_ws' configuration option are 'compact',
'visible', or 'original'. You specified something else (or misspelled one of the
above).
"Unknown 'display' option: %s"
The only valid options for the 'display' configuration option are 'visual' or
'heatmap' or 'events' or 'JSON'. You specified something else (or misspelled one of
the above).
"Invalid 'save_to' option: %s (%s)"
The value associated with the 'save_to' option is expected to be a filehandle opened
for writing, or else a string containing the name of a file that can be opened for
writing. You either passed an unopened filehandle, an unwritable filename, or
something that wasn't a plausible file. Alternatively, if you passed a filepath, was
the directory not accessible to, or writeable by, you?
"Possible typo in %s"
Prior to executing each regex, the module checks for common regex problems that can be
detected statically. For example, it looks for the two most common typos made when
defining and calling independent subpatterns. Namely: writing "(<NAME>...)" instead
of "(?<NAME>...)" and "(&SUBPAT)" instead of "(?&SUBPAT)"
To silence these warnings, just fix the typos.
Or, if the constructs are intentional, change "(<NAME>...)" to "(\<NAME>...)" and
"(&SUBPAT)" to "(\&SUBPAT)"
DEPENDENCIES
This module only works with Perl 5.10.1 and later.
The following modules are used when available:
Term::ANSIColor
Text colouring only works if this module can be loaded. Otherwise, all output will be
monochromatic.
Win32::Console::ANSI
Under Windows, text colouring also requires that this module can be loaded.
Otherwise, all output will be monochromatic.
File::HomeDir
If it can't find a useful value for $ENV{HOME}, Regexp::Debugger attempts to use this
module to determine the user's home directory, in order to search for a .rxrx config
file.
JSON::XS
JSON
JSON::DWIW
JSON::Syck
JSON output (i.e. for the 'save_to' option) is only possible if one of these modules
can be loaded. Otherwise, all JSON output will default to an empty "{}".
Term::ReadKey
Single-character interactions only work if this module can be loaded. Otherwise, all
command interactions will require a "<RETURN>" after them.
Time::HiRes
Autogenerated timestamps (e.g. for snapshots) will only be sub-second accurate if this
module can be loaded. Otherwise, all timestamps will only be to the nearest second.
INCOMPATIBILITIES
None reported, but this module will almost certainly not play nicely with any other that
modifies regexes using "overload::constant".
BUGS AND LIMITATIONS
No bugs have been reported.
Please report any bugs or feature requests to "bug-regexp-debugger@rt.cpan.org", or
through the web interface at <http://rt.cpan.org>.
AUTHOR
Damian Conway "<DCONWAY@CPAN.org>"
LICENCE AND COPYRIGHT
Copyright (c) 2011-2012, Damian Conway "<DCONWAY@CPAN.org>". All rights reserved.
This module is free software; you can redistribute it and/or modify it under the same
terms as Perl itself. See perlartistic.
DISCLAIMER OF WARRANTY
BECAUSE THIS SOFTWARE IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE SOFTWARE,
TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE
COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE SOFTWARE "AS IS" WITHOUT WARRANTY OF
ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO
THE QUALITY AND PERFORMANCE OF THE SOFTWARE IS WITH YOU. SHOULD THE SOFTWARE PROVE
DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR, OR CORRECTION.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT
HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE SOFTWARE AS PERMITTED BY
THE ABOVE LICENCE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL,
INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE
SOFTWARE (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR
LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE SOFTWARE TO OPERATE WITH ANY
OTHER SOFTWARE), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.