Provided by: lintian_2.5.81ubuntu1_all 
NAME
Lintian::Util - Lintian utility functions
SYNOPSIS
use Lintian::Util qw(slurp_entire_file normalize_pkg_path);
my $text = slurp_entire_file('some-file');
if ($text =~ m/regex/) {
# ...
}
my $path = normalize_pkg_path('usr/bin/', '../lib/git-core/git-pull');
if (defined $path) {
# ...
}
my (@paragraphs);
eval { @paragraphs = read_dpkg_control_utf8('some/debian/ctrl/file'); };
if ($@) {
# syntax error etc.
die "ctrl/file: $@";
}
foreach my $para (@paragraphs) {
my $value = $para->{'some-field'};
if (defined $value) {
# ...
}
}
DESCRIPTION
This module contains a number of utility subs that are nice to have, but on their own did
not warrant their own module.
Most subs are imported only on request.
Debian control parsers
At first glance, this module appears to contain several debian control parsers. In
practise, there is only one real parser ("visit_dpkg_paragraph") - the rest are
convenience functions around it.
If you have very large files (e.g. Packages_amd64), you almost certainly want
"visit_dpkg_paragraph". Otherwise, one of the convenience functions are probably what you
are looking for.
Use "get_deb_info" when
You have a .deb (or .udeb) file and you want the control file from it.
Use "get_dsc_info" when
You have a .dsc (or .changes) file. Alternative, it is also useful if you have a
control file and only care about the first paragraph.
Use "read_dpkg_control_utf8" or "read_dpkg_control" when
You have a debian control file (such debian/control) and you want a number of
paragraphs from it.
Use "parse_dpkg_control" when
When you would have used "read_dpkg_control_utf8", except you have an open filehandle
rather than a file name.
CONSTANTS
The following constants can be passed to the Debian control file parser functions to alter
their parsing flag.
DCTRL_DEBCONF_TEMPLATE
The file should be parsed as debconf template. These have slightly syntax rules for
whitespace in some cases.
DCTRL_NO_COMMENTS
The file do not allow comments. With this flag, any comment in the file is considered
a syntax error.
VARIABLES
$PKGNAME_REGEX
Regular expression that matches valid package names. The expression is not anchored
and does not enforce any "boundary" characters.
$PKGREPACK_REGEX
Regular expression that matches "repacked" package names. The expression is not
anchored and does not enforce any "boundary" characters.
$PKGVERSION_REGEX
Regular expression that matches valid package versions. The expression is not
anchored and does not enforce any "boundary" characters.
FUNCTIONS
parse_dpkg_control(HANDLE[, FLAGS[, LINES]])
Reads a debian control file from HANDLE and returns a list of paragraphs in it. A
paragraph is represented via a hashref, which maps (lower cased) field names to their
values.
FLAGS (if given) is a bitmask of the DCTRL_* constants. Please refer to "CONSTANTS"
for the list of constants and their meaning. The default value for FLAGS is 0.
If LINES is given, it should be a reference to an empty list. On return, LINES will
be populated with a hashref for each paragraph (in the same order as the returned
list). Each hashref will also have a special key "START-OF-PARAGRAPH" that gives the
line number of the first field in that paragraph. These hashrefs will map the field
name of the given paragraph to the line number where the field name appeared.
This is a convenience sub around "visit_dpkg_paragraph" and can therefore produce the
same errors as it. Please see "visit_dpkg_paragraph" for the finer semantics of how
the control file is parsed.
NB: parse_dpkg_control does not close the handle for the caller.
visit_dpkg_paragraph (CODE, HANDLE[, FLAGS])
Reads a debian control file from HANDLE and passes each paragraph to CODE. A
paragraph is represented via a hashref, which maps (lower cased) field names to their
values.
FLAGS (if given) is a bitmask of the DCTRL_* constants. Please refer to "CONSTANTS"
for the list of constants and their meaning. The default value for FLAGS is 0.
If the file is empty (i.e. it contains no paragraphs), the method will contain an
empty list. The deb822 contents may be inside a signed PGP message with a signature.
visit_dpkg_paragraph will require the PGP headers to be correct (if present) and
require that the entire file is covered by the signature. However, it will not
validate the signature (in fact, the contents of the PGP SIGNATURE part can be empty).
The signature should be validated separately.
visit_dpkg_paragraph will pass paragraphs to CODE as they are completed. If CODE can
process the paragraphs as they are seen, very large control files can be processed
without keeping all the paragraphs in memory.
As a consequence of how the file is parsed, CODE may be passed a number of (valid)
paragraphs before parsing is stopped due to a syntax error.
NB: visit_dpkg_paragraph does not close the handle for the caller.
CODE is expected to be a callable reference (e.g. a sub) and will be invoked as the
following:
CODE->(PARA, LINE_NUMBERS)
The first argument, PARA, is a hashref to the most recent paragraph parsed. The
second argument, LINE_NUMBERS, is a hashref mapping each of the field names to the
line number where the field name appeared. LINE_NUMBERS will also have a special
key "START-OF-PARAGRAPH" that gives the line number of the first field in that
paragraph.
The return value of CODE is ignored.
If the CODE invokes die (or similar) the error is propagated to the caller.
On syntax errors, visit_dpkg_paragraph will call die with the following string:
"syntax error at line %d: %s\n"
Where %d is the line number of the issue and %s is one of:
Duplicate field %s
The field appeared twice in the paragraph.
Continuation line outside a paragraph (maybe line %d should be " .")
A continuation line appears outside a paragraph - usually caused by an unintended
empty line before it.
Whitespace line not allowed (possibly missing a ".")
An empty continuation line was found. This usually means that a period is missing
to denote an "empty line" in (e.g.) the long description of a package.
Cannot parse line "%s"
Generic error containing the text of the line that confused the parser. Note that
all non-printables in %s will be replaced by underscores.
Comments are not allowed
A comment line appeared and FLAGS contained DCTRL_NO_COMMENTS.
PGP signature seen before start of signed message
A "BEGIN PGP SIGNATURE" header is seen and a "BEGIN PGP MESSAGE" has not been seen
yet.
Two PGP signatures (first one at line %d)
Two "BEGIN PGP SIGNATURE" headers are seen in the same file.
Unexpected %s header
A valid PGP header appears (e.g. "BEGIN PUBLIC KEY BLOCK").
Malformed PGP header
An invalid or malformed PGP header appears.
Expected at most one signed message (previous at line %d)
Two "BEGIN PGP MESSAGE" headers appears in the same message.
End of file but expected an "END PGP SIGNATURE" header
The file ended after a "BEGIN PGP SIGNATURE" header without being followed by an
"END PGP SIGNATURE".
PGP MESSAGE header must be first content if present
The file had content before PGP MESSAGE.
Data after the PGP SIGNATURE
The file had data after the PGP SIGNATURE block ended.
End of file before "BEGIN PGP SIGNATURE"
The file had a "BEGIN PGP MESSAGE" header, but no signature was present.
read_dpkg_control_utf8(FILE[, FLAGS[, LINES]])
read_dpkg_control(FILE[, FLAGS[, LINES]])
This is a convenience function to ease using "parse_dpkg_control" with paths to files
(rather than open handles). The first argument must be the path to a FILE, which
should be read as a debian control file. If the file is empty, an empty list is
returned.
Otherwise, this behaves like:
use autodie;
open(my $fd, '<:encoding(UTF-8)', FILE); # or '<'
my @p = parse_dpkg_control($fd, FLAGS, LINES);
close($fd);
return @p;
This goes without saying that may fail with any of the messages that
"parse_dpkg_control(HANDLE[, FLAGS[, LINES]])" do. It can also emit autodie
exceptions if open or close fails.
get_deb_info(DEBFILE)
Extracts the control file from DEBFILE and returns it as a hashref.
Basically, this is a fancy convenience for setting up an ar + tar pipe and passing
said pipe to "parse_dpkg_control(HANDLE[, FLAGS[, LINES]])".
DEBFILE must be an ar file containing a "control.tar.gz" member, which in turn should
contain a "control" file. If the "control" file is empty this will return an empty
list.
Note: the control file is only expected to have a single paragraph and thus only the
first is returned (in the unlikely case that there are more than one).
This function may fail with any of the messages that "parse_dpkg_control" do. It can
also emit:
"cannot fork to unpack %s: %s\n"
get_dsc_control (DSCFILE)
Convenience function for reading dsc files. It will read the DSCFILE using
"read_dpkg_control(FILE[, FLAGS[, LINES]])" and then return the first paragraph. If
the file has no paragraphs, "undef" is returned instead.
Note: the control file is only expected to have a single paragraph and thus only the
first is returned (in the unlikely case that there are more than one).
This function may fail with any of the messages that "read_dpkg_control(FILE[, FLAGS[,
LINES]])" do.
slurp_entire_file (FOH[, NOCLOSE])
Reads the contents of FOH into memory and return it as a scalar. FOH can be either
the path to a file or an open file handle.
If it is a handle, the optional NOCLOSE parameter can be used to prevent the sub from
closing the handle. The NOCLOSE parameter has no effect if FOH is not a handle.
drain_pipe(FD)
Reads and discards any remaining contents from FD, which is assumed to be a pipe.
This is mostly done to avoid having the "write"-end die with a SIGPIPE due to a
"broken pipe" (which can happen if you just close the pipe).
May cause an exception if there are issues reading from the pipe.
Caveat: This will block until the pipe is closed from the "write"-end, so only use it
with pipes where the "write"-end will eventually close their end by themselves (or
something else will make them close it).
get_file_digest(ALGO, FILE)
Creates an ALGO digest object that is seeded with the contents of FILE. If you just
want the hex digest, please use "get_file_checksum(ALGO, FILE)" instead.
ALGO can be 'md5' or shaX, where X is any number supported by Digest::SHA (e.g.
'sha256').
This sub is a convenience wrapper around Digest::{MD5,SHA}.
get_file_checksum(ALGO, FILE)
Returns a hexadecimal string of the message digest checksum generated by the algorithm
ALGO on FILE.
ALGO can be 'md5' or shaX, where X is any number supported by Digest::SHA (e.g.
'sha256').
This sub is a convenience wrapper around Digest::{MD5,SHA}.
is_string_utf8_encoded(STRING)
Returns a truth value if STRING can be decoded as valid UTF-8.
file_is_encoded_in_non_utf8 (...)
Undocumented
do_fork()
Overrides fork to reset signal handlers etc. in the child.
system_env (CMD)
Behaves like system (CMD) except that the environment of CMD is cleaned (as defined by
"clean_env"(1)).
clean_env ([CLOC])
Destructively cleans %ENV - removes all variables %ENV except a selected few
whitelisted variables.
The list of whitelisted %ENV variables are:
PATH
LC_ALL (*)
TMPDIR
(*) LC_ALL is a special case as clean_env will change its value to either "C.UTF-8" or
"C" (if CLOC is given and a truth value).
perm2oct(PERM)
Translates PERM to an octal permission. PERM should be a string describing the
permissions as done by tar t or ls -l. That is, it should be a string like
"-rw-r--r--".
If the string does not appear to be a valid permission, it will cause a trappable
error.
Examples:
# Good
perm2oct('-rw-r--r--') == 0644
perm2oct('-rwxr-xr-x') == 0755
# Bad
perm2oct('broken') # too short to be recognised
perm2oct('-resurunet') # contains unknown permissions
run_cmd([OPTS, ]COMMAND[, ARGS...])
Executes the given "COMMAND" with the (optional) arguments "ARGS" and returns the
status code as one would see it from a shell script. Shell features cannot be used.
OPTS, if given, is a hash reference with zero or more of the following key-value
pairs:
chdir
The child process with chdir to the given directory before executing the command.
in The STDIN of the child process will be reopened and read from the filename denoted
by the value of this key. By default, STDIN will reopened to read from /dev/null.
out The STDOUT of the child process will be reopened and write to filename denoted by
the value of this key. By default, STDOUT is discarded.
update-env-vars
Each key/value pair defined in the hashref associated with update-env-vars will be
updated in the child processes's environment. If a value is "undef", then the
corresponding environment variable will be removed (if set). Otherwise, the
environment value will be set to that value.
delete_dir (ARGS)
Convenient way of calling rm -fr ARGS.
copy_dir (ARGS)
Convenient way of calling cp -a ARGS.
gunzip_file (IN, OUT)
Decompresses contents of the file IN and stores the contents in the file OUT. IN is
not removed by this call. On error, this function will cause a trappable error.
open_gz (FILE)
Opens a handle that reads from the GZip compressed FILE.
On failure, this sub emits a trappable error.
Note: The handle may be a pipe from an external processes.
touch_file(FILE)
Updates the "mtime" of FILE. If FILE does not exist, it will be created.
On failure, this sub will emit a trappable error.
internal_error (MSG[, ...])
Use to signal an internal error. The argument(s) will used to print a diagnostic
message to the user.
If multiple arguments are given, they will be merged into a single string (by join ('
', @_)). If only one argument is given it will be stringified and used directly.
fail (MSG[, ...])
Deprecated alias of "internal_error".
locate_helper_tool(TOOLNAME)
Given the name of a helper tool, returns the path to it. The tool must be available
in the "helpers" subdir of one of the "lintian root" directories used by Lintian.
The tool name should follow the same rules as check names. Particularly, third-party
checks should namespace their tools in the same way they namespace their checks. E.g.
"python/some-helper".
If the tool cannot be found, this sub will cause a trappable error.
strip ([LINE])
Strips whitespace from the beginning and the end of LINE and returns it. If LINE is
omitted, $_ will be used instead. Example
@lines = map { strip } <$fd>;
In void context, the input argument will be modified so it can be used as a
replacement for chomp in some cases:
while ( my $line = <$fd> ) {
strip ($line);
# $line no longer has any leading or trailing whitespace
}
Otherwise, a copy of the string is returned:
while ( my $orig = <$fd> ) {
my $stripped = strip ($orig);
if ($stripped ne $orig) {
# $orig had leading or/and trailing whitespace
}
}
lstrip ([LINE])
Like strip but only strip leading whitespace.
rstrip ([LINE])
Like strip but only strip trailing whitespace.
check_path (CMD)
Returns 1 if CMD can be found in PATH (i.e. $ENV{PATH}) and is executable. Otherwise,
the function return 0.
dequote_name(STR, REMOVESLASH)
Strip an extra layer quoting in index file names and optionally remove an initial "./"
if any.
Remove initial ./ by default
signal_number2name(NUM)
Given a number, returns the name of the signal (without leading "SIG"). Example:
signal_number2name(2) eq 'INT'
normalize_pkg_path(PATH)
Normalize PATH by removing superfluous path segments. PATH is assumed to be relative
the package root. Note that the result will never start nor end with a slash, even if
PATH does.
As the name suggests, this is a path "normalization" rather than a true path
resolution (for that use Cwd::realpath). Particularly, it assumes none of the path
segments are symlinks.
normalize_pkg_path will return "q{}" (i.e. the empty string) if PATH is normalized to
the root dir and "undef" if the path cannot be normalized without escaping the package
root.
Examples:
normalize_pkg_path('usr/share/java/../../../usr/share/ant/file')
eq 'usr/share/ant/file'
normalize_pkg_path('usr/..') eq q{};
The following will return C<undef>:
normalize_pkg_path('usr/bin/../../../../etc/passwd')
normalize_pkg_path(CURDIR, LINK_TARGET)
Normalize the path obtained by following a link with LINK_TARGET as its target from
CURDIR as the current directory. CURDIR is assumed to be relative to the package
root. Note that the result will never start nor end with a slash, even if CURDIR or
DEST does.
normalize_pkg_path will return "q{}" (i.e. the empty string) if the target is the root
dir and "undef" if the path cannot be normalized without escaping the package root.
CAVEAT: This function is not always sufficient to test if it is safe to open a given
symlink. Use is_ancestor_of for that. If you must use this function, remember to
check that the target is not a symlink (or if it is, that it can be resolved safely).
Examples:
normalize_pkg_path('usr/share/java', '../ant/file') eq 'usr/share/ant/file'
normalize_pkg_path('usr/share/java', '../../../usr/share/ant/file')
normalize_pkg_path('usr/share/java', '/usr/share/ant/file')
eq 'usr/share/ant/file'
normalize_pkg_path('/usr/share/java', '/') eq q{};
normalize_pkg_path('/', 'usr/..') eq q{};
The following will return C<undef>:
normalize_pkg_path('usr/bin', '../../../../etc/passwd')
normalize_pkg_path('usr/bin', '/../etc/passwd')
parse_boolean (STR)
Attempt to parse STR as a boolean and return its value. If STR is not a
valid/recognised boolean, the sub will invoke croak.
The following values recognised (string checks are not case sensitive):
The integer 0 is considered false
Any non-zero integer is considered true
"true", "y" and "yes" are considered true
"false", "n" and "no" are considered false
is_ancestor_of(PARENTDIR, PATH)
Returns true if and only if PATH is PARENTDIR or a path stored somewhere within
PARENTDIR (or its subdirs).
This function will resolve the paths; any failure to resolve the path will cause a
trappable error.
pipe_tee(INHANDLE, OUTHANDLES[, OPTS])
Read bytes from INHANDLE and copy them into all of the handles in the listref
OUTHANDLES. The optional OPTS argument is a hashref of options, see below.
The subroutine will continue to read from INHANDLE until it is exhausted or an error
occurs (either during read or write). In case of errors, a trappable error will be
raised. The handles are left open when the subroutine returns, caller must close them
afterwards.
Caller should ensure that handles are using "blocking" I/O. The subroutine will use
sysread and syswrite when reading and writing.
OPTS, if given, may contain the following key-value pairs:
chunk_size
A suggested buffer size for read/write. If given, it will be to sysread as LENGTH
argument when reading from INHANDLE.
load_state_cache(STATE_DIR)
[Reporting tools only] Load the state cache from STATE_DIR.
save_state_cache(STATE_DIR, STATE)
[Reporting tools only] Save the STATE cache to STATE_DIR.
find_backlog(LINTIAN_VERSION, STATE)
[Reporting tools only] Given the current lintian version and the harness state, return
a list of group ids that are part of the backlog. The list is sorted based on what
version of Lintian processed the package.
Note the result is by design not deterministic to reduce the risk of all large
packages being in the same run (e.g. like gcc-5 + gcc-5-cross + gcc-6 + gcc-6-cross).
untaint(VALUE)
Untaint VALUE
SEE ALSO
lintian(1)