Provided by: lintian_2.5.81ubuntu1_all 
NAME
Lintian::Util - Lintian utility functions
SYNOPSIS
use Lintian::Util qw(slurp_entire_file normalize_pkg_path);
my $text = slurp_entire_file('some-file');
if ($text =~ m/regex/) {
# ...
}
my $path = normalize_pkg_path('usr/bin/', '../lib/git-core/git-pull');
if (defined $path) {
# ...
}
my (@paragraphs);
eval { @paragraphs = read_dpkg_control_utf8('some/debian/ctrl/file'); };
if ($@) {
# syntax error etc.
die "ctrl/file: $@";
}
foreach my $para (@paragraphs) {
my $value = $para->{'some-field'};
if (defined $value) {
# ...
}
}
DESCRIPTION
This module contains a number of utility subs that are nice to have, but on their own did not warrant
their own module.
Most subs are imported only on request.
Debian control parsers
At first glance, this module appears to contain several debian control parsers. In practise, there is
only one real parser ("visit_dpkg_paragraph") - the rest are convenience functions around it.
If you have very large files (e.g. Packages_amd64), you almost certainly want "visit_dpkg_paragraph".
Otherwise, one of the convenience functions are probably what you are looking for.
Use "get_deb_info" when
You have a .deb (or .udeb) file and you want the control file from it.
Use "get_dsc_info" when
You have a .dsc (or .changes) file. Alternative, it is also useful if you have a control file and
only care about the first paragraph.
Use "read_dpkg_control_utf8" or "read_dpkg_control" when
You have a debian control file (such debian/control) and you want a number of paragraphs from it.
Use "parse_dpkg_control" when
When you would have used "read_dpkg_control_utf8", except you have an open filehandle rather than a
file name.
CONSTANTS
The following constants can be passed to the Debian control file parser functions to alter their parsing
flag.
DCTRL_DEBCONF_TEMPLATE
The file should be parsed as debconf template. These have slightly syntax rules for whitespace in
some cases.
DCTRL_NO_COMMENTS
The file do not allow comments. With this flag, any comment in the file is considered a syntax
error.
VARIABLES
$PKGNAME_REGEX
Regular expression that matches valid package names. The expression is not anchored and does not
enforce any "boundary" characters.
$PKGREPACK_REGEX
Regular expression that matches "repacked" package names. The expression is not anchored and does
not enforce any "boundary" characters.
$PKGVERSION_REGEX
Regular expression that matches valid package versions. The expression is not anchored and does not
enforce any "boundary" characters.
FUNCTIONS
parse_dpkg_control(HANDLE[, FLAGS[, LINES]])
Reads a debian control file from HANDLE and returns a list of paragraphs in it. A paragraph is
represented via a hashref, which maps (lower cased) field names to their values.
FLAGS (if given) is a bitmask of the DCTRL_* constants. Please refer to "CONSTANTS" for the list of
constants and their meaning. The default value for FLAGS is 0.
If LINES is given, it should be a reference to an empty list. On return, LINES will be populated
with a hashref for each paragraph (in the same order as the returned list). Each hashref will also
have a special key "START-OF-PARAGRAPH" that gives the line number of the first field in that
paragraph. These hashrefs will map the field name of the given paragraph to the line number where
the field name appeared.
This is a convenience sub around "visit_dpkg_paragraph" and can therefore produce the same errors as
it. Please see "visit_dpkg_paragraph" for the finer semantics of how the control file is parsed.
NB: parse_dpkg_control does not close the handle for the caller.
visit_dpkg_paragraph (CODE, HANDLE[, FLAGS])
Reads a debian control file from HANDLE and passes each paragraph to CODE. A paragraph is
represented via a hashref, which maps (lower cased) field names to their values.
FLAGS (if given) is a bitmask of the DCTRL_* constants. Please refer to "CONSTANTS" for the list of
constants and their meaning. The default value for FLAGS is 0.
If the file is empty (i.e. it contains no paragraphs), the method will contain an empty list. The
deb822 contents may be inside a signed PGP message with a signature.
visit_dpkg_paragraph will require the PGP headers to be correct (if present) and require that the
entire file is covered by the signature. However, it will not validate the signature (in fact, the
contents of the PGP SIGNATURE part can be empty). The signature should be validated separately.
visit_dpkg_paragraph will pass paragraphs to CODE as they are completed. If CODE can process the
paragraphs as they are seen, very large control files can be processed without keeping all the
paragraphs in memory.
As a consequence of how the file is parsed, CODE may be passed a number of (valid) paragraphs before
parsing is stopped due to a syntax error.
NB: visit_dpkg_paragraph does not close the handle for the caller.
CODE is expected to be a callable reference (e.g. a sub) and will be invoked as the following:
CODE->(PARA, LINE_NUMBERS)
The first argument, PARA, is a hashref to the most recent paragraph parsed. The second argument,
LINE_NUMBERS, is a hashref mapping each of the field names to the line number where the field
name appeared. LINE_NUMBERS will also have a special key "START-OF-PARAGRAPH" that gives the
line number of the first field in that paragraph.
The return value of CODE is ignored.
If the CODE invokes die (or similar) the error is propagated to the caller.
On syntax errors, visit_dpkg_paragraph will call die with the following string:
"syntax error at line %d: %s\n"
Where %d is the line number of the issue and %s is one of:
Duplicate field %s
The field appeared twice in the paragraph.
Continuation line outside a paragraph (maybe line %d should be " .")
A continuation line appears outside a paragraph - usually caused by an unintended empty line
before it.
Whitespace line not allowed (possibly missing a ".")
An empty continuation line was found. This usually means that a period is missing to denote an
"empty line" in (e.g.) the long description of a package.
Cannot parse line "%s"
Generic error containing the text of the line that confused the parser. Note that all non-
printables in %s will be replaced by underscores.
Comments are not allowed
A comment line appeared and FLAGS contained DCTRL_NO_COMMENTS.
PGP signature seen before start of signed message
A "BEGIN PGP SIGNATURE" header is seen and a "BEGIN PGP MESSAGE" has not been seen yet.
Two PGP signatures (first one at line %d)
Two "BEGIN PGP SIGNATURE" headers are seen in the same file.
Unexpected %s header
A valid PGP header appears (e.g. "BEGIN PUBLIC KEY BLOCK").
Malformed PGP header
An invalid or malformed PGP header appears.
Expected at most one signed message (previous at line %d)
Two "BEGIN PGP MESSAGE" headers appears in the same message.
End of file but expected an "END PGP SIGNATURE" header
The file ended after a "BEGIN PGP SIGNATURE" header without being followed by an "END PGP
SIGNATURE".
PGP MESSAGE header must be first content if present
The file had content before PGP MESSAGE.
Data after the PGP SIGNATURE
The file had data after the PGP SIGNATURE block ended.
End of file before "BEGIN PGP SIGNATURE"
The file had a "BEGIN PGP MESSAGE" header, but no signature was present.
read_dpkg_control_utf8(FILE[, FLAGS[, LINES]])
read_dpkg_control(FILE[, FLAGS[, LINES]])
This is a convenience function to ease using "parse_dpkg_control" with paths to files (rather than
open handles). The first argument must be the path to a FILE, which should be read as a debian
control file. If the file is empty, an empty list is returned.
Otherwise, this behaves like:
use autodie;
open(my $fd, '<:encoding(UTF-8)', FILE); # or '<'
my @p = parse_dpkg_control($fd, FLAGS, LINES);
close($fd);
return @p;
This goes without saying that may fail with any of the messages that "parse_dpkg_control(HANDLE[,
FLAGS[, LINES]])" do. It can also emit autodie exceptions if open or close fails.
get_deb_info(DEBFILE)
Extracts the control file from DEBFILE and returns it as a hashref.
Basically, this is a fancy convenience for setting up an ar + tar pipe and passing said pipe to
"parse_dpkg_control(HANDLE[, FLAGS[, LINES]])".
DEBFILE must be an ar file containing a "control.tar.gz" member, which in turn should contain a
"control" file. If the "control" file is empty this will return an empty list.
Note: the control file is only expected to have a single paragraph and thus only the first is
returned (in the unlikely case that there are more than one).
This function may fail with any of the messages that "parse_dpkg_control" do. It can also emit:
"cannot fork to unpack %s: %s\n"
get_dsc_control (DSCFILE)
Convenience function for reading dsc files. It will read the DSCFILE using "read_dpkg_control(FILE[,
FLAGS[, LINES]])" and then return the first paragraph. If the file has no paragraphs, "undef" is
returned instead.
Note: the control file is only expected to have a single paragraph and thus only the first is
returned (in the unlikely case that there are more than one).
This function may fail with any of the messages that "read_dpkg_control(FILE[, FLAGS[, LINES]])" do.
slurp_entire_file (FOH[, NOCLOSE])
Reads the contents of FOH into memory and return it as a scalar. FOH can be either the path to a
file or an open file handle.
If it is a handle, the optional NOCLOSE parameter can be used to prevent the sub from closing the
handle. The NOCLOSE parameter has no effect if FOH is not a handle.
drain_pipe(FD)
Reads and discards any remaining contents from FD, which is assumed to be a pipe. This is mostly
done to avoid having the "write"-end die with a SIGPIPE due to a "broken pipe" (which can happen if
you just close the pipe).
May cause an exception if there are issues reading from the pipe.
Caveat: This will block until the pipe is closed from the "write"-end, so only use it with pipes
where the "write"-end will eventually close their end by themselves (or something else will make them
close it).
get_file_digest(ALGO, FILE)
Creates an ALGO digest object that is seeded with the contents of FILE. If you just want the hex
digest, please use "get_file_checksum(ALGO, FILE)" instead.
ALGO can be 'md5' or shaX, where X is any number supported by Digest::SHA (e.g. 'sha256').
This sub is a convenience wrapper around Digest::{MD5,SHA}.
get_file_checksum(ALGO, FILE)
Returns a hexadecimal string of the message digest checksum generated by the algorithm ALGO on FILE.
ALGO can be 'md5' or shaX, where X is any number supported by Digest::SHA (e.g. 'sha256').
This sub is a convenience wrapper around Digest::{MD5,SHA}.
is_string_utf8_encoded(STRING)
Returns a truth value if STRING can be decoded as valid UTF-8.
file_is_encoded_in_non_utf8 (...)
Undocumented
do_fork()
Overrides fork to reset signal handlers etc. in the child.
system_env (CMD)
Behaves like system (CMD) except that the environment of CMD is cleaned (as defined by
"clean_env"(1)).
clean_env ([CLOC])
Destructively cleans %ENV - removes all variables %ENV except a selected few whitelisted variables.
The list of whitelisted %ENV variables are:
PATH
LC_ALL (*)
TMPDIR
(*) LC_ALL is a special case as clean_env will change its value to either "C.UTF-8" or "C" (if CLOC
is given and a truth value).
perm2oct(PERM)
Translates PERM to an octal permission. PERM should be a string describing the permissions as done
by tar t or ls -l. That is, it should be a string like "-rw-r--r--".
If the string does not appear to be a valid permission, it will cause a trappable error.
Examples:
# Good
perm2oct('-rw-r--r--') == 0644
perm2oct('-rwxr-xr-x') == 0755
# Bad
perm2oct('broken') # too short to be recognised
perm2oct('-resurunet') # contains unknown permissions
run_cmd([OPTS, ]COMMAND[, ARGS...])
Executes the given "COMMAND" with the (optional) arguments "ARGS" and returns the status code as one
would see it from a shell script. Shell features cannot be used.
OPTS, if given, is a hash reference with zero or more of the following key-value pairs:
chdir
The child process with chdir to the given directory before executing the command.
in The STDIN of the child process will be reopened and read from the filename denoted by the value
of this key. By default, STDIN will reopened to read from /dev/null.
out The STDOUT of the child process will be reopened and write to filename denoted by the value of
this key. By default, STDOUT is discarded.
update-env-vars
Each key/value pair defined in the hashref associated with update-env-vars will be updated in the
child processes's environment. If a value is "undef", then the corresponding environment
variable will be removed (if set). Otherwise, the environment value will be set to that value.
delete_dir (ARGS)
Convenient way of calling rm -fr ARGS.
copy_dir (ARGS)
Convenient way of calling cp -a ARGS.
gunzip_file (IN, OUT)
Decompresses contents of the file IN and stores the contents in the file OUT. IN is not removed by
this call. On error, this function will cause a trappable error.
open_gz (FILE)
Opens a handle that reads from the GZip compressed FILE.
On failure, this sub emits a trappable error.
Note: The handle may be a pipe from an external processes.
touch_file(FILE)
Updates the "mtime" of FILE. If FILE does not exist, it will be created.
On failure, this sub will emit a trappable error.
internal_error (MSG[, ...])
Use to signal an internal error. The argument(s) will used to print a diagnostic message to the user.
If multiple arguments are given, they will be merged into a single string (by join (' ', @_)). If
only one argument is given it will be stringified and used directly.
fail (MSG[, ...])
Deprecated alias of "internal_error".
locate_helper_tool(TOOLNAME)
Given the name of a helper tool, returns the path to it. The tool must be available in the "helpers"
subdir of one of the "lintian root" directories used by Lintian.
The tool name should follow the same rules as check names. Particularly, third-party checks should
namespace their tools in the same way they namespace their checks. E.g. "python/some-helper".
If the tool cannot be found, this sub will cause a trappable error.
strip ([LINE])
Strips whitespace from the beginning and the end of LINE and returns it. If LINE is omitted, $_ will
be used instead. Example
@lines = map { strip } <$fd>;
In void context, the input argument will be modified so it can be used as a replacement for chomp in
some cases:
while ( my $line = <$fd> ) {
strip ($line);
# $line no longer has any leading or trailing whitespace
}
Otherwise, a copy of the string is returned:
while ( my $orig = <$fd> ) {
my $stripped = strip ($orig);
if ($stripped ne $orig) {
# $orig had leading or/and trailing whitespace
}
}
lstrip ([LINE])
Like strip but only strip leading whitespace.
rstrip ([LINE])
Like strip but only strip trailing whitespace.
check_path (CMD)
Returns 1 if CMD can be found in PATH (i.e. $ENV{PATH}) and is executable. Otherwise, the function
return 0.
dequote_name(STR, REMOVESLASH)
Strip an extra layer quoting in index file names and optionally remove an initial "./" if any.
Remove initial ./ by default
signal_number2name(NUM)
Given a number, returns the name of the signal (without leading "SIG"). Example:
signal_number2name(2) eq 'INT'
normalize_pkg_path(PATH)
Normalize PATH by removing superfluous path segments. PATH is assumed to be relative the package
root. Note that the result will never start nor end with a slash, even if PATH does.
As the name suggests, this is a path "normalization" rather than a true path resolution (for that use
Cwd::realpath). Particularly, it assumes none of the path segments are symlinks.
normalize_pkg_path will return "q{}" (i.e. the empty string) if PATH is normalized to the root dir
and "undef" if the path cannot be normalized without escaping the package root.
Examples:
normalize_pkg_path('usr/share/java/../../../usr/share/ant/file')
eq 'usr/share/ant/file'
normalize_pkg_path('usr/..') eq q{};
The following will return C<undef>:
normalize_pkg_path('usr/bin/../../../../etc/passwd')
normalize_pkg_path(CURDIR, LINK_TARGET)
Normalize the path obtained by following a link with LINK_TARGET as its target from CURDIR as the
current directory. CURDIR is assumed to be relative to the package root. Note that the result will
never start nor end with a slash, even if CURDIR or DEST does.
normalize_pkg_path will return "q{}" (i.e. the empty string) if the target is the root dir and
"undef" if the path cannot be normalized without escaping the package root.
CAVEAT: This function is not always sufficient to test if it is safe to open a given symlink. Use
is_ancestor_of for that. If you must use this function, remember to check that the target is not a
symlink (or if it is, that it can be resolved safely).
Examples:
normalize_pkg_path('usr/share/java', '../ant/file') eq 'usr/share/ant/file'
normalize_pkg_path('usr/share/java', '../../../usr/share/ant/file')
normalize_pkg_path('usr/share/java', '/usr/share/ant/file')
eq 'usr/share/ant/file'
normalize_pkg_path('/usr/share/java', '/') eq q{};
normalize_pkg_path('/', 'usr/..') eq q{};
The following will return C<undef>:
normalize_pkg_path('usr/bin', '../../../../etc/passwd')
normalize_pkg_path('usr/bin', '/../etc/passwd')
parse_boolean (STR)
Attempt to parse STR as a boolean and return its value. If STR is not a valid/recognised boolean,
the sub will invoke croak.
The following values recognised (string checks are not case sensitive):
The integer 0 is considered false
Any non-zero integer is considered true
"true", "y" and "yes" are considered true
"false", "n" and "no" are considered false
is_ancestor_of(PARENTDIR, PATH)
Returns true if and only if PATH is PARENTDIR or a path stored somewhere within PARENTDIR (or its
subdirs).
This function will resolve the paths; any failure to resolve the path will cause a trappable error.
pipe_tee(INHANDLE, OUTHANDLES[, OPTS])
Read bytes from INHANDLE and copy them into all of the handles in the listref OUTHANDLES. The
optional OPTS argument is a hashref of options, see below.
The subroutine will continue to read from INHANDLE until it is exhausted or an error occurs (either
during read or write). In case of errors, a trappable error will be raised. The handles are left
open when the subroutine returns, caller must close them afterwards.
Caller should ensure that handles are using "blocking" I/O. The subroutine will use sysread and
syswrite when reading and writing.
OPTS, if given, may contain the following key-value pairs:
chunk_size
A suggested buffer size for read/write. If given, it will be to sysread as LENGTH argument when
reading from INHANDLE.
load_state_cache(STATE_DIR)
[Reporting tools only] Load the state cache from STATE_DIR.
save_state_cache(STATE_DIR, STATE)
[Reporting tools only] Save the STATE cache to STATE_DIR.
find_backlog(LINTIAN_VERSION, STATE)
[Reporting tools only] Given the current lintian version and the harness state, return a list of
group ids that are part of the backlog. The list is sorted based on what version of Lintian
processed the package.
Note the result is by design not deterministic to reduce the risk of all large packages being in the
same run (e.g. like gcc-5 + gcc-5-cross + gcc-6 + gcc-6-cross).
untaint(VALUE)
Untaint VALUE
SEE ALSO
lintian(1)
Lintian v2.5.81ubuntu1 2018-04-08 Lintian::Util(3)