Ubuntu Manpage: md5deep - Compute and compare MD5 message digests

NAME

       md5deep - Compute and compare MD5 message digests
       sha1deep - Compute and compare SHA-1 message digests
       sha256deep - Compute and compare SHA-256 message digests
       tigerdeep - Compute and compare Tiger message digests
       whirlpooldeep - Compute and compare Whirlpool message digests

SYNOPSIS

       md5deep -v | -V | -h
       md5deep  [-m|-M|-x|-X  <file>]   [-a|-A  <hash>]  [-f  <file>]  [-p  <size>]  [-i  <size>]
       [-tnwzresS0lbkqZud] [-F <bum>] [-o <fbcplsde>]  [-j <num>] [[FILES]

DESCRIPTION

Computes the hashes, or message digest, for any number of files while optionally
recursively digging through the directory structure. Can also take a list of known hashes
and display the filenames of input files whose hashes either do or do not match any of the
known hashes. Errors are reported to standard error. If no FILES are specified, reads
from standard input.

-p <size>
Piecewise mode. Breaks files into chunks before hashing. Chunks may be specified
using IEC multipliers b, k, m, g, t, p, or e. (Never let it be said that the
author didn't plan ahead!) This mode cannot be used with the -z mode.

-i|-I <size>
Size threshold mode. Only hash files smaller than the given the threshold. In -i
mode, simply omits those files larger than the threshold. In -I mode, displays all
files, but uses asterisks for the hashes of files larger than the threshold. Sizes
may be specified using IEC multipliers b, k, m, g, t, p, or e.

-r Enables recursive mode. All subdirectories are traversed. Please note that
recursive mode cannot be used to examine all files of a given file extension. For
example, calling md5deep -r *.txt will examine all files in directories that end in
.txt.

-e Displays a progress indicator and estimate of time remaining for each file being
processed. Time estimates for files larger than 4GB are not available on Windows.
This mode may not be used with th -p mode.

-m <file>
Enables matching mode. The file given should be a list of known hashes. The input
files are examined one at a time, and only those files that match the list of known
hashes are output. This flag may be used more than once to add multiple sets of
known hashes. Acceptable formats for lists of known hashes are plain (such as those
generated by md5deep or md5sum), Hashkeeper files, iLook, and the National Software
Reference Library (NSRL) as produced by the National Institute for Standards in
Technology.
If standard input is used with the -m flag, displays "stdin" if the input matches
one of the hashes in the list of known hashes. If the hash does not match, the
program displays no output.
This flag may not be used in conjunction with the -x, -X, or -A flags. See the
section "UNICODE SUPPORT" below.

-x <file>
Same as the -m flag above, but does negative matching. That is, only those files
NOT in the list of known hashes are displayed.
This flag may not be used in conjunction with the -m, -M, or -a flags. See the
section "UNICODE SUPPORT" below.

-M and -X <file>
Same as -m and -x above, but displays the hash for each file that does (or does
not) match the list of known hashes.

-a <hash>
Adds a single hash to the list of known hashes used for matching mode, and if not
already enabled, enables matching mode. Adding single hashes cannot, by itself, be
used to print the hashes of matching files like the -M flag does. When used in
conjunction with the -w flag, the filename displayed is just the hash submitted on
the command line.
This flag may not be used in conjunction with the -x, -X, or -A flags.

-A <hash>
Same as -a above, but does negative matching. This flag may not be used in
conjunction with the -m, -M, or -A flags.

-f <file>
Takes a list of files to be hashed from the specified file. Each line is assumed to
be a filename. This flag can only be used once per invocation. If it's used a
second time, the second instance will clobber the first.
Note that you can still use other flags, such as the -m or -x modes, and submit
additional FILES on the command line.

-w During any of the matching modes (-m,-M,-x,or -X), displays the filename of the
known hash that matched the input file. See the section "UNICODE SUPPORT" below.

-t Display a timestamp in GMT with each result. On Windows this timestamp will be the
file's creation time. On all other systems it should be the file's change time.

-n During any of the matching modes (-m,-M,-x,or -X), displays only the filenames of
any known hashes that were not matched by any of the input files.

-s Enables silent mode. All error messages are supressed.

-S Like silent mode, but still displays warnings on improperly formatted hashes in the
list of known hashes.

-z Enables file size mode. Prepends the hash with a ten digit representation of the
size of each file processed. If the file size is greater than 9999999999 bytes
(about 9.3GB) the program displays 9999999999 for the size.

-q Quiet mode. File names are omitted from the output.

-Z Produces output in Triage format. Each line contans the file's size, a tab, a hash
of the first 512 bytes, a tab, the hash of the complete file, a tab, and the file
name. These values are intended in increasing order of specificity. That is, two
files with different sizes cannot possibly match. This is a fast comparison and
should be done first. Next, two files with different partial hashes cannot possibly
match. This is often faster than hashing the whole file. Finally, if those two
pieces align, then it's worth reading and hashing the entire file.

-0 Uses a NULL character (/0) to terminate each line instead of a newline. Useful for
processing filenames with strange characters.

-l Enables relative file paths. Instead of printing the absolute path for each file,
displays the relative file path as indicated on the command line. This flag may not
be used in conjunction with the -b flag.

-b Enables bare mode. Strips any leading directory information from displayed
filenames. This flag may not be used in conjunction with the -l flag.

-k Enables asterisk mode. An asterisk is inserted in lieu of a second space between
the filename and the hash, just like md5sum in its binary (-b) mode.

-c Enables comma separated values output, or CSV mode. This mode has the side effect
of removing the 10 digit size limitation from -z mode. Also note that asterisks
from -k mode are not displayed when in CSV mode.

-o <bcpflsd>
Enables expert mode. Allows the user specify which (and only which) types of files
are processed. Directory processing is still controlled with the -r flag. The
expert mode options allowed are:
f - Regular files
b - Block Devices
c - Character Devices
p - Named Pipes
l - Symbolic Links
s - Sockets
d - Solaris Doors
e - Windows PE executables

-jnn Controls multi-threading. By default the program will create one producer thread to
scan the file system and one hashing thread per CPU core. Multi-threading causes
output filenames to be in non-deterministic order, as files that take longer to
hash will be delayed while they are hashed. If a deterministic order is required,
specify -j0 to disable multi-threading

-d Output in Digital Forensics XML (DFXML) format.

-u Quote Unicode output. For example, the snowman is shown as U+C426.

-F<bum>
Specifies the input mode that is used to read files. The default is -Fb (buffered
I/O) which reads files with fopen(). Specifying -Fu will use unbuffered I/O and
read the file with open(). Specifying -Fm will use memory-mapped I/O which will be
faster on some platforms, but which (currently) will not work with files that
produce I/O errors.

-h Show a help screen and exit.

-v Show the version number and exit.

-V Show copyright information and exit.

UNICODE SUPPORT

       As  of  version  3.0  the  program  supports  Unicode characters in filenames on Microsoft
       Windows systems for filenames specified on the command line with globbing  (e.g.  *),  for
       files  specified  with  the -f of files to hash, and for files read from directories using
       the -r option.

       By default all program input and output should be in  UTF-8.   The  program  automatically
       converts this to UTF-16 for opening files).

       On  Unix/Linux/MacOS,  you  should  use  a terminal emulator that supports UTF-8 and UTF-8
       characters in filenames will be properly displayed.

       On Windows, please note that the onsole is not capiable of displaying Unicode  characters.
       You  must  either  redirect  output  to  a  file and open the file with Wordpad (which can
       display Unicode), or you must specify the -u option to quote Unicode using standard U+XXXX
       notation.

       Currently  the  file  name  of  a  file  containing known hashes may not be specified as a
       unicode filename, but you can specify the name using tab completition or an asterisk (e.g.
       md5deep -m *.txt where there is only one file with a .txt extension).

RETURN VALUE

       Returns  a  bit-wise  value  based  on  the success of the operation and the status of any
       matching operations.

       0      Success. Note that the program considers itself successful even when it  encounters
              read  errors,  permission denied errors, or finds directories when not in recursive
              mode.

       1      Unused hashes. Under any of the matching modes, returns this value if one  or  more
              of the known hashes was not matched by any of the input files.

       2      Unmatched  inputs.  Under  any  of the matching modes, returns this value if one or
              more of the input values did not match any of the known hashes.

       64     User error, such as trying to do both positive and negative matching  at  the  same
              time.

       128    Internal  error,  such as memory corruption or uncaught cycle.  All internal errors
              should be reported to the developer! See the section "Reporting Bugs" below.

AUTHOR

       md5deep was written by Jesse Kornblum, research@jessekornblum.com and Simson Garfinkel.

KNOWN ISSUES

       Using the -r flag cannot be used to recursively process all files of a given extension  in
       a  directory.  This  is  a  feature,  not  a bug.  If you need to do this, use the find(1)
       command.

REPORTING BUGS

       We take all bug reports very seriously. Any bug that jeopardizes the forensic integrity of
       this  program  could  have  serious  consequenses on people's lives. When submitting a bug
       report, please include a description of the problem, how you found it,  and  your  contact
       information.

       Send bug reports to the author at the address above.

COPYRIGHT

       This  program  is  a  work  of the US Government. In accordance with 17 USC 105, copyright
       protection is not available for any work of the US Government.   This  program  is  PUBLIC
       DOMAIN.  Portions  of  this  program  contain code that is licensed under the terms of the
       General Public License (GPL).  Those portions retain their original copyright and license.
       See the file COPYING for more details.

       There  is  NO  warranty  for  this  program; not even for MERCHANTABILITY or FITNESS FOR A
       PARTICULAR PURPOSE.