Provided by: ddpt_0.92-1_amd64 
NAME
ddpt - copies data between files and storage devices. Support for devices that understand the SCSI
command set.
SYNOPSIS
ddpt [bpt=BPT[,OBPC]] [bs=BS] [cdbsz=6|10|12|16] [coe=0|1] [coe_limit=CL] [conv=CONVS] [count=COUNT]
[ibs=IBS] if=IFILE [iflag=FLAGS] [obs=OBS] [of=OFILE] [of2=OFILE2] [oflag=FLAGS] [retries=RETR]
[seek=SEEK] [skip=SKIP] [status=STAT] [verbose=VERB] [--help] [--verbose] [--version] [--wscan]
For comparison here is the synopsis for GNU's dd command:
dd [bs=BS] [cbs=CBS] [conv=CONVS] [count=COUNT] [ibs=IBS] [if=IFILE] [iflag=FLAGS] [obs=OBS] [of=OFILE]
[oflag=FLAGS] [seek=SEEK] [skip=SKIP] [status=STAT] [--help] [--version]
DESCRIPTION
Copy data between files or read data from a file. Specialized for "files" that are storage devices,
especially those that can use the SCSI command sets (e.g. SATA and SAS disks, plus DVD drives). Can issue
SCSI commands in pass-through ("pt") mode. Similar syntax and semantics to the Unix dd(1) command.
For comparison, the SYNOPSIS section above shows both the ddpt command line options followed by GNU's
dd(1) command line options. Broadly speaking ddpt can be considered a super-set of dd. See the section on
DD DIFFERENCES for significant differences between ddpt and dd.
ddpt does a segmented copy, first reading in BPT*IBS bytes from IFILE (or less if near the end of the
copy) into a copy buffer. In the absence of the various options and conditions that bypass the write
operation, the copy buffer is then written out to OFILE. The copy process continues working its way along
IFILE and OFILE until either COUNT is exhausted, an end of file is detected, or an error occurs. If IBS
and OBS are different, ddpt restricts the value of OBS such that the copy buffer is an integral number
output blocks (i.e. (((IBS*BPT) % OBS) == 0) ). In the following descriptions, "segment" refers to all or
part of a copy buffer.
The term "pt device" is used for a pass-through device to which SCSI commands like READ(10) and WRITE(10)
may be sent. A pt device may only be able to process SCSI commands in which case the "pt" flag is
assumed. The ability to recognize such a pt only device may vary depending on the operating system (e.g.
in Linux '/dev/sg2' is recognized). However if a device can process either normal UNIX read()/write()
calls or pass-through SCSI commands then the default is to use UNIX read()/write() calls. That default
can be overridden by using the "pt" flag (e.g. "if=/dev/sdc iflag=pt"). When pt access is specified any
partition information is ignored. So "if=/dev/sdc2 iflag=pt skip=3" will start at logical block address
3 of '/dev/sdc'. As a protection measure in version 0.92 ddpt will only accept that if the force flag is
given in addition to pt.
OPTIONS
bpt=BPT[,OBPC]
where BPT is Blocks Per Transfer. The copy is made up of multiple transfers, each first reading
BPT input blocks (i.e. BPT*IBS bytes) from IFILE into the copy buffer and then from that copy
buffer writing BPT*IBS/OBS output blocks to OFILE. This continues until the copy is finished, with
the last transfer being potentially shorter. The default BPT value varies depending on IBS. When
IBS < 8, BPT is 8192; when IBS < 64, BPT is 1024; when IBS < 1024, BPT is 128; when IBS < 8192,
BPT is 16; when IBS < 32768, BPT is 4; else BPT defaults to 1. If BPT is given as 0 it is treated
as the default value. For "bs=512", BPT defaults to 128 so that 64 KiB (or less) is read from
IFILE into the copy buffer.
The optional OBPC (Output Blocks Per Check) argument controls controls the granularity of sparse
writes, write sparing and trim checks. The default granularity is the size of the copy buffer
(i.e. BPT*IBS bytes). That can be reduced by specifying OBPC. The finest granularity is when OBPC
is 1 which implies the unit of each check is OBS bytes. When OBPC is 0, or not given, the default
granularity is used. Large OBPC values are rounded down so that OBPC*OBS does not exceed the size
of the copy buffer.
bs=BS where BS is the IFILE and OFILE block size in bytes. Conflicts with either "ibs=" or "obs="
options. The value of BS is placed in IBS and OBS. If IFILE or OFILE is a "pt" device then BS
must be the logical block size of the device. See the DD DIFFERENCES section below. Default is 512
which to date has been correct for hard disks. Other logical block sizes are 2048 bytes for DVDs
and 4096 bytes for the coming generation of hard disks.
cdbsz=6 | 10 | 12 | 16
size of SCSI READ and/or WRITE commands issued on pt devices. Default is 10 byte SCSI command
blocks (unless calculations indicate that a 4 byte block number may be exceeded or BPT is greater
than 16 bits (65535), in which case it defaults to 16 byte SCSI commands).
coe=0 | 1
set to 1 for continue on error. Applies to errors on input and output pt devices plus input from
block devices or regular files. Errors on other files will stop ddpt. Default is 0 which implies
stop on any error. See the 'coe' flag for more information.
coe_limit=CL
where CL is the maximum number of consecutive bad blocks stepped over (due to "coe=1") on reads
before the copy terminates. The default is 0 which is interpreted as no limit. This option is
meant to stop the copy soon after unrecorded media is detected while still offering "continue on
error" capability.
conv=CONVS
see the CONVERSIONS section below.
count=COUNT
copy COUNT input blocks from IFILE to OFILE. If this option is not given (or COUNT is '-1') then
the COUNT may be deduced from either IFILE or OFILE. See the COUNT section below.
ibs=IBS
where IBS is the IFILE block size in bytes. The default value is BS or its default (512).
Conflicts the "bs=" option (e.g. giving both "bs=512 ibs=512" is considered a syntax error).
if=IFILE
read from IFILE. This option must be given (i.e. it is mandatory). If IFILE is '-' then stdin is
read. Starts reading at the beginning of IFILE unless SKIP is given.
iflag=FLAGS
where FLAGS is a comma separated list of one or more flags outlined in the FLAGS section below.
These flags are associated with IFILE and are ignored when IFILE is stdin.
obs=OBS
where OBS is the OFILE block size in bytes. The default value is BS or its default (512).
Conflicts the "bs=" option (e.g. giving both "bs=512 obs=512" is considered a syntax error). If
OBS is given then it has the following restriction: the integer expression (((IBS * BPT) % OBS) ==
0) must be true. Stated another way: the copy buffer size must be an integral multiple of OBS. If
of2=OFILE2 is given then OBS is its block size as well.
of=OFILE
write to OFILE. The default value is /dev/null . If OFILE is '-' then writes to stdout. If OFILE
is /dev/null then no actual writes are performed. If OFILE is '.' (period) then it is treated the
same way as /dev/null . If OFILE exists then it is _not_ truncated unless "oflag=trunc" is given.
See section on DD DIFFERENCES.
of2=OFILE2
write output to OFILE2. The default action is not to do this additional write (i.e. when this
option is not given). OFILE2 is assumed to be a regular file or a fifo (i.e. a named pipe). OFILE2
is opened for writing and is created if necessary. If OFILE2 is a fifo (named pipe) then some
other command should be consuming that data (e.g. 'md5sum OFILE2'), otherwise this utility will
block. The write to OFILE2 occurs before the write to OFILE and prior to sparse writing and write
sparing logic. So everything read is written to OFILE2.
oflag=FLAGS
where FLAGS is a comma separated list of one or more flags outlined in the FLAGS section. These
flags are associated with OFILE and are ignored when OFILE is /dev/null, '.' (period), or stdout.
retries=RETR
sometimes retries at the host are useful, for example when there is a transport error. When RETR
is greater than zero then SCSI READs and WRITEs are retried on error, RETR times. Default value is
zero. Only applies to errors on pt devices.
seek=SEEK
start writing SEEK blocks (each of OBS bytes) from the start of OFILE. Default is block 0 (i.e.
start of file). The SEEK value may exceed the number of OBS-sized blocks in OFILE.
skip=SKIP
start reading SKIP blocks (each of IBS bytes) from the start of IFILE. Default is block 0 (i.e.
start of file). The SKIP value must be less than or equal to the number of IBS-sized blocks in
IFILE.
status=STAT
the STAT value of 'noxfer' suppresses the throughput speed and the copy time output at the end of
the copy. The "status=noxfer" option was recently introduced to GNU's dd command. The default
action of ddpt is to show the throughput (in megabytes per second) and the time taken to do the
copy after the "records in" and "records out" lines at the end of the copy. As a convenience the
value 'null' is accepted for STAT and does nothing.
verbose=VERB
as VERB increases so does the amount of debug output sent to stderr. Default value is zero which
yields the minimum amount of debug output. A value of 1 reports extra information that is not
repetitive. A value 2 reports cdbs and responses for SCSI commands that are not repetitive (i.e.
other that READ and WRITE). Error processing is not considered repetitive. Values of 3 and 4 yield
output for all SCSI commands, plus Unix read() and write() calls, so there can be a lot of output.
If VERB is "-1" then output otherwise sent to stderr is redirected to /dev/null .
-h, --help
outputs usage message and exits.
-v, --verbose
equivalent of verbose=1. If --verbose appears twice then that is equivalent to verbose=2. Also -vv
is equivalent to verbose=2.
-V, --version
outputs version number information and exits.
-w, --wscan
this option is available in Windows only. It lists storage device names and the corresponding
volumes, if any. When used twice it adds the "bus type" of the closest transport (e.g. a SATA disk
in a USB connected enclosure has bus type Usb). When used three times a SCSI adapter scan is
added. When used four times only a SCSI adapter scan is shown. See EXAMPLES section below and the
README.win32 file.
COUNT
When the count=COUNT option is not given (or COUNT is '-1') then an attempt is made to deduce COUNT as
follows.
When both or either IFILE and OFILE are block devices, then the minimum size, expressed in units of input
blocks, is used. When both or either IFILE and OFILE are pass-through devices, then the minimum size,
expressed in units of input blocks, is used.
If a regular file is used as input, its size, expressed in units of input blocks (and rounded up if
necessary) is used. Note that the rounding up of the deduced COUNT may result in a partial read of the
last input block and a corresponding partial write to OFILE if it is a regular file.
The size of pt devices is deduced from the SCSI READ CAPACITY command. Block device sizes (or their
partition sizes) are obtained from the operating system, if available.
If skip=SKIP or skip=SEEK are given and the COUNT is deduced (i.e. not explicitly given) then that size
is scaled back so that the copy will not overrun the file or device.
If COUNT is not given and IFILE is a fifo (and stdin is treated as a fifo) then IFILE is read until an
EOF is detected. If COUNT is not given and IFILE is a /dev/zero (or equivalent) then zeros are read
until an error occurs (e.g. file system full).
If COUNT is not given and cannot be deduced then an error message is issued and no copy takes place.
CONVERSIONS
One or more conversions can be given to the "conv=" option. If more than one is given, they should be
comma separated. ddpt does not perform the traditional dd conversions (e.g. ASCII to EBCDIC). Recently
added conversions overlap somewhat with the flags so some conversions are now supported by ddpt.
fdatasync
equivalent to "oflag=fdatasync". Flushes data associated with the OFILE to storage at the end of
the copy. This conversion is for compatibility with GNU's dd.
fsync equivalent to "oflag=fsync". Flushes data and meta-data associated with the OFILE to storage at
the end of the copy. This conversion is for compatibility with GNU's dd.
noerror
this conversion is very close to "iflag=coe" and is treated as such. See the "coe" flag. Note that
an error on OFILE will stop the copy.
null has no affect, just a placeholder.
resume See "resume" in the FLAGS sections for more information.
sparing
See "sparing" in the FLAGS sections for more information.
sparse FreeBSD supports "conv=sparse" so the same syntax is supported in ddpt. See "sparse" in the FLAGS
sections for more information.
sync is ignored by ddpt. With dd it means supply zero fill (rather than skip) and is typically used
like this "conv=noerror,sync" to have the same functionality as ddpt's "iflag=coe".
trunc if OFILE is a regular file then truncate it prior to starting the copy. See "trunc" in the FLAGS
section.
FLAGS
A list of flags and their meanings follow. The flag name is followed by one or two indications in square
brackets. The first indication is either "[i]", "[o]" or "[io]" indicating this flag is active for the
IFILE, OFILE or both the IFILE and the OFILE. The second indication contains some combination of "reg",
"blk" or "pt" indicating whether the flag applies to a regular file, a block device (accessed via Unix
read() and write() commands) or a pass-through device respectively.
append [o] [reg]
causes the O_APPEND flag to be added to the open of OFILE. For regular files this will lead to
data appended to the end of any existing data. Conflicts the seek=SEEK option. The default action
of this utility is to overwrite any existing data from the beginning of OFILE or, if SEEK is
given, starting at block SEEK. Note that attempting to 'append' to a device file (e.g. a disk)
will usually be ignored or may cause an error to be reported.
coe [io] [pt], [i] [reg,blk]
continue on error. 'iflag=coe oflag=coe' and 'coe=1' are equivalent. Errors occurring on output
regular or block files will stop ddpt. Error messages are sent to stderr. This flag is similar to
'conv=noerror,sync' in the dd(1) utility. Unrecovered errors are counted and output in the summary
at the end of the copy.
This paragraph is about coe on pt devices. A medium, hardware or blank check error while reading
will re-read blocks prior to the bad block, then try to recover the bad block, supplying zeros if
that fails, and finally reread the blocks after the bad block. A medium, hardware or blank check
error while writing is noted and ignored. SCSI disks may automatically try and remap faulty
sectors (see the AWRE and ARRE in the read write error recovery mode page (the sdparm utility can
access these attributes)). If bad LBAs are reported by the pass-through then the LBA of the lowest
and highest bad block is also output.
This paragraph is about coe on input regular files and block devices. When a EIO or EREMOTEIO
error is detected on a normal segment read then the segment is re-read one block (i.e. IBS bytes)
at a time. Any block that yields a EIO or EREMOTEIO error is replaced by zeros. Any other error, a
short read or an end of file will terminate the copy, usually after the data that has been read is
written to the output file.
direct [io] [reg,blk]
causes the O_DIRECT flag to be added to the open of IFILE and/or OFILE. This flag requires some
memory alignment on IO. Hence user memory buffers are aligned to the page size. May have no effect
on pt devices. This flag will bypass caching/buffering normally done by block layer. Beware of
data coherency issues if the same locations have been recently accessed via the block layer in its
normal mode (i.e. non-direct). See open(2) man page.
dpo [io] [pt]
set the DPO bit (disable page out) in SCSI READ and WRITE commands. Not supported for 6 byte cdb
variants of READ and WRITE. Indicates that data is unlikely to be required to stay in device (e.g.
disk) cache. May speed media copy and/or cause a media copy to have less impact on other device
users.
errblk [i] [pt] [experimental]
attempts to create or append to a file called "errblk.txt" in the current directory the logical
block addresses of blocks that cannot be read. The first (appended) line is "# start <timestamp>".
That is followed by the LBAs in hex (and prefixed with "0x") of any block that cannot be read, one
LBA per line. If the sense data does not correctly identify the LBA of the first error in the
range it was asked to read then a LBA range is output in the form of the lowest and the highest
LBA in the range separated by a "-". At the end of the copy a line with "# stop <timestamp>" is
appended to "errblk.txt". Typically used with "coe".
excl [io] [reg,blk]
causes the O_EXCL flag to be added to the open of IFILE and/or OFILE. See open(2) man page.
fdatasync [o] [reg,blk]
Flushes data associated with the OFILE to storage at the end of the copy.
flock [io] [reg,blk,pt]
after opening the associated file (i.e. IFILE and/or OFILE) an attempt is made to get an advisory
exclusive lock with the flock() system call. The flock arguments are "FLOCK_EX | FLOCK_NB" which
will cause the lock to be taken if available else a "temporarily unavailable" error is generated.
An exit status of 90 is produced in the latter case and no copy is done. See flock(2) man page.
force [io] [pt]
override difference between given block size and the block size found by the SCSI READ CAPACITY
command. Use the given block size. Without this flag the copy would not be performed. pt access to
what appears to be a block partition is aborted in version 0.92; that can be overridden by the
force flag. For related reasons the 'norcap' flag requires this flag when applied to a block
device accessed via pt.
fsync [o] [reg,blk]
Flushes data and metadata (describing the file) associated with the OFILE to storage at the end of
the copy.
fua [io] [pt]
causes the FUA (force unit access) bit to be set in SCSI READ and/or WRITE commands. The 6 byte
variants of the SCSI READ and WRITE commands do not support the FUA bit.
fua_nv [io] [pt]
causes the FUA_NV (force unit access non-volatile cache) bit to be set in SCSI READ and/or WRITE
commands. This only has an effect with pt devices. The 6 byte variants of the SCSI READ and WRITE
commands do not support the FUA_NV bit.
nocache [io] [reg,blk]
use posix_fadvise() to advise corresponding file there is no need to fill the file buffer with
recently read or written blocks. If used with "iflag=" it will increase the read ahead on IFILE.
norcap [io] [pt]
do not perform SCSI READ CAPACITY command on the corresponding pt device. If used on block device
accessed via pt then 'force' flag is also required. This is to warn about using pt access on what
may be a block device partition.
nowrite [o] [reg,blk,pt]
bypass writes to OFILE. The "records out" count is not incremented. OFILE is still opened but
"oflag=trunc" if given is ignored. Also the ftruncate call associated with the sparse flag is
ignored (i.e. bypassed). Commands such as trim and SCSI SYNCHRONIZE CACHE are still sent.
null [io]
has no affect, just a placeholder.
pt [io] [blk,pt]
causes a device to be accessed in "pt" mode. In "pt" mode SCSI READ and WRITE commands are sent to
access blocks rather than standard UNIX read() and write() commands. The "pt" mode may be implicit
if the device is only capable of passing through SCSI commands (e.g. the /dev/sg devices in
Linux). This flag is needed for device nodes that can be accessed both via standard UNIX read()
and write() commands as well as SCSI commands. Such devices default standard UNIX read() and
write() commands in the absence of this flag.
resume [o] [reg]
when a copy is interrupted (e.g. with Control-C from the keyboard) then using the same invocation
again with the addition of "oflag=resume" will attempt to restart the copy from the point of the
interrupt (or just before that point). It is harmless to use "oflag=resume" when OFILE doesn't
exist or is zero length. If the length of OFILE is greater than or equal to the length implied by
a ddpt invocation that includes "oflag=resume" then no further data is copied.
self [io] [pt]
used together with trim flag to do a self trim (trim of segments of a pt device that contain all
zeros). If OFILE is not given, then it is set to the same as IFILE. If SEEK is not given it set to
the same value as SKIP (possibly adjusted if IBS and OBS are different). Implicitly sets "nowrite"
flag.
sparing [o] [reg,blk,pt]
during the copy each IBS * BPT byte segment is read from IFILE into a buffer. Then, instead of
writing that buffer to OFILE, the corresponding segment is read from OFILE into another buffer. If
the two buffers are different, the former buffer is written to the OFILE. If the two buffers
compare equal then the write to OFILE is not performed. Write sparing is useful when a write
operation is significantly slower than a read. Under some conditions flash memory devices have
slow writes plus an upper limit on the number of times the same cell can be rewritten. The
granularity of the comparison can be reduced from the default IBS * BPT byte segment with the the
OBPC value given to the "bpt=" option. The finest granularity is when OBPC is 1 which implies OBS
bytes.
sparse [o] [reg,blk,pt]
after each IBS * BPT byte segment is read from IFILE, it is checked to see if it is all zeros. If
so, that segment is not written to OFILE. See the section on SPARSE WRITES below. The granularity
of the zero comparison can be reduced from the default IBS * BPT byte segment with the OBPC value
given to the "bpt=" option.
ssync [o] [pt]
if OFILE is in "pt" mode then the SCSI SYNCHRONIZE CACHE command is sent to OFILE at the end of
the copy.
strunc [o] [reg]
perform a sparse copy with a ftruncate system call to extend the length of the OFILE if required.
See the sparse flag and the section on SPARSE WRITES below.
sync [io] [reg,blk]
causes the O_SYNC flag to be added to the open of IFILE and/or OFILE. See open(2) man page.
trim [io] [pt] [experimental]
similar logic to the "sparse" option. However instead of skipping segments that are full of zeros
a "trim" command is sent to OFILE. Usually set as an oflag argument but for self trim can be used
as an iflag argument (e.g. "iflag=self,trim"). Depending on the usage this may require the device
to support "deterministic read zero after trim". See the TRIM, UNMAP AND WRITE SAME section below.
trunc [o] [reg]
if OFILE is a regular file then it is truncated prior to starting the copy. If SEEK is not given
or 0 then OFILE is truncated to zero length; when SEEK is larger than zero the truncation takes
place at file byte pointer SEEK*OBS. Ignored if "oflag=append". Conflicts with "oflag=sparing".
SPARSE WRITES
Bypassing writes of blocks full of zeros can save a lot of IO. However with regular files, bypassed
writes at the end of the copy can lead to an OFILE which is shorter than it would have been without
sparse writes. This can lead to integrity checking programs like md5sum and sha1sum generating different
values.
This utility has two ways of handling this file length problem: writing the last block (even if it is
full of zeros) or using the ftruncate system call. A third approach is to ignore the problem (i.e.
leaving OFILE shorter). The ftruncate approach is used when "oflag=strunc" while the last block is
written when "oflag=sparse". To ignore the file length issue use "oflag=sparse,sparse". Note that if
OFILE's length is already correct or longer than required, no action is taken.
The support for sparse writing of regular files may depend on the OS, the file system and the settings of
OFILE. POSIX makes few guarantees when the ftruncate system call is used to extend a file's length, as
may occur when "oflag=strunc". Further, primitive file systems like VFAT may not accept sparse writes or
simulate the effect by writing blocks of zeros. The latter approach will defeat any sparse writing
performance gain.
TRIM, UNMAP AND WRITE SAME
This is a new storage feature often associated with Solid State Disks (SSDs) or disk arrays with "thin
provisioning". In the ATA command set (ACS-2) the relevant command is DATA SET MANAGEMENT with the TRIM
bit set. In the SCSI command set (SBC-3) it is either the UNMAP or WRITE SAME command. Note there is no
TRIM command however the term is frequently used in the technical press.
Trim is a way of telling a storage device that blocks are no longer needed. Keeping the pool of
unwritten blocks large is important for the write performance of SSDs and the thrifty use of real storage
in thin provisioned arrays. Currently file systems in recent OSes may issue trims associated with file
deletes. The trim option in ddpt may be useful when a partition or a whole SSD is to be "deleted". Note
that ddpt is bypassing file systems in that it only offers trim on pass-through (pt) devices.
This utility issues SCSI commands to pt devices and for "trim" currently issues a SCSI WRITE SAME(16)
command with the UNMAP bit set. If the pt device is a SSD with a ATA interface then recent versions of
Linux will translate the SCSI WRITE SAME to the ATA DATA SET MANAGEMENT command with the TRIM bit set.
The maximum size of each "trim" command sent is the size of the copy buffer (i.e. IBS * BPT bytes). And
that maximum can be reduced with the OBPC argument of the "bpt=" option.
The trim can be used various ways. One way is a copy where the copy buffer (or some part of it) is
checked for zeros as is done by the sparse oflag. When a zero segment is found, a trim "command" is sent
to the OFILE. For example:
ddpt if=dsk.img bs=512 of=/dev/sdc oflag=pt,trim
The copy buffer is 64 KiB (since BPT and OBPC default to 128 when "bs=512") and it is checked for all
zeros. If it is all zeros then a trim command is sent to the corresponding location of /dev/sdc which is
accessed via the pt interface. If it is not all zeros then a SCSI WRITE command is sent. Another way is
to trim all or part of a disk. To trim a whole disk (i.e. deleting all its data):
ddpt if=/dev/zero bs=512 of=/dev/sdc oflag=pt,trim
A third way is to "self-trim" which is to only trim those parts of a disk that contain segments full of
zeros:
ddpt if=/dev/sdc skip=0x2300 bs=512 iflag=pt,self,trim count=0x1234f0
The "self" oflag automatically sets up the output side of the copy to send trim commands (if required)
back the the same device (i.e. /dev/sdc). If this example was self-trimming a partition then the
partition would start at LBA 0x2300 and be 0x1234f0 blocks long.
Some random product examples: the Intel X25-M G2 SSDs have trim with recent firmware and they do
deterministic read zero after trim. The Seagate Pulsar SSD has an ATA interface which supports the
deterministic reads of zero after the DATA SET MANAGEMENT command with the TRIM option.
DD DIFFERENCES
dd defaults "if=" and "of=" to stdin and stdout respectively. This follows Unix filter conventions.
However since dd and ddpt are often used to read binary data for timing purposes, having to supply
"of=/dev/null" can be easily forgotten. Without it dd will potentially spew binary data on the console.
So ddpt has changed its defaults: the "if=IFILE" is now mandatory and to read from stdin "if=-" can be
used; "of=OFILE" remains optional but its default changes to "/dev/null" (or "NUL" in Windows). To send
output to stdout ddpt accepts "of=-".
dd truncates OFILE unless "conv=notrunc" is given. When dd truncates, it truncates to zero length unless
SEEK is greater than zero. ddpt does not truncate OFILE by default. If OFILE exists it will be
overwritten. The overwrite starts at block zero unless SEEK or "oflag=append" is given. If OFILE is a
regular file then "oflag=trunc" (or "conv=trunc") will truncate OFILE prior to the copy.
Numeric arguments to ddpt can be given in hexadecimal, either with a leading "0x" or "0X" or with a
trailing "h". Note that dd accepts "0x123" but interprets it as "0 * 123" (i.e. zero). ddpt will also
interpret "x" as multiplies unless the left operand is zero (e.g. "0x123"). So both dd and ddpt will
interpret "skip=2x123" as "skip=246".
Terabyte size disks make it impractical to copy all the data into a buffer before writing it out.
Therefore both dd and ddpt read a relatively small amount of data into a copy (or transfer) buffer then
write it out to the destination, repeating this process until the COUNT is exhausted.
A major difference in ddpt is the addition of BPT to control the size of the copy buffer. With dd, IBS is
the size of the copy buffer and the unit of SKIP and COUNT. With ddpt, IBS * BPT is the size of the copy
buffer and IBS is the unit of SKIP and COUNT. This allows ddpt to have its IBS set to the logical block
size of IFILE without unduly restricting the size of the copy buffer. And setting IBS (and OBS for OFILE)
accurately is required when the pass-through interface is used since with the SCSI READ and WRITE
commands the logical block size is implicit.
The way dd handles its copy buffer (outlined in SUSv4 description of dd) is relatively complex,
especially when IBS and OBS are different sizes. The restriction that ddpt places on IBS and OBS ( i.e.
(((IBS*BPT) % OBS) == 0) ) means that a single copy buffer can be used since its size is a multiple of
both IBS and OBS. Being able to precisely define the copy buffer size in ddpt makes sparse writing, write
sparing and trim operations simpler to define and the user to control.
ddpt does not support dd's "cbs=" option (conversion block size). If the "cbs=" option is given to ddpt
then it is ignored.
NOTES
A partial write is a write to the OFILE of less than OBS bytes. This typically occurs at the end of a
copy. dd can do partial writes. ddpt does partial writes to regular files and fifos (including stdout).
However ddpt ignores partial writes when OFILE is a block device or a pt device. When ddpt ignores a
partial write, it sends a warning to the console (stderr).
At the end of the copy two lines are output to the console:
<in_full>+<in_partial> records in
<out_full>+<out_partial> records out
The "records in" line is the number of full input blocks (each of IBS bytes) that have been read plus the
number of partial blocks ( usually less than IBS bytes) that have been read. Following the lead of dd
when 'iflag=coe' is active a block that cannot be read (and has zeros substituted for its output) is
regarded as a partial read. The "records out" line is the number of full output blocks (each of OBS
bytes) that have been written plus the number of partial blocks (usually less than OBS bytes) that have
been written.
Block devices (e.g. /dev/sda and /dev/hda) can be given for IFILE. If neither 'iflag=direct' nor
'iflag=pt' is given then normal block IO involving buffering and caching is performed. If 'iflag=direct'
is given then the buffering and caching is bypassed (this is applicable to both SCSI devices and ATA
disks). When 'iflag=pt' is given SCSI commands are sent to the device which bypasses most of the actions
performed by the block layer. The same applies for block devices given for OFILE.
BPT, BS, COUNT, IBS, OBPC, OBS, SKIP and SEEK may include one of these multiplicative suffixes: c C *1; w
W *2; b B *512; k K KiB *1,024; KB *1,000; m M MiB *1,048,576; MB *1,000,000 . This pattern continues for
"G", "T" and "P". The latter two suffixes can only be used for COUNT, SKIP and SEEK. Also a suffix of
the form "x<n>" multiplies the leading number by <n>; however the combinations "0x" and "0X" are treated
differently, see the next paragraph. These multiplicative suffixes are compatible with GNU's dd command
(since 2002) which claims compliance with the SI and with IEC 60027-2 standards.
Alternatively numerical values can be given in hexadecimal preceded by either "0x" or "0X" (or with a
trailing "h" or "H"). When hex numbers are given, multipliers cannot be used.
The COUNT, SKIP and SEEK arguments can take 64 bit values (i.e. very big numbers). Other numerical values
are limited to what can fit in a signed 32 bit number.
All informative, warning and error output is sent to stderr so that dd's output file can be stdout and
remain unpolluted. If no options are given, then the usage message is output and nothing else happens.
Disk partition information can often be found with fdisk(8) [the "-ul" argument is useful in this
respect]. Also parted(8) can be used like this: 'parted /dev/sda unit s print' .
For pt devices this utility issues SCSI READ and WRITE (SBC) commands which are appropriate for disks and
reading from CD/DVD/BD drives. Those commands are not formatted correctly for tape devices so ddpt should
not be used on tape devices. If the largest block address of the requested transfer exceeds a 32 bit
block number (i.e 0xffffffff) then a warning is issued and the sg device is accessed via SCSI READ(16)
and WRITE(16) commands.
The attributes of a block device (e.g. partitions) are ignored when the pt flag is used. Hence the whole
device is read (rather than just the second partition) by this invocation:
ddpt if=/dev/sdb2 iflag=pt of=t bs=512
Assuming /dev/sdb and /dev/sg2 refer to the same device, then after the following two invocations, the
contents of the files "t", "tt" and "ttt" should be same:
ddpt if=/dev/sdb of=tt bs=512
ddpt if=/dev/sg2 of=ttt bs=512
EXAMPLES
The examples in this page use Linux device names. For suitable device names in other supported Operating
Systems see this web page: http://sg.danny.cz/sg/device_name.html . The sg3_utils(8) man page in the
sg3_utils package also covers device naming.
ddpt usage looks quite similar to dd:
ddpt if=/dev/sg0 of=t bs=512 count=1MB
This will copy 1 million 512 byte blocks from the device associated with /dev/sg0 (which should have 512
byte blocks) to a file called t. Assuming /dev/sda and /dev/sg0 are the same device then the above is
equivalent to:
dd if=/dev/sda iflag=direct of=t bs=512 count=1000000
although dd's speed may improve if bs was larger and count was suitably reduced. The use of the
'iflag=direct' option bypasses the buffering and caching that is usually done on a block device.
The dd command's bs argument can be thought of as roughly equivalent to ddpt's bs*bpt . dd almost assumes
buffering on a block device and will work as long as bs is a multiple of the actual logical block size.
Since ddpt can work at a lower level in some cases the bs argument must be a disk's actual logical block
size. Thus the bpt argument was introduced to make the copy more efficient. So these two invocations are
roughly equivalent:
dd if=/dev/sda of=t bs=8k count=64
ddpt if=/dev/sda of=t bs=512 bpt=16 count=1k
In both cases the total number of bytes moved is bs*count . And that will be done by reading 8k (8192
bytes) into a buffer then writing out that buffer to the file t. The read write sequence continues until
the count is complete or an error occurs.
The 'of2=' option can save time when the input would otherwise need to be read twice. For example, to
copy data and take a md5sum of it without needing to re-read the data:
mkfifo fif
md5sum fif &
ddpt if=/dev/sg3 iflag=coe of=sg3.img oflag=sparse of2=fif bs=512
This will image /dev/sg3 (e.g. an unmounted disk) and place the contents in the (sparse) file sg3.img .
Without re-reading the data it will also perform a md5sum calculation on the image.
Now we use sparse writing logic to get some idea of how many blocks on a disk are full of zeros. After a
SCSI FORMAT or an ATA SECURITY ERASE command a disk may be all zeros.
ddpt if=/dev/sdc bs=512 oflag=sparse
Since no "of=" option is given, output goes to /dev/null so nothing is actually written so the "records
out" will be zero. However there will be a count of "records in" and "bypassed records out". If /dev/sdc
is full of zeros then "records in" and "bypassed records out" will be the same. Since the "bpt=" option
is not given it defaults to "bpt=128,128" so the copy buffer will be 64 KiB and the sparse check for
zeros will be done with 64 KiB (128 block) granularity.
For examples of the trim and self,trim options see the section above on TRIM, UNMAP AND WRITE SAME.
Following is an example run on a Windows OS using the '--wscan' option which shows the available device
names (e.g. PD1) and the associated volume name(s):
ddpt -w
PD0 [C] FUJITSU MHY2160BH 0000
PD1 [DF] WD 2500BEV External 1.05 WD-WXE90
CDROM0 [E] MATSHITA DVD/CDRW UJDA775 CB03
So, for example, volumes D: and F: reside on PhysicalDisk1 (abbreviated to "PD1") which is manufactured
by WD (Western Digital).
Further examples can be found on this web page: http://sg.danny.cz/sg/ddpt.html . There is a text file
called ddpt_examples.txt in the "doc" directory of this package's distribution tarball.
SIGNALS
The signal handling has been borrowed from dd: SIGINT, SIGQUIT and SIGPIPE output the number of remaining
blocks to be transferred and the records in + out counts; then they have their default action. SIGUSR1
causes the same information to be output and the copy continues. All output caused by signals is sent to
stderr.
EXIT STATUS
To aid scripts that call ddpt, the exit status is set to indicate success (0) or failure (1 or more).
Note that some of the lower values correspond to the SCSI sense key values. The exit status values are:
0 success
1 syntax error. Either illegal command line options, options with bad arguments or a combination of
options that is not permitted.
2 the device reports that it is not ready for the operation requested. The device may be in the
process of becoming ready (e.g. spinning up but not at speed) so the utility may work after a
wait.
3 the device reports a medium or hardware error (or a blank check). For example an attempt to read a
corrupted block on a disk will yield this value.
5 the device reports an "illegal request" with an additional sense code other than "invalid
operation code". This is often a supported command with a field set requesting an unsupported
capability.
6 the device reports a "unit attention" condition. This usually indicates that something unrelated
to the requested command has occurred (e.g. a device reset) potentially before the current SCSI
command was sent. The requested command has not been executed by the device. Note that unit
attention conditions are usually only reported once by a device.
9 the device reports an illegal request with an additional sense code of "invalid operation code"
which means that it doesn't support the requested command.
11 the device reports an aborted command. In some cases aborted commands can be retried immediately
(e.g. if the transport aborted the command due to congestion).
15 the utility is unable to open, close or use the given IFILE or OFILE. The given file name could be
incorrect or there may be permission problems. Adding the -v option may give more information.
20 the device reports it has a check condition but "no sense". It is unlikely that this value will
occur as an exit status.
21 the device reports a "recovered error". The requested command was successful. Most likely a
utility will report a recovered error to stderr and continue, probably leaving the utility with an
exit status of 0 .
33 the command sent to device has timed out. This occurs in Linux only; in other ports a command
timeout will appear as a transport (or OS) error.
90 the flock flag has been given on a device and some other process holds the advisory exclusive
lock.
97 the response to a SCSI command failed sanity checks.
98 the device reports it has a check condition but the error doesn't fit into any of the above
categories.
99 any errors that can't be categorized into values 1 to 98 may yield this value. This includes
transport and operating system errors after the command has been sent to the device.
AUTHORS
Written by Doug Gilbert
REPORTING BUGS
Report bugs to <dgilbert at interlog dot com>.
COPYRIGHT
Copyright © 2008-2011 Douglas Gilbert
This software is distributed under the GPL version 2. There is NO warranty; not even for MERCHANTABILITY
or FITNESS FOR A PARTICULAR PURPOSE.
SEE ALSO
There is a web page discussing ddpt at http://sg.danny.cz/sg/ddpt.html
The lmbench package contains lmdd which is also interesting. For moving data to and from tapes see dt
which is found at http://www.scsifaq.org/RMiller_Tools/index.html
To change mode parameters that effect a SCSI device's caching and error recovery see sdparm(sdparm)
To scan and repair disk partitions see TestDisk (testdisk).
Additional references: dd(1), ddrescue(GNU), open(2), flock(2), sg_dd,sg3_utils(sg3_utils)
ddpt-0.92 February 2011 DDPT(8)