Provided by: ddpt_0.97-1_amd64 
NAME
ddpt_sgl - helper for ddpt utility's scatter gather lists
SYNOPSIS
ddpt_sgl [--action=ACT] [--a-sgl=SGL] [--b-sgl=SGL] [--chs=CHS] [--degen=DV] [--document]
[--elem=SE[,LE]] [--extension=FNE] [--flexible] [--help] [--hex] [--iaf=IAF] [--index=IA]
[--interleave=IL] [--non-overlap] [--out=O_SGL] [--quiet] [--round=RB] [--sort-cmp=SC]
[--stats] [--verbose] [--version]
DESCRIPTION
This utility is a scatter gather list helper for the ddpt utility which copies data
between or within SCSI devices (logical units). While ddpt's command line syntax is
modelled on that of the POSIX dd command, this utility has a more standard Unix command
line syntax with both short and long variants of each option.
Scatter gather lists (sgl_s) are made up of scatter gather elements. Each element is made
up a starting logical block address (LBA) and a number of blocks (NUM) from and including
that LBA.
The scatter gather lists can also be viewed as arrays in which elements can be accessed by
an index. Multiple sgl elements can be accessed with an array of indexes, hence index
arrays. Indexes in this utility start at 0 and run to (n - 1) where n is the number of
elements in the sgl. Also negative indexes are permitted where -1 is the index of the last
sgl element, -2 is the index of the second last sgl element, etc.
For "twin" actions there is an assumed relationship between a-sgl and b-sgl as there is
between two sgl_s used as the gather list (e.g. skip=) and the scatter list (e.g. seek=)
in the ddpt utility. Breaking it down to individual logical blocks: LBAr0 is read and its
data is written to LBAw0, LBAr1-->LBAw1, LBAr2-->LBAw2, etc; or more generally
LBAr_n-->LNAw_n. Many actions will change the order in which those "read-write" items are
performed, the twin portion of the action attempts to maintain the LBAr_n-->LNAw_n
mapping. Generally speaking, copies are the same no matter what order the LBAs are read
and written. One exception is an overlapping scatter list (i.e. on the write side) in
which case the order of writes onto the same LBA does matter, hence there is an option to
check sgl_s are well-formed in that respect: --non-overlap.
For background on scatter gather lists see the section of that name in the ddpt(8) manpage
found in this package. There is a web page at https://sg.danny.cz/sg/ddpt.html .
OPTIONS
Arguments to long options are mandatory for short options as well.
-a, --action=ACT
ACT is some action to perform on the given scatter gather list(s). To list the
available actions set ACT to 'xxx' (or 'enum'). The available actions are listed in
the ACTIONS section below.
-A, --a-sgl=SGL
SGL is a scatter gather list, a sequence of comma separated unsigned integers (up
to 64 bits each). SGL has several forms, the simplest is:
LBA0,NUM0,LBA1,NUM1,LBA2,NUM2... and there should be an even number of values with
the exception of LBA0 appearing by itself. In this case NUM0 is assumed to be 0.
Other SGL forms are '@<filename>' and 'H@<filename>' where the contents of the
<filename> is parsed as a scatter gather list. Since there are two options for
inputting SGLs, this one is termed as the 'a-sgl'.
See the section on FILE FORMATS below and the section on SCATTER GATHER LISTS in
the ddpt(8) manpage for more information on sgl_s and their associated terminology.
-B, --b-sgl=SGL
SGL is a scatter gather list, a second list termed as the 'b-sgl' to differentiate
it from the other sgl (a-sgl).
-C, --chs=CHS
CHS is a 3 element tuple, separated by commas. Currently 16 bit values from 1 to
0xffff are accepted (the cylinder can be one more: 0x10000 (or 65536)). The first
value is the number of cylinders, the second value is the number of heads (limited
to 16), and the final value is the number of sectors per track (limited to 255).
Sectors are counted origin 1 according to CHS conventions (cf. normal LBAs which
nearly always count from 0).
-D, --degen=DV
DV of 0 (the default) means that all degenerate elements (apart from the last) are
ignored (and dropped from the internal representation which may later be written to
an output file). If DV is odd then a degenerate element's LBA is taken into account
when calculating the highest and lowest LBA in a sgl (and may be included in an
output file). If DV is even (apart from a DV of 0) then a degenerate element's LBA
it taken into account when determining if a sgl is monotonic increasing, decreasing
or neither (and may be included in an output file).
-d, --document
this option causes information about the a sgl or index array to be written as
comments (i.e. lines starting with '#') to the beginning of output file(s) created
by this utility.
If this option is given twice then the command line that caused the output is added
to the file as a comment (before any numbers are output).
-E, --elem=SE[,LE]
this option allows a single sgl element (at position SE (starting element index))
to be output to O_SGL or O_SGL.FNE (or IAF). SE is origin 0. If LE (last element
index) is given then a range of sgl elements are output starting at index SE to
index LE inclusive. If a "twin" operation is being performed then this option only
applies to the "a" side output, not the "twin" side. This option is ignored by the
output of the split_n and tsplit_n actions.
Negative values for either SE or LE count from the end of sgl. For example
--elem=0,-1 refers to the whole of the list.
If LE is less than SE (after any negative indexes are converted to their equivalent
positive index) then that range is output in reverse.
-e, --extension=FNE
FNE is the filename extension used when output filenames are generated. For
non-split actions the generated filenames are of the form: O_SGL.FNE . For the
split_n action the generated filenames are of the form: O_SGL[1..n].FNE . For the
tsplit_n action the a-sg is named as per the previous sentence, while for the b-sgl
the generated filenames are of the form: O_SGL[1..n]_t.FNE .
If O_SGL is '-' (by itself) then all output is sent to stdout and this option is
ignored.
-f, --flexible
this option effects the parsing (reading) of sgl_s and index arrays that are in
files which are in hexadecimal. Such files should have a leading line (i.e. before
any numbers) with 'HEX' on it. Without this option any such file must be invoked
with 'H@' before the filename; in other words the 'H' in the invocation needs to
match the 'HEX' in the file. With this option a file can be invoked with '@' and if
a line with 'HEX' is parsed before any numbers then it switches to hexadecimal
mode; so that all the parsed numbers are assumed to be in hexadecimal.
-h, --help
outputs the usage message summarizing command line options then exits.
-H, --hex
used to define the numeric format of sgl and index array elements written to output
(often a file named O_SGL or stdout). If not given then only decimal values are
written to output. If this option is given once then hexadecimal values, prefixed
with '0x', are written. If this option is given twice then a line with the string
'HEX' is written to output, before any values, and those values are implicitly
hexadecimal (i.e. no leading '0x' nor 'h' suffix).
-I, --iaf=IAF
where IAF is a filename (or '-' for stdout) to write an index array to. The only
action that generates an index array currently is --action=sort (and tsort). This
option can be together used with, or in place of, the --out=O_SGL option.
The --document, --elem=SE[,LE] and --hex options effect what is written. See the
section on FILE FORMATS below.
-x, --index=IA
where IA is one or more indexes, comma separated or, if prefixed by "@" or "H@", a
filename containing a list of indexes. These indexes are used by the
--action=select and --action=tselect to select elements from the 'a-sgl'. Positive
and negative indexes that are too large (in absolute terms) are ignored and create
noise if the --verbose option is given. See the section on FILE FORMATS below.
-i, --interleave=IL
IL is an integer, starting from 0. When IL is 0 (the default) there is no
interleave. The interleave only effects the split_n and tsplit_n actions and when
greater than zero is the maximum number of logical blocks written in each segment
in the output file, prior to moving to the next output file.
For the case where IL is 1 and --action=split_1 is given then the output file will
have every LBA (given by the a-sgl) as a separate sgl element (and thus each will
have a NUM of 1).
For the tsplit_n action the interleave is only applied to the a-sgl but it does
effect the twin sgl files.
-N, --non-overlap
Checks any given sgl and any resulting sgl (from an action) to see if any portion
of the sgl overlaps. This is done by first sorting each sgl by the LBA field, then
checking every element against the previous one to determine if there is overlap.
SCSI commands that accept sgl_s process degenerate elements without error but if
two elements in a WRITE command overlap then it is the storage device's choice
which one to WRITE first. The last one to be written will be the one read in
subsequent read operations.
If no errors are detected then if (all) are non-overlapping then 0 is returned. If
no errors are detected then if (any) are overlapping then 36 is returned.
-o, --out=O_SGL
O_SGL is the name of a file to write a resultant scatter gather list to. If O_SGL
is '-' then the output is directed to stdout. If O_SGL starts with '+' then the
output is appended to the file whose name follows the '+'.
For the split and tsplit actions, the leading '+' is interpreted as appended to all
files that meet the template and exist, otherwise the file is created. If '-' is
given then all output is directed to stdout (and the --extension=FNE option, if
given, is ignored).
-q, --quiet
suppresses warning and messages announcing an action has succeeded. When this
option is given, actions that have a logical (boolean) result don't output messages
but still yield an indicative exit status. The exit status will typically be either
0 for true or 36 for false. are typically sent to stderr.
-r, --round=RB
RB is the number of round blocks. Without the option the split_n action will divide
the number of blocks to be split by '<n>' (or use IL) to get a nominal value. This
value is the number of blocks taken from the a-sgl before moving to the next output
file. The RB value (default 0) is the maximum number of blocks the nominal value
may be changed by to align with an existing element boundary in the a-sgl.
If the number of blocks in 'a-sgl is less than 10 or RB is greater than one third
of the nominal value, then RB is ignored (with a notification written to stderr).
For the tsplit_n action this option only applies to the a-sgl.
-S, --sort-cmp=SC
where SC is a value indicating what the sort action's comparison will be. When SC
is 0 (the default) the sort is ascending based on the LBA; when it is 1 the sort is
descending based on LBA. When SC is 2 the sort is ascending based on NUM; when it
is 3 the sort is descending based on NUM. Any other value is mapped to 0. All sorts
are stable which means that sgl elements with the same LBA (in the case of SC being
0 or 1) keep their same relative position. A side effect of this is that the
ascending and descending sorts are not always reversals of one another.
-s, --stats
print out sgl statistics on any given sgl and any resultant sgl.
-v, --verbose
increase the level of verbosity, (i.e. debug output).
-V, --version
print the version string and then exit.
ACTIONS
Actions are given on the command line as part of the --action=ACT option. Currently only
one action is allowed per invocation. If more are allowed in the future, they will be
comma separated and performed in the order in which they appear (i.e. left to right).
If no action is given and the --a-sgl=SGL and --out=O_SGL options (with no --b-sgl=SGL
option) are given then the a-sgl is copied to O_SGL (or O_SGL.FNE if the --extension=FNE
option is given).
The actions are listed below in alphabetical order.
append-b2a
appends the b-sgl to the end of the a-sgl and outputs the result to O_SGL (or
O_SGL.FNE if the --extension=FNE option is given). Requires the --a-sgl=SGL,
--b-sgl=SGL and --out=O_SGL options.
divisible<n>[,L|N] or divisible_<n>[,L|N]
where <n> is an integer, 1 or higher. This action checks if each LBA and NUM in
a-sgl is divisible by <n> (where 'is divisible' is equivalent to having a remainder
of zero). If all are divisible then true is returned (i.e. the exit status 0);
otherwise false is returned (i.e. exit status 36).
If the optional ",L" suffix (or ",LBA") is given then only each LBA element in
a-sgl is checked for divisibility. If the optional ",N" suffix (or ",NUM") then
only each NUM element in a-sgl is checked for divisibility.
The output of the string to stderr announcing divisibility, or lack of it, can be
suppressed by the --quiet option.
enum prints out the list of supported actions then exits. Giving the action 'xxx' has
the same effect.
equal this action compares the sgl_s given to --a-sgl=SGL and --b-sgl=SGL. If the same
LBAs are in the same order with the same overall number of blocks (but not
necessarily the same number of elements) then true is returned (i.e. the exit
status 0); otherwise false is returned (i.e. exit status 36). For example the two
element sgl "0x10,0x5, 0x15,0x2" is 'equal' to the one element sgl "0x10, 0x7".
The output of the string to stderr announcing equality, or lack of it, can be
suppressed by the --quiet option.
none this action does nothing. This is the default action. If --a-sgl=SGL and
--out=O_SGL options are given and no other action, then a-sgl is copied to O_SGL.
It is a placeholder.
part-equal
this action is similar to the equal action but relaxes the condition that both
lists must have the same overall number of blocks. For example the two element sgl
"0x10,0x5, 0x15,0x2" is 'part-equal' to the one element sgl "0x10, 0x12".
part-same
this action is similar to the same action but relaxes the condition that both lists
must have the same overall number of blocks. For example the two element sgl
"0x15,0x2,0x10,0x5" is 'part-same' as the one element sgl "0x10, 0x12".
same this action is similar to the equal action but relaxes the condition that both
lists must be in the same order. The implementation sorts both given lists before
comparing them. For example the two element sgl "0x15,0x2, 0x10,0x5" is the 'same'
as the one element sgl "0x10, 0x7".
scale<n> or scale_<n>
where <n> is an integer, positive or negative but not zero. When <n> is positive
then the starting LBA and the NUM in each a-sgl element is multiplied by <n> . The
new (scaled) sgl is written to O_SGL (or O_SGL.FNE if the --extension=FNE option is
given).
When <n> is negative then the absolute value of <n> is used as a divisor for each
starting LBA and NUM in each a-sgl element.
As an example: converting a 512 byte logical block (LB) size sgl to a 4096 byte LB
size and vice versa is relatively common. To convert from 4096 --> 512 byte LB size
then --action=scale_8 is appropriate. To convert from 512 --> 4096 byte LB size
then --action=scale_-8 is appropriate.
Note: because an integer division is used (that rounds 'towards zero') when <n> is
negative then LBs or NUMs may be "lost" in this conversion. This can be checked
beforehand with the --action=divisible<n>[,L|N] option. For example: for 512 -->
4096 conversions: --action=divisible_8 will report if any starting LBAs or NUMs are
not divisible be 8 and hence are not able to be precisely represented as 4096 byte
LB addresses or number of 4096 byte blocks.
select this action can be used to select a subset (or superset) of the a-sgl in the
specified order. Alternatively it can be seen as re-ordering the elements in a-sgl
such as is done toward the end of a sort operation. Assuming all the indexes in IA
are valid, then the O_SGL file will have the same number of elements as there are
indexes in IA.
This option requires non-empty --a-sgl=SGL and --index=IA options, plus the
--out=O_SGL option.
sort this action will sort the sgl given by --a-sgl=SGL in ascending order by LBA. The
resulting sgl is output to O_SGL (or O_SGL.FNE if the --extension=FNE option is
given).
The sort is "stable", so if two elements have the same starting LBA then they will
appear in the same relative order in the output.
split<n> or split_<n>
where <n> is an integer, 1 or higher. This action divides --a-sgl=SGL into <n>
roughly equal length (i.e. number of blocks) output sgl_s. The output files are
named "O_SGL<1..n>" or "O_SGL<1..n>.FNE". Both the --interleave=IL and --round=RB
options are taken into account during the split process.
to-chs this action takes the 'flat' LBA SGL given to --a-sgl=SGL and converts it into CHS
(cylinder/head/sector) based SGL which is written out as directed to --out=O_SGL.
This action requires the --chs=CHS option as well as the --a-sgl=SGL and
--out=O_SGL options.
tselect
this is a "twin select" action that selects from --a-sgl=SGL (a-sgl) then re-orders
--b-sgl=SGL (b-sgl) in unison. The select from a-sgl is the same as described under
the select action above. Additionally b-sgl is is broken up so it has "breaks" at
the same positions (i.e. number of blocks from the start of the sgl) as a-sgl does;
plus the "breaks" b-sgl has already got. So the "broken up" b-sgl will have at
least as many elements as a-sgl. The output of the re-ordered b-sgl is then written
to O_SGL_t or O_SGL_t.FNE if the --extension=FNE option is given.
tsort this is a "twin sort" action that sorts --a-sgl=SGL (a-sgl) and re-orders
--b-sgl=SGL (b-sgl) in unison. The sort of a-sgl is the same as described under the
sort action above. Additionally b-sgl is is broken up so it has "breaks" at the
same positions (i.e. number of blocks from the start of the sgl) as a-sgl does;
plus the "breaks" b-sgl has already got. So the "broken up" b-sgl will have at
least as many elements as a-sgl. The re-ordering vector generated by the stable
sort of a-sgl is then applied to the broken up b-sgl. The output of the re-ordered
b-sgl is then written to O_SGL_t or O_SGL_t.FNE if the --extension=FNE option is
given.
tsplit<n> or tsplit_<n>
this is a "twin split" action that splits the --a-sgl=SGL and --b-sgl=SGL into
separate series of output files. These separate series maintain the LBA to LBA
correspondence of the original a_sgl and b_sgl lists. <n> is an integer, 1 or
higher. This action divides --a-sgl=SGL into <n> roughly equal length (i.e. number
of blocks) output sgl_s. The "roughly equal length" is influenced by the
--interleave=IL and --round=RB options. The output filenames are generated the same
way as described for the split action. The sgl from --a-sgl=SGL is expected to be a
"hard" sgl which means its last element should not be degenerate (i.e. have a NUM
of 0).
The second half of the "twin split" is to split the --b-sgl=SGL sgl. The same
number of output files are used as for the 'A' side but the filenames have a
slightly different form: "O_SGL<1..n>_t" or "O_SGL<1..n>_t.FNE" (if the
--extension=FNE option is given). The critical point of this split is that it moves
in lockstep with the 'A' side split in the sense that whatever block count an 'A'
side segment uses, the following 'B' side segment split uses the same block count.
The sgl from --b-sgl=SGL may be a "hard" or "soft" sgl. In the simplest case the
'B' side sgl can be just '0' which gets expanded to '0,0' (i.e. degenerate list
starting at LBA 0); this will use the overall block count from the 'A' side.
FILE FORMATS
Both sgl_s and index arrays can be read from, or written to, files. The options that
supply sgl_s or index arrays to be read (e.g. --a-sgl=SGL, --b-sgl=SGL and --index=IA) by
default allow them to be given directly on the command line. These will typically be comma
separated lists (although space and tab could be used as separators if they were
appropriately escaped). So with these options when reading sgl_s and index arrays, a
leading "@" or "H@" is needed to indicate that a file name follows.
By default, numbers given in this utility and other utilities in this package are assumed
to be in decimal. Hexadecimal (hex) numbers can be given with either a leading "0x" or
trailing "h". A whole file can be flagged as containing hex numbers (and thus not needing
a leading "0x" nor trailing "h" on each number) by using "H@" on the command line before
the filename. The file itself may contain a line with 'HEX' in it, prior to any numbers
that are to be parsed. If the --flexible option is given then "@" can be used before the
filename and when 'HEX' is detected in the file (before any numbers) the code switches to
hex mode. Without the --flexible option "H@" must be use before the filename. As a
convenience the 'HEX' string may appear after hex numbers have been decoded and it will be
ignored. This is to allow hex sgl_s files to be concatenated together and still be parsed
without error.
A file being parsed may contain comments following a "#" symbols. Everything from and
include the hash mark to the end of a line is ignored. Blank lines and "whitespace"
(spaces, tabs, CRs and LFs) are also ignored.
If large sgl_s or index arrays are being used it is better to have one element per line in
the file to be read. This is because a line is not expected to be over 1024 bytes long
with more than 254 parsable items on it. This utility imposes no limit on the number of
lines a file to be parsed may have.
Files to be written out by this utility have their names specified by the --out=O_SGL
(optionally together with --extension=FNE) and the --iaf=IAF options. Unlike the file
reading options, no "@" character should be placed in front of the filename on the command
line. If a filename of "-" is given then output is written to stdout instead of a file.
stdout is normally the console. If the filename starts with "+" then that character is
skipped the output will be appended to that file, if it exists. If the filename starts
with "+" and the file does not exist then it is created. If "+" is not given and the file
already exists then it is truncated (to 0) then overwritten. Some output file names have
numbers (e.g. as a result of the --action=spilt_<n> option) or "_t" (e.g. as a result of
"twin" actions) appended to them (before the extension, if any). Sgl elements are output
one per line, with a comma separating the LBA and the NUM. Index arrays are output one
element (an index) per line. The --hex option controls the form of those numbers output.
If --hex is not given, the numbers are output in decimal. If the --hex option is given one
the number are output in hex with a "0x" prefix. If the --hex option is given twice then
the line 'HEX' is written to the file before any numbers and those numbers are in hex
without any adornment (i.e. with no leading "0x").
If the --document option is given then some information including a date timestamp of
generation is placed as comments at the beginning of files that are written out by this
utility. If the --document option is given twice then the invocation line of this utility
that caused the output is placed in the written file as an additional comment.
The written file format is compatible with the read file format. So, for example, a sgl
generated by a invocation of this utility can later be used as a file to be read by
another invocation of this utility.
EXIT STATUS
The exit status of ddpt_sgl is 0 when it is successful. Note that some options and actions
that return a boolean value return 0 for true and 36 for false. Otherwise the exit status
for this utility is the same as that for ddpt. See the EXIT STATUS section in the ddpt man
page.
EXAMPLES
Examples are needed. See testing/test_sgl.sh script in this package. That script can be
run without root permissions and places its work file (sgl_s) in the /tmp directory.
AUTHORS
Written by Douglas Gilbert.
REPORTING BUGS
Report bugs to <dgilbert at interlog dot com>.
COPYRIGHT
Copyright © 2020-2021 Douglas Gilbert
This software is distributed under a FreeBSD license. There is NO warranty; not even for
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
SEE ALSO
ddpt(8), ddptctl(8)