Provided by: fitsh_0.9.4-1_amd64 
NAME
grmatch - pairing lines by involving identifier or cross matching
SYNOPSIS
grmatch [options] -r <reference> -i <input> [-o <output>]
DESCRIPTION
The program `grmatch` matches lines read from two input files, namely from a reference and
from an input file. All implemented algorithms are symmetric, in the manner that the
result should be the same if these two files are swapped. The only case when the order of
these files is important is when a geometrical transformation is also returned (see point
matching below), in this case the swapping of the files results the inverse form of the
original transformation. The lines (rows) can be matched using various criteria. 1. Lines
can be matched by identifier, where the identifier can be any concatenation of arbitrary,
space-separated columns found in the files. Generally, the identifier is represented by a
single column (e.g. it is an astronomical catalog identifier). The behaviour of the
program can be tuned for the cases when there are more than one rows with the same
identifier. 2. Lines can be matched using a 2-dimensional point matchig algorithm. In this
method, the program expects two-two columns both from the reference and input files which
can be treated as X and Y coordinates. If both point lists are known, the program tries to
find the appropriate geometrical transformation which transforms the points from the
frame of the reference list to the frame of the input list and, simultaneously, tries to
find as many pairs as possible. The parameters of the geometrical transformation and the
whole algorithm can be fine-tuned. 3. Lines can be matched using arbitrary- (N-)
dimensional coordinate matching algorithm. This method expects N-N columns both from the
reference and input files which can be treated as X_1, ..., X_N Cartesian coordinates and
the method assumes both of the point sets in the same reference frame. The point 'A' from
the reference list and the point 'P' from the input list forms a pair if the closest
point to 'A' from the input list is 'P' and vice versa.
OPTIONS
General options:
-h, --help
Give general summary about the command line options.
--long-help, --help-long
Gives a detailed list of command line options.
--wiki-help, --help-wiki, --mediawiki-help, --help-mediawiki
Gives a detailed list of command line options in Mediawiki format.
--version, --version-short, --short-version
Give some version information about the program.
-C, --comment
Comment the output (both the transformation file and the match file).
Options for input/output specifications:
-r <file>, --reference <file> --input-reference <file>
Mandatory, name of the reference file.
<inputfile>, -i <inputile>, --input <inputfile>
Name of the input file. If this switch is omitted, the input isread from stdin
(specifying some input is mandatory).
--separator-reference <char>|space, --separator-input <char>|space
Character for separating the fields of the reference and the input input files,
respectively. By default, the separation is done using whitespaces, it can be
ephasized by defining 'space' here. Otherwise, the character <char> should only be
a single character. For instance, use '--separator-reference ,' and/or
'--separator-input ,' to process CSV files.
-o <output>, --output <output>, --output-matched <output>
Name of the output file, containing the matched lines. The matched lines are pasted
lines, the first part is from the reference file and the second part is from the
input file, these two parts are concatenated by a TAB character. This switch is
optional, if it is not specified, no such output will be generated.
--output-matched-reference <out>, --output-matched-input <out>
Name of the output file, containing the lines corresponding to matches but only
from the reference file or from the input file, respectively.
--output-excluded-reference <out>, --output-excluded-input <out>
Names of the files which contain the valid but excluded lines from the reference
and from the input. These outputs are disjoint from the previous output and
altogether contaions all valid lines.
--output-id <out>
Name of the file which contaions only the identifiers of the matched lines. If the
primary matching method was not identifier matching, one should specify the column
indices of the identifiers by --col-ref-id and --col-inp-id also.
--output-transformation <output-transformation-file>
Name of the output file containing the geometrical transformation, in
human-readable format, if the matching method was point matching (in other case,
this option has no effect). The commented version of this file includes some
statistics about the matching (the total number of lines used and matched,
the required CPU time, the final triangulation level, the fit residuals and other
things like these).
In all of the above input/output file specifications, the replacement of the file name
by "-" (a single minus sign) forces the reading from stdin or writing to stdout. Note that
all parts of the any line after "#" (hashmark) are treated as a comment, therefore
ignored.
General options for point matching:
--match-points
This switch forces the usage of the point matching method. By default, this
method is assumed to be used, therefore this switch can be omitted.
--col-ref <x>,<y>, --col-inp <x>,<y>
The column indices containing the X and Y coordinates, for the reference and for
the input file, respectively. The index of the first column is always 1, the
index of the second is 2 and so on. Lines in which these columns do not contain
valid real numbers bers are omitted.
-a <order>, --order <order>
This switch specifies the polynomial order of the resulted geometrical
transformation. It can be arbitrary positive integer. Note that if the order is
A, at least (A+1)*(A+2)/2 valid points are needed both from the reference and both
from the input file to fit the transformation.
--max-distance <maxdist>
The maximal accepted distance between the matched points in the coordinate frame
of the input coordinate list (and not in the coordinate frame of the reference
coordinate list). Possible pairs (which are valid pairs due to the symmetric
coordinate matching algorihms) are excluded if their Eucledian distance is larger
than maxdist. Note that this option has no initial value, therefore, if omitted,
all possible pairs due to the symmetric matching are resulted, which, in certain
cases in practice, can result unexpected behaviour. One should always specify a
reasonable maximal distance which can be estimated only by the knowledge of
the physics of the input files.
See more options concerning to point matching in the section "Fine-Tuning of Point
Matching" below. That section also describes the tuning of the triangulation used by
the point matching algorithm. For a more detailed description about the point matching
algorithms based on pattern and triangle matching see [1], [2] or [3].
General options for coordinate matching:
--match-coord, --match-coords
This switch forces the usage of the coordinate matching method. Note that because
of the common options with the point matching method, one should specify this
switch to force the usage of the coordinate matching method (the default method is
point matching, see above).
--col-ref <x>[,<y>,[<z>...]] --col-inp <x>[,<y>,[<z>...]]
The column indices containing the spatial coordinates, for the reference and for
the input file, respectively. The index of the first column is always 1, the
index of the second is 2 and so on. Lines in which these columns do not contain
valid real numbers are omitted. Note that the dimension of the coordinate
matching space is specified indirectly, by the number of column indices listed
here. Because of this, the number of column indices should be the same for the
reference and input, in other case, when the dimensions are mismatched, the
program exits unsuccessfully.
--max-distance <maxdist>
The maximal accepted distance between the matched points. Possible pairs (which
are valid pairs due to the symmetric coordinate matching algorihms) are excluded if
their Eucledian distance is larger than maxdist. Note that this option has no
initial value, therefore, if omitted, all possible pairs due to the symmetric
matching are resulted (see also point matching, above).
General options for identifier matching:
--match-id, --match-identifiers
This switch forces the usage of the identifier matching method.
--col-ref-id <i>[,<j>,[<k>...]] --col-inp-id <i>[,<j>,[<k>...]]
Column index or indices containing the identifiers, from the reference and from
the input file, respectively.
--no-ambiguity, --first-ambiguity, --any-ambiguity, --full-ambiguity
These options tune the behaviour of the matching when there is more than one
occurrence of a given identifier in the reference and/or input file. If
--no-ambiguity is specified, these identifiers are discarded, this is the default
method. If --first-ambiguity is specified, only the first occurence is treated as
a matched line, independently from the number of occurrences. If the switch
--any-ambiguity is specified, the lines are paired sequentally, until there is
any left from the reference and from the input. For example, if there is 4
occurrences in the reference and 6 in the input file of a given identifier, 4
matched pairs are returned. Otherwise, if --full-ambiguity is specified, all
possible combinations of the lines are treated as matched lines. For example, if
there is 4 occurrences in the reference and 6 in the input file of a given
identifier, all 4*6=24 combinations are returned as matched pairs.
Fine-tuning of point matching:
--triangulation <parameters>
This switch is followed by comma-separated directives, which specify the
parameters of the triangulation-based point matching algorithm:
delaunay, level=<level>, full, auto, unitarity=<U>
These directives specify the triangulation level used for point matching.
"delaunay" forces the usage only of the Delaunay-triangles. This is the fastest
method, however, it is only working if the points in the reference and input lists
are almost competely overlapping and describe almost the same point sets
(within a ratio of common points above 60-70%). The "level" specifies the
level of the expansion of the Delaunay-triangulation (see [1] for more details).
In practice, the lower the ratio of common points and/or the ratio of the
overlapping, the higher level should be used. Specifying "level=1" or "level=2"
gives a robust but still fast method for general usage. The directive "full"
forces full triangulation. This can be overwhelmingly slow and annoying and
requires tons of memory if there are more than 40-50 points (the amounts of these
resources are proportional to the 6th(!) and 3rd power of the number of the
points, respectively). The directive "auto" increases the level of the
triangulation expansion automatically until a proper match is found. A match is
considered as a good match if the unitarity of the transformation is less than the
unitarity U specified by the "unitarity=U" directive (see also the section
Notes/Unitarity below).
mixed, conformable, reverse
These directives define the chirality of the triangle spaces to be used.
Practically, it means the following. If we don't know whether the input and
reference lists are inverted respecting to each other, one should use "mixed"
triangle space. If we are sure about that the input and reference lists are
not inverted, we can use "conformable" triangle space. If we know that the
input and reference lists are inverted, we can use "reverse" space. Note that
although "mixed" triangle space can always result a good match, it is a wise
idea to fix the chirality by specifying "conformable" or "reverse" if we really
know that the point sets are not inverted or inverted respecting to each
other. If the chirality is fixed, the program yields more matched pairs,
the appropriate triangulation level can be smaller and in "auto" mode, the
program returns the match definitely faster.
maxnumber=<max>, maxref=<mr>, maxinp=<mi>
These directives specify the maximal number of points which are used for
triangulation (for any type of triangulation). If "maxnumber" is specified,
it is equivalent to define "maxref" and "maxinp" with the same values. Then, the
first <mr> points from the reference and the first <mi> points from the input
list are used to generate the triangle sets. The "first" points are selected
using the optional information found in one of the columns, see the following
switches.
(Note that there should be only one --triangulation switch, all desired directives should
be written in the same argument, separated by commas.)
--col-ref-ordering [-]<w>, --col-inp-ordering [-]<w>.
These switches specify one-one column index from the reference and from the input
files which are used to order these lists and select the first "maxref" and
"maxinp" points (see above) for the generation of the two triangle meshes.
Both columns should contain valid real numbers, otherwise the whole(!) line
is excluded (not only from sorting but from the whole matching procedure). If there
is no negative sign before the column index, the data are sorted in
descending(!) order, therefore the lines with the lines with the highest(!) values
are selected for triangulation. If there is a negative sign before the index,
the data are sorted in ascending order by these values, therefore the lines
with the smallest(!) values are selected for triangulation. For example, if we want
to match star lists, we might want to use only the brightest ones to
generate the triangle sets. If the brightnesses of the stars are specified by
their fluxes, we should not use the negative sign (the list should be sorted in
descending order to select the first few lines as the brightest stars), and if
the brightness is known by the magnitude, we have to use the negative sign.
--fit iterations=<N>,firstrejection=<F>,sigma=<S>
Like --triangulation, this switch is followed by some directives. These
directives specify the number <N> of iterations ("iterations=<N>") for point
matching. The "firstrejection" directive speciy the serial number <F> of the
first iteration where points farer than <S> "sigma" level are excluded in the next
iteration. Note that in practice these type of iteration is really not
important (due to, for instance, the limitations of the outliers by the
--max-distance switch), however, some suspicious users can be convinced by such
arguments.
--weight reference|input,column=<wi>,[magnitude],[power=<p>]
These directives specify the weights which are used during the fit of the
geometrical transformation. For example, in practice it is useful in the
following situation. We try to match star lists, then the fainter stars are
believed to have higher astrometrical errors, therefore they should have smaller
influence in the fit. We can take the weights from the reference (specify
"reference") and from the input (specify "input"), from the column specified by the
weight-index. The weights can be derived from stellar magnitudes, if so,
specify "magnitude" to convert the read values in magnitude to flux. The real
weights then is the "power"th power of the flux. The default value of the
"power" is 1, however, for the maximum-likelihood estimation of an assumed
Gaussian distribution, the weights should be the second power of the fluxes.
Some notes on unitarity. The unitarity of a geometrical transformation measures how it
differs from the closest transformation which is affine and a combination of dilation,
rotation and shift. For such a transformation the unitarity is 0 and if the
second-order terms in a transformation distort a such unitary transformation, the
unitarity will have the same magnitude like the magnitude of this second-order effect.
For example, to map a part of a sphere with the size of d degrees will have an unitarity
of 1-cos(d). Therefore, for astrometrical purposes, a reasonable value of the critical
unitarity in "auto" triangulation mode can be estimated as 2 or 3 times 1-cos(d/2)
where d is the size of the field in which astrometry should be performed.
REPORTING BUGS
Report bugs to <apal@szofi.net>, see also https://fitsh.net/.
COPYRIGHT
Copyright © 1996, 2002, 2004-2008, 2010-2016, 2018-2020; Pal, Andras <apal@szofi.net>