Provided by: libhtml-tableextract-perl_2.15-1_all bug

NAME

       HTML::TableExtract - Perl module for extracting the content contained in tables within an
       HTML document, either as text or encoded element trees.

SYNOPSIS

        # Matched tables are returned as table objects; tables can be matched
        # using column headers, depth, count within a depth, table tag
        # attributes, or some combination of the four.

        # Example: Using column header information.
        # Assume an HTML document with tables that have "Date", "Price", and
        # "Cost" somewhere in a row. The columns beneath those headings are
        # what you want to extract. They will be returned in the same order as
        # you specified the headers since 'automap' is enabled by default.

        use HTML::TableExtract;
        my $te = HTML::TableExtract->new( headers => [qw(Date Price Cost)] );
        $te->parse($html_string);

        # Examine all matching tables
        foreach my $ts ($te->tables) {
          print "Table (", join(',', $ts->coords), "):\n";
          foreach my $row ($ts->rows) {
             print join(',', @$row), "\n";
          }
        }

        # Shorthand...top level rows() method assumes the first table found in
        # the document if no arguments are supplied.
        foreach my $row ($te->rows) {
           print join(',', @$row), "\n";
        }

        # Example: Using depth and count information.
        # Every table in the document has a unique depth and count tuple, so
        # when both are specified it is a unique table. Depth and count both
        # begin with 0, so in this case we are looking for a table (depth 2)
        # within a table (depth 1) within a table (depth 0, which is the top
        # level HTML document). In addition, it must be the third (count 2)
        # such instance of a table at that depth.

        my $te = HTML::TableExtract->new( depth => 2, count => 2 );
        $te->parse_file($html_file);
        foreach my $ts ($te->tables) {
           print "Table found at ", join(',', $ts->coords), ":\n";
           foreach my $row ($ts->rows) {
              print "   ", join(',', @$row), "\n";
           }
        }

        # Example: Using table tag attributes.
        # If multiple attributes are specified, all must be present and equal
        # for match to occur.

        my $te = HTML::TableExtract->new( attribs => { border => 1 } );
        $te->parse($html_string);
        foreach my $ts ($te->tables) {
          print "Table with border=1 found at ", join(',', $ts->coords), ":\n";
          foreach my $row ($ts->rows) {
             print "   ", join(',', @$row), "\n";
          }
        }

        # Example: Extracting as an HTML::Element tree structure
        # Rather than extracting raw text, the html can be converted into a
        # tree of element objects. The HTML document is composed of
        # HTML::Element objects and the tables are HTML::ElementTable
        # structures. Using this, the contents of tables within a document can
        # be edited in-place.

        use HTML::TableExtract qw(tree);
        my $te = HTML::TableExtract->new( headers => qw(Fee Fie Foe Fum) );
        $te->parse_file($html_file);
        my $table = $te->first_table_found;
        my $table_tree = $table->tree;
        $table_tree->cell(4,4)->replace_content('Golden Goose');
        my $table_html = $table_tree->as_HTML;
        my $table_text = $table_tree->as_text;
        my $document_tree = $te->tree;
        my $document_html = $document_tree->as_HTML;

DESCRIPTION

       HTML::TableExtract is a subclass of HTML::Parser that serves to extract the information
       from tables of interest contained within an HTML document. The information from each
       extracted table is stored in table objects. Tables can be extracted as text, HTML, or
       HTML::ElementTable structures (for in-place editing or manipulation).

       There are currently four constraints available to specify which tables you would like to
       extract from a document: Headers, Depth, Count, and Attributes.

       Headers, the most flexible and adaptive of the techniques, involves specifying text in an
       array that you expect to appear above the data in the tables of interest. Once all headers
       have been located in a row of that table, all further cells beneath the columns that
       matched your headers are extracted. All other columns are ignored: think of it as vertical
       slices through a table. In addition, TableExtract automatically rearranges each row in the
       same order as the headers you provided. If you would like to disable this, set automap to
       0 during object creation, and instead rely on the column_map() method to find out the
       order in which the headers were found. Furthermore, TableExtract will automatically
       compensate for cell span issues so that columns are really the same columns as you would
       visually see in a browser. This behavior can be disabled by setting the gridmap parameter
       to 0. HTML is stripped from the entire textual content of a cell before header matches are
       attempted -- unless the keep_html parameter was enabled.

       Depth and Count are more specific ways to specify tables in relation to one another. Depth
       represents how deeply a table resides in other tables. The depth of a top-level table in
       the document is 0. A table within a top-level table has a depth of 1, and so on. Each
       depth can be thought of as a layer; tables sharing the same depth are on the same layer.
       Within each of these layers, Count represents the order in which a table was seen at that
       depth, starting with 0. Providing both a depth and a count will uniquely specify a table
       within a document.

       Attributes match based on the attributes of the html <table> tag, for example, border
       widths or background color.

       Each of the Headers, Depth, Count, and Attributes specifications are cumulative in their
       effect on the overall extraction.  For instance, if you specify only a Depth, then you get
       all tables at that depth (note that these could very well reside in separate higher- level
       tables throughout the document since depth extends across tables).  If you specify only a
       Count, then the tables at that Count from all depths are returned (i.e., the nth
       occurrence of a table at each depth). If you only specify Headers, then you get all tables
       in the document containing those column headers. If you have specified multiple
       constraints of Headers, Depth, Count, and Attributes, then each constraint has veto power
       over whether a particular table is extracted.

       If no Headers, Depth, Count, or Attributes are specified, then all tables match.

       When extracting only text from tables, the text is decoded with HTML::Entities by default;
       this can be disabled by setting the decode parameter to 0.

   Extraction Modes
       The default mode of extraction for HTML::TableExtract is raw text or HTML. In this mode,
       embedded tables are completely decoupled from one another. In this case,
       HTML::TableExtract is a subclass of HTML::Parser:

         use HTML::TableExtract;

       Alternatively, tables can be extracted as HTML::ElementTable structures, which are in turn
       embedded in an HTML::Element tree representing the entire HTML document. Embedded tables
       are not decoupled from one another since this tree structure must be maintained. In this
       case, HTML::TableExtract is a subclass of HTML::TreeBuilder (itself a subclass of
       HTML:::Parser):

         use HTML::TableExtract qw(tree);

       In either case, the basic interface for HTML::TableExtract and the resulting table objects
       remains the same -- all that changes is what you can do with the resulting data.

       HTML::TableExtract is a subclass of HTML::Parser, and as such inherits all of its basic
       methods such as "parse()" and "parse_file()". During scans, "start()", "end()", and
       "text()" are utilized. Feel free to override them, but if you do not eventually invoke
       them in the SUPER class with some content, results are not guaranteed.

   Advice
       The main point of this module was to provide a flexible method of extracting tabular
       information from HTML documents without relying to heavily on the document layout. For
       that reason, I suggest using Headers whenever possible -- that way, you are anchoring your
       extraction on what the document is trying to communicate rather than some feature of the
       HTML comprising the document (other than the fact that the data is contained in a table).

METHODS

       The following are the top-level methods of the HTML::TableExtract object. Tables that have
       matched a query are actually returned as separate objects of type
       HTML::TableExtract::Table. These table objects have their own methods, documented further
       below.

   CONSTRUCTOR
       new()
           Return a new HTML::TableExtract object. Valid attributes are:

           headers
               Passed as an array reference, headers specify strings of interest at the top of
               columns within targeted tables. They can be either strings or regular expressions
               (qr//). If they are strings, they will eventually be passed through a non-
               anchored, case-insensitive regular expression, so regexp special characters are
               allowed.

               The table row containing the headers is not returned, unless "keep_headers" was
               specified or you are extracting into an element tree. In either case the header
               row can be accessed via the hrow() method from within the table object.

               Columns that are not beneath one of the provided headers will be ignored unless
               "slice_columns" was set to 0. Columns will, by default, be rearranged into the
               same order as the headers you provide (see the automap parameter for more
               information) unless "slice_columns" is 0.

               Additionally, by default columns are considered what you would see visually
               beneath that header when the table is rendered in a browser.  See the "gridmap"
               parameter for more information.

               HTML within a header is stripped before the match is attempted, unless the
               "keep_html" parameter was specified and "strip_html_on_match" is false.

           depth
               Specify how embedded in other tables your tables of interest should be.  Top-level
               tables in the HTML document have a depth of 0, tables within top-level tables have
               a depth of 1, and so on.

           count
               Specify which table within each depth you are interested in, beginning with 0.

           attribs
               Passed as a hash reference, attribs specify attributes of interest within the HTML
               <table> tag itself.

           automap
               Automatically applies the ordering reported by column_map() to the rows returned
               by rows(). This only makes a difference if you have specified Headers and they
               turn out to be in a different order in the table than what you specified. Automap
               will rearrange the columns in the same order as the headers appear. To get the
               original ordering, you will need to take another slice of each row using
               column_map(). automap is enabled by default.

           slice_columns
               Enabled by default, this option controls whether vertical slices are returned from
               under headers that match. When disabled, all columns of the matching table are
               retained, regardles of whether they had a matching header above them. Disabling
               this also disables "automap".

           keep_headers
               Disabled by default, and only applicable when header constraints have been
               specified, "keep_headers" will retain the matching header row as the first row of
               table data when enabled. This option has no effect if extracting into an element
               tree structure. In any case, the header row is accessible from the table method
               "hrow()".

           gridmap
               Controls whether the table contents are returned as a grid or a tree.  ROWSPAN and
               COLSPAN issues are compensated for, and columns really are columns. Empty phantom
               cells are created where they would have been obscured by ROWSPAN or COLSPAN
               settings. This really becomes an issue when extracting columns beneath headers.
               Enabled by default.

           subtables
               Extract all tables embedded within matched tables.

           decode
               Automatically decode retrieved text with HTML::Entities::decode_entities().
               Enabled by default. Has no effect if "keep_html" was specified or if extracting
               into an element tree structure.

           br_translate
               Translate <br> tags into newlines. Sometimes the remaining text can be hard to
               parse if the <br> tag is simply dropped. Enabled by default. Has no effect if
               keep_html is enabled or if extracting into an element tree structure.

           keep_html
               Return the raw HTML contained in the cell, rather than just the visible text.
               Embedded tables are not retained in the HTML extracted from a cell. Patterns for
               header matches must take into account HTML in the string if this option is
               enabled. This option has no effect if extracting into an elment tree structure.

           strip_html_on_match
               When "keep_html" is enabled, HTML is stripped by default during attempts at
               matching header strings (so if "strip_html_on_match" is not enabled and
               "keep_html" is, you would have to include potential HTML tags in the regexp for
               header matches). Stripped header tags are replaced with an empty string, e.g. 'hot
               d<em>og</em>' would become 'hot dog' before attempting a match.

           error_handle
               Filehandle where error messages are printed. STDERR by default.

           debug
               Prints some debugging information to STDERR, more for higher values.  If
               "error_handle" was provided, messages are printed there rather than STDERR.

   REGULAR METHODS
       The following methods are invoked directly from an HTML::TableExtract object.

       depths()
           Returns all depths that contained matched tables in the document.

       counts($depth)
           For a particular depth, returns all counts that contained matched tables.

       table($depth, $count)
           For a particular depth and count, return the table object for the table found, if any.

       tables()
           Return table objects for all tables that matched. Returns an empty list if no tables
           matched.

       first_table_found()
           Return the table state object for the first table matched in the document. Returns
           undef if no tables were matched.

       current_table()
           Returns the current table object while parsing the HTML. Only useful if you're messing
           around with overriding HTML::Parser methods.

       tree()
           If the module was invoked in tree extraction mode, returns a reference to the top node
           of the HTML::Element tree structure for the entire document (which includes,
           ultimately, all tables within the document).

       tables_report([$show_content, $col_sep])
           Return a string summarizing extracted tables, along with their depth and count.
           Optionally takes a $show_content flag which will dump the extracted contents of each
           table as well with columns separated by $col_sep. Default $col_sep is ':'.

       tables_dump([$show_content, $col_sep])
           Same as "tables_report()" except dump the information to STDOUT.

       start
       end
       text
           These are the hooks into HTML::Parser. If you want to subclass this module and have
           things work, you must at some point call these with content.

   DEPRECATED METHODS
       Tables used to be called 'table states'. Accordingly, the following methods still work but
       have been deprecated:

       table_state()
           Is now table()

       table_states()
           Is now tables()

       first_table_state_found()
           Is now first_table_found()

   TABLE METHODS
       The following methods are invoked from an HTML::TableExtract::Table object, such as those
       returned from the "tables()" method.

       rows()
           Return all rows within a matched table. Each row returned is a reference to an array
           containing the text, HTML, or reference to the HTML::Element object of each cell
           depending the mode of extraction. Tables with rowspan or colspan attributes will have
           some cells containing undef.  Returns a list or a reference to an array depending on
           context.

       columns()
           Return all columns within a matched table. Each column returned is a reference to an
           array containing the text, HTML, or reference to HTML::Element object of each cell
           depending on the mode of extraction.  Tables with rowspan or colspan attributes will
           have some cells containing undef.

       row($row)
           Return a particular row from within a matched table either as a list or an array
           reference, depending on context.

       column($col)
           Return a particular column from within a matched table as a list or an array
           reference, depending on context.

       cell($row,$col)
           Return a particular item from within a matched table, whether it be the text, HTML, or
           reference to the HTML::Element object of that cell, depending on the mode of
           extraction. If the cell was covered due to rowspan or colspan effects, will return
           undef.

       space($row,$col)
           The same as cell(), except in cases where the given coordinates were covered due to
           rowspan or colspan issues, in which case the content of the covering cell is returned
           rather than undef.

       depth()
           Return the depth at which this table was found.

       count()
           Return the count for this table within the depth it was found.

       coords()
           Return depth and count in a list.

       tree()
           If the module was invoked in tree extraction mode, this accessor provides a reference
           to the HTML::ElementTable structure encompassing the table.

       hrow()
           Returns the header row as a list when headers were specified as a constraint. If
           "keep_headers" was specified initially, this is equivalent to the first row returned
           by the "rows()" method.

       column_map()
           Return the order (via indices) in which the provided headers were found.  These
           indices can be used as slices on rows to either order the rows in the same order as
           headers or restore the rows to their natural order, depending on whether the rows have
           been pre-adjusted using the automap parameter.

       lineage()
           Returns the path of matched tables that led to matching this table. The path is a list
           of array refs containing depth, count, row, and column values for each ancestor table
           involved. Note that corresponding table objects will not exist for ancestral tables
           that did not match specified constraints.

NOTES ON TREE EXTRACTION MODE

       As mentioned above, HTML::TableExtract can be invoked in 'tree' mode where the resulting
       HTML and extracted tables are encoded in HTML::Element tree structures:

         use HTML::TableExtract 'tree';

       There are a number of things to take note of while using this mode. The entire HTML
       document is encoded into an HTML::Element tree. Each table is part of this structure, but
       nevertheless is tracked separately via an HTML::ElementTable structure, which is a
       specialized form of HTML::Element tree.

       The HTML::ElementTable objects are accessible by invoking the tree() method from within
       each table object returned by HTML::TableExtract. The HTML::ElementTable objects have
       their own row(), col(), and cell() methods (among others). These are not to be confused
       with the row() and column() methods provided by the HTML::TableExtract::Table objects.

       For example, the row() method from HTML::ElementTable will provide a reference to a 'glob'
       of all the elements in that row. Actions (such as setting attributes) performed on that
       row reference will affect all elements within that row. On the other hand, the row()
       method from the HTML::TableExtract::Table object will return an array (either by reference
       or list, depending on context) of the contents of each cell within the row. In tree mode,
       the content is represented by individual references to each cell -- these are references
       to the same HTML::Element objects that reside in the HTML::Element tree.

       The cell() methods provided in both cases will therefore return references to the same
       object. The exception to this is when a 'cell' in the table grid was originally 'covered'
       due to rowspan or colspan issues -- in this case the cell content will be undef. Likewise,
       the row() or column() methods from HTML::TableExtract::Table objects will return arrays
       potentially containing a mixture of object references and undefs.  If you're going to be
       doing lots of manipulation of the table elements, it might be more efficient to access
       them via the methods provided by the HTML::ElementTable object instead. See
       HTML::ElementTable for more information on how to manipulate those objects.

       An alternative to the cell() method in HTML::TableExtract::Table is the space() method. It
       is largely similar to cell(), except when given coordinates of a cell that was covered due
       to rowspan or colspan effects, it will return the contents of the cell that was covering
       that space rather than undef. So if, for example, cell (0,0) had a rowspan of 2 and
       colspan of 2, cell(1,1) would return undef and space(1,1) would return the same content as
       cell(0,0) or space(0,0).

REQUIRES

       HTML::Parser(3), HTML::Entities(3)

OPTIONALLY REQUIRES

       HTML::TreeBuilder(3), HTML::ElementTable(3)

AUTHOR

       Matthew P. Sisk, <sisk@mojotoad.com>

COPYRIGHT

       Copyright (c) 2000-2017 Matthew P. Sisk.  All rights reserved. All wrongs revenged. This
       program is free software; you can redistribute it and/or modify it under the same terms as
       Perl itself.

SEE ALSO

       HTML::Parser(3), HTML::TreeBuilder(3), HTML::ElementTable(3), perl(1).