Provided by: libweb-query-perl_0.39-1_all bug

NAME

       Web::Query - Yet another scraping library like jQuery

VERSION

       version 0.39

SYNOPSIS

           use Web::Query;

           wq('http://www.w3.org/TR/html401/')
               ->find('div.head dt')
               ->each(sub {
                   my $i = shift;
                   printf("%d %s\n", $i+1, $_->text);
               });

DESCRIPTION

       Web::Query is a yet another scraping framework, have a jQuery like interface.

       Yes, I know Ingy's pQuery. But it's just a alpha quality. It doesn't works.  Web::Query
       built at top of the CPAN modules, HTML::TreeBuilder::XPath, LWP::UserAgent, and
       HTML::Selector::XPath.

       So, this module uses HTML::Selector::XPath and only supports the CSS 3 selector supported
       by that module.  Web::Query doesn't support jQuery's extended queries(yet?). If a selector
       is passed as a scalar ref, it'll be taken as a straight XPath expression.

           $wq( '<div><p>hello</p><p>there</p></div>' )->find( 'p' );       # css selector
           $wq( '<div><p>hello</p><p>there</p></div>' )->find( \'/div/p' ); # xpath selector

       THIS LIBRARY IS UNDER DEVELOPMENT. ANY API MAY CHANGE WITHOUT NOTICE.

FUNCTIONS

       "wq($stuff)"
           This is a shortcut for "Web::Query->new($stuff)". This function is exported by
           default.

METHODS

   CONSTRUCTORS
       my $q = Web::Query->new($stuff, \%options )
           Create new instance of Web::Query. You can make the instance from URL(http, https,
           file scheme), HTML in string, URL in string, URI object, "undef", and either one
           HTML::Element object or an array ref of them.

               # all valid creators
               $q = Web::Query->new( 'http://techblog.babyl.ca' );
               $q = Web::Query->new( '<p>foo</p>' );
               $q = Web::Query->new( undef );

           This method throw the exception on unknown $stuff.

           This method returns undefined value on non-successful response with URL.

           Currently, the only two valid options are indent, which will be used as the
           indentation string if the object is printed, and no_space_compacting, which will
           prevent the compaction of whitespace characters in text blocks.

       my $q = Web::Query->new_from_element($element: HTML::Element)
           Create new instance of Web::Query from instance of HTML::Element.

       "my $q = Web::Query->new_from_html($html: Str)"
           Create new instance of Web::Query from HTML.

       my $q = Web::Query->new_from_url($url: Str)
           Create new instance of Web::Query from URL.

           If the response is not success(It means /^20[0-9]$/), this method returns undefined
           value.

           You can get a last result of response, use the $Web::Query::RESPONSE.

           Here is a best practical code:

               my $url = 'http://example.com/';
               my $q = Web::Query->new_from_url($url)
                   or die "Cannot get a resource from $url: " . Web::Query->last_response()->status_line;

       my $q = Web::Query->new_from_file($file_name: Str)
           Create new instance of Web::Query from file name.

   TRAVERSING
       add

       Returns a new object augmented with the new element(s).

       add($html)
           An HTML fragment to add to the set of matched elements.

       add(@elements)
           One or more @elements to add to the set of matched elements.

           @elements that already are part of the set are not added a second time.

               my $group = $wq->find('#foo');         # collection has 1 element
               $group = $group->add( '#bar', $wq );   # 2 elements
               $group->add( '#foo', $wq );            # still 2 elements

       add($wq)
           An existing Web::Query object to add to the set of matched elements.

       add($selector, $context)
           $selector is a string representing a selector expression to find additional elements
           to add to the set of matched elements.

           $context is the point in the document at which the selector should begin matching

       contents

       Get the immediate children of each element in the set of matched elements, including text
       and comment nodes.

       each

       Visit each nodes. $i is a counter value, 0 origin. $elem is iteration item.  $_ is
       localized by $elem.

           $q->each(sub { my ($i, $elem) = @_; ... })

       end

       Back to the before context like jQuery.

       filter

       Reduce the elements to those that pass the function's test.

           $q->filter(sub { my ($i, $elem) = @_; ... })

       find

       Get the descendants of each element in the current set of matched elements, filtered by a
       selector.

           my $q2 = $q->find($selector); # $selector is a CSS3 selector.

       NOTE If you want to match the element itself, use "filter".

       INCOMPATIBLE CHANGE From v0.14 to v0.19 (inclusive) find() also matched the element
       itself, which is not jQuery compatible.  You can achieve that result using "filter()",
       "add()" and "find()":

           my $wq = wq('<div class="foo"><p class="foo">bar</p></div>'); # needed because we don't have a global document like jQuery does
           print $wq->filter('.foo')->add($wq->find('.foo'))->as_html; # <div class="foo"><p class="foo">bar</p></div><p class="foo">bar</p>

       first

       Return the first matching element.

       This method constructs a new Web::Query object from the first matching element.

       last

       Return the last matching element.

       This method constructs a new Web::Query object from the last matching element.

       match($selector)

       Returns a boolean indicating if the elements match the $selector.

       In scalar context returns only the boolean for the first element.

       For the reverse of "not()", see "filter()".

       not($selector)

       Returns all the elements not matching the $selector.

           # $do_for_love will be every thing, except #that
           my $do_for_love = $wq->find('thing')->not('#that');

       and_back

       Add the previous set of elements to the current one.

           # get the h1 plus everything until the next h1
           $wq->find('h1')->next_until('h1')->and_back;

       map

       Creates a new array with the results of calling a provided function on every element.

           $q->map(sub { my ($i, $elem) = @_; ... })

       parent

       Get the parent of each element in the current set of matched elements.

       prev

       Get the previous node of each element in the current set of matched elements.

           my $prev = $q->prev;

       next

       Get the next node of each element in the current set of matched elements.

          my $next = $q->next;

       next_until( $selector )

       Get all subsequent siblings, up to (but not including) the next node matched $selector.

   MANIPULATION
       add_class

       Adds the specified class(es) to each of the set of matched elements.

           # add class 'foo' to <p> elements
           wq('<div><p>foo</p><p>bar</p></div>')->find('p')->add_class('foo');

       toggle_class( @classes )

       Toggles the given class or classes on each of the element. I.e., if the element had the
       class, it'll be removed, and if it hadn't, it'll be added.

       Classes are toggled once, no matter how many times they appear in the argument list.

           $q->toggle_class( 'foo', 'foo', 'bar' );

           # equivalent to

           $q->toggle_class('foo')->toggle_class('bar');

           # and not

           $q->toggle_class('foo')->toggle_class('foo')->toggle_class('bar');

       after

       Insert content, specified by the parameter, after each element in the set of matched
       elements.

           wq('<div><p>foo</p></div>')->find('p')
                                      ->after('<b>bar</b>')
                                      ->end
                                      ->as_html; # <div><p>foo</p><b>bar</b></div>

       The content can be anything accepted by "new".

       append

       Insert content, specified by the parameter, to the end of each element in the set of
       matched elements.

           wq('<div></div>')->append('<p>foo</p>')->as_html; # <div><p>foo</p></div>

       The content can be anything accepted by "new".

       as_html

       Returns the string representations of either the first or all elements, depending if
       called in list or scalar context.

       If given an argument "join", the string representations of the elements will be
       concatenated with the given string.

           wq( '<div><p>foo</p><p>bar</p></div>' )
               ->find('p')
               ->as_html( join => '!' );
           # <p>foo</p>!<p>bar</p>

       " attr "

       Get/set attribute values.

       In getter mode, it'll return either the values of the attribute for all elements of the
       set, or only the first one depending of the calling context.

           my @values = $q->attr('style');      # style of all elements
           my $first_value = $q->attr('style'); # style of first element

       In setter mode, it'll set attributes value for all elements, and return back the original
       object for easy chaining.

           $q->attr( 'alt' => 'a picture' )->find( ... );

           # can pass more than 1 element too
           $q->attr( alt => 'a picture', src => 'file:///...' );

       The value passed for an attribute can be a code ref. In that case, the code will be called
       with $_ set to the current attribute value.  If the code modifies $_, the attribute will
       be updated with the new value.

           $q->attr( alt => sub { $_ ||= 'A picture' } );

       " id "

       Get/set the elements's id attribute.

       In getter mode, it behaves just like "attr()".

       In setter mode, it behaves like "attr()", but with the following exceptions.

       If the attribute value is a scalar, it'll be only assigned to the first element of the set
       (as ids are supposed to be unique), and the returned object will only contain that first
       element.

           my $first_element = $q->id('the_one');

       It's possible to set the ids of all the elements by passing a sub to "id()". The sub is
       given the same arguments as for "each()", and its return value is taken to be the new id
       of the elements.

           $q->id( sub { my $i = shift;  'foo_' . $i } );

       " name "

       Get/set the elements's 'name' attribute.

           my $name = $q->name;  # equivalent to $q->attr( 'name' );

           $q->name( 'foo' );    # equivalent to $q->attr( name => 'foo' );

       " data "

       Get/set the elements's 'data-*name*' attributes.

           my $data = $q->data('foo');  # equivalent to $q->attr( 'data-foo' );

           $q->data( 'foo' => 'bar' );  # equivalent to $q->attr( 'data-foo' => 'bar' );

       tagname

       Get/Set the tag name of elements.

           my $name = $q->tagname;

           $q->tagname($new_name);

       before

       Insert content, specified by the parameter, before each element in the set of matched
       elements.

           wq('<div><p>foo</p></div>')->find('p')
                                      ->before('<b>bar</b>')
                                      ->end
                                      ->as_html; # <div><b>bar</b><p>foo</p></div>

       The content can be anything accepted by "new".

       clone

       Create a deep copy of the set of matched elements.

       detach

       Remove the set of matched elements from the DOM.

       has_class

       Determine whether any of the matched elements are assigned the given class.

       " html "

       Get/Set the innerHTML.

           my @html = $q->html();

           my $html = $q->html(); # 1st matching element only

           $q->html('<p>foo</p>');

       insert_before

       Insert every element in the set of matched elements before the target.

       insert_after

       Insert every element in the set of matched elements after the target.

       " prepend "

       Insert content, specified by the parameter, to the beginning of each element in the set of
       matched elements.

       remove

       Delete the elements associated with the object from the DOM.

           # remove all <blink> tags from the document
           $q->find('blink')->remove;

       remove_class

       Remove a single class, multiple classes, or all classes from each element in the set of
       matched elements.

       replace_with

       Replace the elements of the object with the provided replacement.  The replacement can be
       a string, a "Web::Query" object or an anonymous function. The anonymous function is passed
       the index of the current node and the node itself (with is also localized as $_).

           my $q = wq( '<p><b>Abra</b><i>cada</i><u>bra</u></p>' );

           $q->find('b')->replace_with('<a>Ocus</a>);
               # <p><a>Ocus</a><i>cada</i><u>bra</u></p>

           $q->find('u')->replace_with($q->find('b'));
               # <p><i>cada</i><b>Abra</b></p>

           $q->find('i')->replace_with(sub{
               my $name = $_->text;
               return "<$name></$name>";
           });
               # <p><b>Abra</b><cada></cada><u>bra</u></p>

       size

       Return the number of elements in the Web::Query object.

           wq('<div><p>foo</p><p>bar</p></div>')->find('p')->size; # 2

       text

       Get/Set the text.

           my @text = $q->text();

           my $text = $q->text(); # 1st matching element only

           $q->text('text');

       If called in a scalar context, only return the string representation of the first element

   OTHERS
       Web::Query->last_response()
           Returns last HTTP response status that generated by "new_from_url()".

HOW DO I CUSTOMIZE USER AGENT?

       You can specify your own instance of LWP::UserAgent.

           $Web::Query::UserAgent = LWP::UserAgent->new( agent => 'Mozilla/5.0' );

FAQ AND TROUBLESHOOTING

   How to find XML processing instructions in a document?
       It's possible with Web::Query::LibXML and by using an xpath expression with "find()":

           # find <?xml-stylesheet ... ?>
           $q->find(\"//processing-instruction('xml-stylesheet')");

       However, note that the support for processing instructions in
       HTML::TreeBuilder::LibXML::Node is sketchy, so there are methods like "attr()" that won't
       work.

   Can't get the content of script elements
       The <script> tag is treated differently by HTML::TreeBuilder, the parser used by
       Web::Query. To retrieve the content, you can use either the method "html()" (with the
       caveat that the content will be escaped), or use Web::Query::LibXML, which parse the
       'script' element differently.

           my $node = "<script>var x = '<p>foo</p>';</script>";

           say Web::Query::wq( $node )->text;
               # nothing is printed!

           say Web::Query::wq( $node )->html;
               # var x = &#39;&lt;p&gt;foo&lt;/p&gt;&#39;;

           say Web::Query::LibXML::wq( $node )->text;
               # var x = '<p>foo</p>';

           say Web::Query::LibXML::wq( $node )->html;
               # var x = '&lt;p&gt;foo&lt;/p&gt;';

INCOMPATIBLE CHANGES

       0.10
           new_from_url() is no longer throws exception on bad response from HTTP server.

AUTHOR

       Tokuhiro Matsuno <tokuhirom AAJKLFJEF@ GMAIL COM>

SEE ALSO

       pQuery
       XML::LibXML::jQuery

LICENSE

       Copyright (C) Tokuhiro Matsuno

       This library is free software; you can redistribute it and/or modify it under the same
       terms as Perl itself.

BUGS

       Please report any bugs or feature requests on the bugtracker website
       https://github.com/tokuhirom/Web-Query/issues

       When submitting a bug or request, please include a test-file or a patch to an existing
       test-file that illustrates the bug or desired feature.