Provided by: libcatmandu-importer-getjson-perl_0.51-1_all bug


       Catmandu::Importer::getJSON - load JSON-encoded data from a server using a GET HTTP


       The following three examples are equivalent:

               file => \"\n"
           )->each(sub { my ($record) = @_; ... );

               url  => "",
               file => \"/alice.json\n/bob.json"
           )->each(sub { my ($record) = @_; ... );

               url  => "{name}.json",
               file => \"{\"name\":\"alice\"}\n{\"name\":\"bob\"}"
           )->each(sub { my ($record) = @_; ... );

       For more convenience the catmandu command line client can be used:

           echo | catmandu convert getJSON to YAML
           catmandu convert getJSON --from to YAML
           catmandu convert getJSON --dry 1 --url http://{domain}/robots.txt < domains


       This Catmandu::Importer performs a HTTP GET request to load JSON-encoded data from a
       server. The importer expects a line-separated input. Each line corresponds to a HTTP
       request that is mapped to a JSON-record on success. The following input formats are

       plain URL
           A line that starts with ""http://"" or ""https://"" is used as plain URL.

       URL path
           A line that starts with ""/"" is appended to the configured url parameter.

           A JSON object with variables to be used with an URL template or as HTTP query
           parameters. For instance the input line "{"name":"Karl Marx"}" with URL
           "" or the input line "{"entity":"person","name":"Karl
           Marx"}" with URL template "{entity}{?id}{?name}{?q}" are both
           expanded to <>.

       If the JSON data returned in a HTTP response is a JSON array, its elements are imported as
       multiple items. If a JSON object is returned, it is imported as one item.


       url An URI or an URI templates (URI::Template) as defined by RFC 6570
           <> to load JSON from. If no url is configured, plain
           URLs must be provided as input or option "from" must be used instead.

           A plain URL to load JSON without reading any input lines.

       timeout / agent / proxy / headers
           Optional HTTP client settings.

           Instance of a Furl HTTP client to perform requests with.

       dry Don't do any HTTP requests but return URLs that data would be queried from.

       file / fh
           Input to read lines from (see Catmandu::Importer). Defaults to STDIN.

       fix An optional fix to be applied on every item (see Catmandu::Fix).

           Number of seconds to wait between requests.

           Cache JSON response of URLs to not request the same URL twice. HTTP error codes in the
           4xx range (e.g. 404) are also cached but 5xx errors are not.

           The value of this option can be any objects that implements method "get" and "set"
           (e.g. "CHI"), an existing directory for file caching, a true value to enable global
           in-memory-caching, or a false value to disable caching (default).

           File caching uses file names based on MD5 of an URL so for instance
           "" is cached as "4389382917e51695b759543fdfd5f690.json".

           Show error messages on the standard error.


       Returns the UNIX timestamp right before the last request. This can be used for instance to
       add timestamps or the measure how fast requests were responded.

   construct_url( [ $base_url, ] $vars_url_or_path )
       Returns an URL given a hash reference with variables, a plain URL or an URL path. The
       optional first argument can be used to override option "url".

           $importer->construct_url( %query_vars )
           $importer->construct_url( $importer->url, %query_vars ) # equivalent

       Perform a HTTP GET request of a given URL including logging, caching, request hook etc.
       Returns a hash/array reference or "undef".


       This importer provides two methods to filter requests and responses, respectively. See
       Catmandu::Importer::Wikidata for an example.

       Gets a whitespace-trimmed input line and is expected to return an unblessed hash
       reference, an URL, or undef. Errors are catched and treated equal to undef.

       Gets the queried response object and is expected to return an object.


       URLs are emitted before each request on DEBUG log level.


       Future versions of this module may also support asynchronous HTTP fetching modules such as
       HTTP::Async, for retrieving multiple URLs at the same time.


       Catmandu::Fix::get_json provides this importer as fix function.


       Copyright Jakob Voss, 2014-
       This library is free software; you can redistribute it and/or modify it under the same
       terms as Perl itself.