Provided by: libnet-async-http-perl_0.48-1_all 
NAME
"Net::Async::HTTP" - use HTTP with "IO::Async"
SYNOPSIS
use Future::AsyncAwait;
use IO::Async::Loop;
use Net::Async::HTTP;
use URI;
my $loop = IO::Async::Loop->new();
my $http = Net::Async::HTTP->new();
$loop->add( $http );
my $response = await $http->do_request(
uri => URI->new( "http://www.cpan.org/" ),
);
print "Front page of http://www.cpan.org/ is:\n";
print $response->as_string;
DESCRIPTION
This object class implements an asynchronous HTTP user agent. It sends requests to
servers, returning Future instances to yield responses when they are received. The object
supports multiple concurrent connections to servers, and allows multiple requests in the
pipeline to any one connection. Normally, only one such object will be needed per program
to support any number of requests.
As well as using futures the module also supports a callback-based interface.
This module optionally supports SSL connections, if IO::Async::SSL is installed. If so,
SSL can be requested either by passing a URI with the "https" scheme, or by passing a true
value as the "SSL" parameter.
Connection Pooling
There are three ways in which connections to HTTP server hosts are managed by this object,
controlled by the value of "max_connections_per_host". This controls when new connections
are established to servers, as compared to waiting for existing connections to be free, as
new requests are made to them.
They are:
max_connections_per_host = 1
This is the default setting. In this mode, there will be one connection per host on
which there are active or pending requests. If new requests are made while an existing
one is outstanding, they will be queued to wait for it.
If pipelining is active on the connection (because both the "pipeline" option is true
and the connection is known to be an HTTP/1.1 server), then requests will be pipelined
into the connection awaiting their response. If not, they will be queued awaiting a
response to the previous before sending the next.
max_connections_per_host > 1
In this mode, there can be more than one connection per host. If a new request is made,
it will try to re-use idle connections if there are any, or if they are all busy it will
create a new connection to the host, up to the configured limit.
max_connections_per_host = 0
In this mode, there is no upper limit to the number of connections per host. Every new
request will try to reuse an idle connection, or else create a new one if all the
existing ones are busy.
These modes all apply per hostname / server port pair; they do not affect the behaviour of
connections made to differing hostnames, or differing ports on the same hostname.
PARAMETERS
The following named parameters may be passed to "new" or "configure":
user_agent => STRING
A string to set in the "User-Agent" HTTP header. If not supplied, one will be constructed
that declares "Net::Async::HTTP" and the version number.
headers => ARRAY or HASH
Since version 0.45.
A set of extra headers to apply to every outgoing request. May be specified either as an
even-sized array containing key/value pairs, or a hash.
Individual header values may be added or changed without replacing the entire set by using
the configure method and passing a key called "+headers":
$http->configure( +headers => { One_More => "Key" } );
max_redirects => INT
Optional. How many levels of redirection to follow. If not supplied, will default to 3.
Give 0 to disable redirection entirely.
max_in_flight => INT
Optional. The maximum number of in-flight requests to allow per host when pipelining is
enabled and supported on that host. If more requests are made over this limit they will be
queued internally by the object and not sent to the server until responses are received.
If not supplied, will default to 4. Give 0 to disable the limit entirely.
max_connections_per_host => INT
Optional. Controls the maximum number of connections per hostname/server port pair, before
requests will be queued awaiting one to be free. Give 0 to disable the limit entirely. See
also the "Connection Pooling" section documented above.
Currently, if not supplied it will default to 1. However, it has been found in practice
that most programs will raise this limit to something higher, perhaps 3 or 4. Therefore, a
future version of this module may set a higher value.
To test if your application will handle this correctly, you can set a different default by
setting an environment variable:
$ NET_ASYNC_HTTP_MAXCONNS=3 perl ...
timeout => NUM
Optional. How long in seconds to wait before giving up on a request. If not supplied then
no default will be applied, and no timeout will take place.
stall_timeout => NUM
Optional. How long in seconds to wait after each write or read of data on a socket, before
giving up on a request. This may be more useful than "timeout" on large-file operations,
as it will not time out provided that regular progress is still being made.
proxy_host => STRING
proxy_port => INT
Optional. Default values to apply to each "request" method.
cookie_jar => HTTP::Cookies
Optional. A reference to a HTTP::Cookies object. Will be used to set cookies in requests
and store them from responses.
pipeline => BOOL
Optional. If false, disables HTTP/1.1-style request pipelining.
close_after_request => BOOL
Since version 0.45.
Optional. If true, will set the "Connection: close" header on outgoing requests and
disable pipelining, thus making every request use a new connection.
family => INT
local_host => STRING
local_port => INT
local_addrs => ARRAY
local_addr => HASH or ARRAY
Optional. Parameters to pass on to the "connect" method used to connect sockets to HTTP
servers. Sets the socket family and local socket address to "bind()" to. For more detail,
see the documentation in IO::Async::Connector.
fail_on_error => BOOL
Optional. Affects the behaviour of response handling when a "4xx" or "5xx" response code
is received. When false, these responses will be processed as other responses and yielded
as the result of the future, or passed to the "on_response" callback. When true, such an
error response causes the future to fail, or the "on_error" callback to be invoked.
The HTTP response and request objects will be passed as well as the code and message, and
the failure name will be "http".
( $code_message, "http", $response, $request ) = $f->failure
$on_error->( "$code $message", $response, $request )
read_len => INT
write_len => INT
Optional. Used to set the reading and writing buffer lengths on the underlying
"IO::Async::Stream" objects that represent connections to the server. If not define, a
default of 64 KiB will be used.
ip_tos => INT or STRING
Optional. Used to set the "IP_TOS" socket option on client sockets. If given, should
either be a "IPTOS_*" constant, or one of the string names "lowdelay", "throughput",
"reliability" or "mincost". If undefined or left absent, no option will be set.
decode_content => BOOL
Optional. If true, incoming responses that have a recognised "Content-Encoding" are
handled by the module, and decompressed content is passed to the body handling callback or
returned in the "HTTP::Response". See "CONTENT DECODING" below for details of which
encoding types are recognised. When this option is enabled, outgoing requests also have
the "Accept-Encoding" header added to them if it does not already exist.
Currently the default is false, because this behaviour is new, but it may default to true
in a later version. Applications which care which behaviour applies should set this to a
defined value to ensure it doesn't change.
SSL_*
Additionally, any parameters whose names start with "SSL_" will be stored and passed on
requests to perform SSL requests. This simplifies configuration of common SSL parameters.
require_SSL => BOOL
Optional. If true, then any attempt to make a request that does not use SSL (either by
calling "request", or as a result of a redirection) will immediately fail.
SOCKS_*
Since version 0.42.
Additionally, any parameters whose names start with "SOCKS_" will be stored and used by
Net::Async::SOCKS to establish connections via a configured proxy.
METHODS
The following methods documented in an "await" expression return Future instances.
When returning a Future, the following methods all indicate HTTP-level errors using the
Future failure name of "http". If the error relates to a specific response it will be
included. The original request is also included.
$f->fail( $message, "http", $response, $request )
do_request
$response = await $http->do_request( %args );
Send an HTTP request to a server, returning a Future that will yield the response. The
request may be represented by an HTTP::Request object, or a URI object, depending on the
arguments passed.
The following named arguments are used for "HTTP::Request"s:
request => HTTP::Request
A reference to an "HTTP::Request" object
host => STRING
Hostname of the server to connect to
port => INT or STRING
Optional. Port number or service of the server to connect to. If not defined, will
default to "http" or "https" depending on whether SSL is being used.
family => INT
Optional. Restricts the socket family for connecting. If not defined, will default
to the globally-configured value in the object.
SSL => BOOL
Optional. If true, an SSL connection will be used.
The following named arguments are used for "URI" requests:
uri => URI or STRING
A reference to a "URI" object, or a plain string giving the request URI. If the
scheme is "https" then an SSL connection will be used.
method => STRING
Optional. The HTTP method name. If missing, "GET" is used.
content => STRING or ARRAY ref
Optional. The body content to use for "PUT" or "POST" requests.
If this is a plain scalar it will be used directly, and a "content_type" field
must also be supplied to describe it.
If this is an ARRAY ref and the request method is "POST", it will be form encoded.
It should contain an even-sized list of field names and values. For more detail
see "POST" in HTTP::Request::Common.
content_type => STRING
The type of non-form data "content".
user => STRING
pass => STRING
Optional. If both are given, the HTTP Basic Authorization header will be sent with
these details.
headers => ARRAY|HASH
Optional. If provided, contains additional HTTP headers to set on the constructed
request object. If provided as an ARRAY reference, it should contain an even-sized
list of name/value pairs.
proxy_host => STRING
proxy_port => INT
Optional. Override the hostname or port number implied by the URI.
For either request type, it takes the following arguments:
request_body => STRING | CODE | Future
Optional. Allows request body content to be generated by a future or callback,
rather than being provided as part of the "request" object. This can either be a
plain string, a "CODE" reference to a generator function, or a future.
As this is passed to the underlying IO::Async::Stream "write" method, the usual
semantics apply here. If passed a "CODE" reference, it will be called repeatedly
whenever it's safe to write. The code should should return "undef" to indicate
completion. If passed a "Future" it is expected to eventually yield the body
value.
As with the "content" parameter, the "content_type" field should be specified
explicitly in the request header, as should the content length (typically via the
HTTP::Request "content_length" method). See also examples/PUT.pl.
expect_continue => BOOL
Optional. If true, sets the "Expect" request header to the value "100-continue"
and does not send the "request_body" parameter until a "100 Continue" response is
received from the server. If an error response is received then the "request_body"
code, if present, will not be invoked.
on_ready => CODE
Optional. A callback that is invoked once a socket connection is established with
the HTTP server, but before the request is actually sent over it. This may be used
by the client code to inspect the socket, or perform any other operations on it.
This code is expected to return a "Future"; only once that has completed will the
request cycle continue. If it fails, that failure is propagated to the caller.
$f = $on_ready->( $connection )
on_redirect => CODE
Optional. A callback that is invoked if a redirect response is received, before
the new location is fetched. It will be passed the response and the new URL.
$on_redirect->( $response, $location )
on_body_write => CODE
Optional. A callback that is invoked after each successful "syswrite" of the body
content. This may be used to implement an upload progress indicator or similar. It
will be passed the total number of bytes of body content written so far (i.e.
excluding bytes consumed in the header).
$on_body_write->( $written )
max_redirects => INT
Optional. How many levels of redirection to follow. If not supplied, will default
to the value given in the constructor.
timeout => NUM
stall_timeout => NUM
Optional. Overrides the object's configured timeout values for this one request.
If not specified, will use the configured defaults.
On a timeout, the returned future will fail with either "timeout" or
"stall_timeout" as the operation name.
( $message, "timeout" ) = $f->failure
do_request (void)
$http->do_request( %args )
When not returning a future, the following extra arguments are used as callbacks instead:
on_response => CODE
A callback that is invoked when a response to this request has been received. It
will be passed an HTTP::Response object containing the response the server sent.
$on_response->( $response )
on_header => CODE
Alternative to "on_response". A callback that is invoked when the header of a
response has been received. It is expected to return a "CODE" reference for
handling chunks of body content. This "CODE" reference will be invoked with no
arguments once the end of the request has been reached, and whatever it returns
will be used as the result of the returned "Future", if there is one.
$on_body_chunk = $on_header->( $header )
$on_body_chunk->( $data )
$response = $on_body_chunk->()
on_error => CODE
A callback that is invoked if an error occurs while trying to send the request or
obtain the response. It will be passed an error message.
$on_error->( $message )
If this is invoked because of a received "4xx" or "5xx" error code in an HTTP
response, it will be invoked with the response and request objects as well.
$on_error->( $message, $response, $request )
GET, HEAD, PUT, ...
$response = await $http->GET( $uri, %args );
$response = await $http->HEAD( $uri, %args );
$response = await $http->PUT( $uri, $content, %args );
$response = await $http->POST( $uri, $content, %args );
$response = await $http->PATCH( $uri, $content, %args );
Convenient wrappers for performing "GET", "HEAD", "PUT", "POST" or "PATCH" requests with a
"URI" object and few if any other arguments, returning a "Future".
Remember that "POST" with non-form data (as indicated by a plain scalar instead of an
"ARRAY" reference of form data name/value pairs) needs a "content_type" key in %args.
SUBCLASS METHODS
The following methods are intended as points for subclasses to override, to add extra
functionallity.
prepare_request
$http->prepare_request( $request )
Called just before the "HTTP::Request" object is sent to the server.
process_response
$http->process_response( $response )
Called after a non-redirect "HTTP::Response" has been received from a server. The
originating request will be set in the object.
CONTENT DECODING
If the required decompression modules are installed and available, compressed content can
be decoded. If the received "Content-Encoding" is recognised and the required module is
available, the content is transparently decoded and the decoded content is returned in the
resulting response object, or passed to the data chunk handler. In this case, the original
"Content-Encoding" header will be deleted from the response, and its value will be
available instead as "X-Original-Content-Encoding".
The following content encoding types are recognised by these modules:
• gzip (q=0.7) and deflate (q=0.5)
Recognised if Compress::Raw::Zlib version 2.057 or newer is installed.
• bzip2 (q=0.8)
Recognised if Compress::Bzip2 version 2.10 or newer is installed.
Other content encoding types can be registered by calling the following method
register_decoder
Net::Async::HTTP->register_decoder( $name, $q, $make_decoder )
Registers an encoding type called $name, at the quality value $q. In order to decode this
encoding type, $make_decoder will be invoked with no paramters, and expected to return a
CODE reference to perform one instance of decoding.
$decoder = $make_decoder->()
This decoder will be invoked on string buffers to decode them until the end of stream is
reached, when it will be invoked with no arguments.
$content = $decoder->( $encoded_content )
$content = $decoder->() # EOS
EXAMPLES
Concurrent GET
The "Future"-returning "GET" method makes it easy to await multiple URLs at once, by using
the Future::Utils "fmap_void" utility
use Future::AsyncAwait;
use Future::Utils qw( fmap_void );
my @URLs = ( ... );
my $http = Net::Async::HTTP->new( ... );
$loop->add( $http );
my $future = fmap_void {
my ( $url ) = @_;
$http->GET( $url )
->on_done( sub {
my $response = shift;
say "$url succeeded: ", $response->code;
say " Content-Type:", $response->content_type;
} )
->on_fail( sub {
my $failure = shift;
say "$url failed: $failure";
} );
} foreach => \@URLs,
concurrent => 5;
await $future;
SEE ALSO
• <http://tools.ietf.org/html/rfc2616> - Hypertext Transfer Protocol -- HTTP/1.1
SPONSORS
Parts of this code, or bugfixes to it were paid for by
• SocialFlow <http://www.socialflow.com>
• Shadowcat Systems <http://www.shadow.cat>
• NET-A-PORTER <http://www.net-a-porter.com>
• Cisco <http://www.cisco.com>
AUTHOR
Paul Evans <leonerd@leonerd.org.uk>