/ 
Fetch-0.06
/ Fetch

NAME

Fetch - HTTP/2 Future-based user agent

VERSION

Version 0.06

SYNOPSIS

use Fetch;

# works out of the box - no event loop to set up
my $res = Fetch->new->get('https://example.com')->get;
print $res->content if $res->is_success;

# every call returns a Fetch::Future; ->get awaits it
my $ua  = Fetch->new(timeout => 10);
my $f   = $ua->post('https://api/things',
                    headers => { 'Content-Type' => 'application/json' },
                    body    => '{"name":"x"}');
my $res = $f->get;

# many requests concurrently on one loop
my @futs = map { $ua->get($_) } @urls;
Fetch::Future->needs_all(@futs)->get;
my @bodies = map { $_->get->content } @futs;

# a live WebSocket
my $ws = $ua->websocket('wss://host/socket')->get;
$ws->send('hello');
my $reply = $ws->next_message->get;

DESCRIPTION

Fetch is an HTTP user agent whose socket, TLS, HTTP/2 framing and HTTP/1.1 parsing hot path lives in vendored C, and whose asynchronous results are Fetch::Future objects that compose with the Hyperman event loop and other CPAN loops (IO::Async, AnyEvent) - or with nothing at all, since Fetch ships its own event loop (Fetch::Loop::Standalone) and uses it automatically.

HTTP/1.1 and HTTP/2 (ALPN-negotiated over TLS), over cleartext and TLS, with keep-alive connection pooling, redirect following, per-request timeouts, streaming response bodies, a cookie jar, JSON request/response helpers, and native WebSockets. Every request method returns a Fetch::Future; ->get on one awaits it, pumping whichever event loop is active (its own if none), so the same code serves both a simple synchronous call and thousands of requests multiplexed on one loop.

CONSTRUCTOR

new(%args)

my $ua = Fetch->new(
    timeout    => 10,
    tls_verify => 1,
    headers    => { 'Accept' => 'application/json' },
    cookie_jar => 1,
);

Create a user agent. All arguments are optional:

loop

The event loop to run on. Omit it and Fetch uses its own Fetch::Loop::Standalone (so ->get just works with no framework). Pass a raw IO::Async::Loop or Hyperman::Loop and it is wrapped automatically; pass the string 'AnyEvent' to drive AnyEvent; or pass a ready-made Fetch::Loop adapter. See "EVENT LOOPS".

headers

Default headers sent on every request, as a hashref, an arrayref of name => value pairs (duplicates preserved), or a Fetch::Headers. Per-request headers are merged on top (see "REQUEST OPTIONS").

agent

The User-Agent string. Defaults to "Fetch/$VERSION".

tls_verify

Whether to verify the peer certificate and hostname for https. Default true. Overridable per request.

timeout

Default per-request deadline in seconds (fractional allowed). 0 (the default) means no timeout. Overridable per request.

max_redirects

How many redirects to follow. Default 5; 0 disables following. Overridable per request.

keep_alive

Reuse connections via a keep-alive pool (default true). Set false to close every connection after one request.

pool_size

Maximum idle connections the keep-alive pool parks (default 32).

A Fetch::CookieJar to store and send cookies (applied across redirects), or a true scalar to create a fresh one. Default: no jar.

simple_response

Resolve requests to a plain unblessed hashref { status => ..., headers => [k, v, ...], content => ... } instead of a blessed Fetch::Response. Read fields directly ($res->{status}, $res->{content}) rather than via methods. This skips the response object's method dispatch on the hot path; the saving is small (a couple of percent) and only shows when you actually read the response, so reach for it when you are consuming millions of responses and want the leanest possible per-response cost. Default false.

REQUEST METHODS

Each returns a Fetch::Future that resolves to a Fetch::Response (or fails with an error string). They never block; call ->get on the future to await the result.

get / head / delete

my $f = $ua->get($url, %opt);

post / put

my $f = $ua->post($url, body => $bytes, %opt);

request($method, $url, %opt)

my $f = $ua->request('PATCH', $url, body => $bytes, %opt);

The general form the verb helpers dispatch to; use it for any method.

REQUEST OPTIONS

Passed as a trailing key => value list to any request method:

headers

Extra headers for this request - a hashref, an arrayref of pairs (keeping duplicate names, e.g. multiple X-* values), or a Fetch::Headers. Each named field overrides the agent default of the same name.

body

The request body (bytes). Content-Length is added automatically unless you set it yourself.

json

A Perl data structure to send as a JSON body: it is encoded and Content-Type: application/json is set (unless you gave your own). Takes precedence over body. Pair it with "json" in Fetch::Response to decode the reply. Encoding uses Cpanel::JSON::XS when installed, else core JSON::PP.

my $res  = $ua->post($url, json => { name => 'x', ok => \1 })->get;
my $data = $res->json;
timeout

Override the agent timeout for this request (seconds; 0 disables it).

tls_verify

Override certificate/hostname verification for this https request.

max_redirects

Override how many redirects this request follows.

on_body

A coderef called with each body chunk as it arrives, instead of buffering. Suits large downloads and server-sent events: the buffer is compacted so an endless stream does not grow memory, and the resolved response body is empty.

$ua->get($url, on_body => sub { my ($chunk) = @_; print $chunk })->get;
on_headers

A coderef called once, as soon as the response status line and headers have been parsed - before any on_body chunk - with the status code and the header list:

$ua->get($url,
    on_headers => sub { my ($status, $headers) = @_; ... },  # $headers: [k,v,...]
    on_body    => sub { my ($chunk) = @_; ... },
)->get;

This lets a streaming consumer act on the status and headers up front (for example, to open a downstream writer) rather than waiting for the whole response. Not fired for a WebSocket upgrade.

WEBSOCKETS

websocket($url, %opt)

my $ws = $ua->websocket('ws://host/echo')->get;   # or wss://

Open a WebSocket (RFC 6455). Returns a Fetch::Future that resolves, after the 101 handshake, to a Fetch::WebSocket for sending and receiving messages. Accepts ws:///wss:// (and http/https); tls_verify and timeout options apply to the handshake.

ACCESSORS

loop

The event-loop adapter this agent runs on.

The Fetch::CookieJar in use, or undef.

THE RESULT

Awaiting a request future yields a Fetch::Response with status, headers (a Fetch::Headers), content, header($name), is_success and is_redirect. A failed request (connection error, timeout, bad TLS, a rejected WebSocket upgrade) fails the future with a message; ->get rethrows it, or inspect $f->failure / $f->is_failed.

EVENT LOOPS

$future->get awaits by pumping the active loop, so a bare synchronous call needs no setup. Hand Fetch a loop to cooperate with an existing async program - requests then fly concurrently on that one loop without blocking it:

use IO::Async::Loop;
my $loop = IO::Async::Loop->new;
my $ua   = Fetch->new(loop => $loop);       # shares this loop
my $f    = $ua->get('https://example.com/');
$loop->loop_once until $f->is_ready;
my $res  = $f->get;

Supported loops: the built-in Fetch::Loop::Standalone, plus Fetch::Loop::IOAsync, Fetch::Loop::AnyEvent and Fetch::Loop::Hyperman.

C ABI

Fetch exposes a small C ABI so another XS module can drive it from C, with no per-request Perl round trip on the hot path. It is how Reverse::Proxy builds its upstream requests entirely in C. This is not part of the Perl API - if you are writing Perl, use the request methods above; the ABI is only for XS consumers.

The ABI is a versioned function pointer table. A consumer vendors a copy of the header fetch_abi.h (shipped in this distribution under include/fetch/) at a pinned FETCH_ABI_VERSION, then at boot resolves the table and checks the version:

#include "fetch_abi.h"   /* vendored, after the perl.h includes */

/* in BOOT: */
IV p = 0;
if (call_pv("Fetch::_abi_ptr", G_SCALAR) > 0) { SPAGAIN; p = POPi; PUTBACK; }
const fetch_abi *FETCH = NULL;
if (p) {
    const fetch_abi *a = INT2PTR(const fetch_abi *, p);
    if (a && a->abi_version == FETCH_ABI_VERSION) FETCH = a;
}
/* FETCH == NULL means the running Fetch is too old / mismatched; the
 * consumer decides whether that is a hard error or a Perl fallback. */

Fetch::_abi_ptr

Returns the address of Fetch's fetch_abi table as an IV. Call it once, at boot, and INT2PTR the result to a const fetch_abi *. A version mismatch must never be treated as a crash - compare abi_version first and fall back.

The table

fetch_abi (see fetch_abi.h for the exact signatures and ownership rules) holds, after abi_version:

ua_new(kv, nkv)

Construct a Fetch user agent from nkv flat key/value SVs (the same options new takes: loop, pool_size, tls_verify, timeout, agent, headers, cookie_jar, keep_alive, max_redirects, simple_response). Returns the blessed Fetch UA SV (+1 owned). A consumer may instead call Fetch->new from Perl once and cache the object; ua_new just removes that last Perl call.

request(ua_sv, method, url, hdrs, nhdrs, body, blen, timeout, max_redirects, map, ud)

Issue one HTTP request on ua_sv, building it entirely from C. The headers are a flat fetch_hdr array; max_redirects below zero means "use the UA default". Returns a Fetch::Future SV (+1 owned) - hand it to an awaiting server or call ->get on it. When the request settles, your map callback shapes the value the future resolves to (for a proxy, a PSGI [ status, \@headers, \@body ]).

res_parts(res, status, headers, body)

Pull the status, the flat header AV and the content SV out of an already-resolved response (a Fetch::Response or a simple_response hash) with no method dispatch. Any out-pointer may be NULL; the returned headers and body are borrowed. For the blocking (awaited) path.

request_stream(ua_sv, method, url, hdrs, nhdrs, body, blen, timeout, max_redirects, on_headers, on_body, on_done, ud)

Like request, but streams the response instead of buffering it: on_headers fires once up front with the status and header AV, on_body once per body chunk, and on_done at completion (with success/failure). Returns the request future SV (+1 owned) - keep it alive until on_done fires. Lets a consumer forward a large download or an endless SSE stream with flat memory.

tunnel_connect(host, port, tls, verify) and friends

A raw blocking upstream TCP connection Fetch owns, for a proxy's Upgrade/WebSocket tunnel where the consumer splices bytes both ways itself. When tls is true the connection reuses Fetch's own client SSL_CTX (SNI, and hostname/certificate verification when verify is true), so the consumer tunnels to a wss/https upstream without linking OpenSSL. This is what lets Reverse::Proxy tunnel to a TLS upstream.

Unlike the rest of the table, these six entries are pure C - they take no pTHX and touch no SV, so they can be called directly from inside a select() splice loop:

  • tunnel_connect(host, port, tls, verify) - open the connection; returns an opaque handle, or NULL on failure (DNS, connect, or TLS handshake).

  • tunnel_fd(conn) - the underlying socket fd, to hand to select()/poll().

  • tunnel_read(conn, buf, len) - read bytes; returns the count (>0), 0 at EOF, or -1 on error.

  • tunnel_write_all(conn, buf, len) - write the whole buffer; returns 0 when all bytes are written, -1 on error.

  • tunnel_pending(conn) - bytes already buffered inside the TLS layer; drain these before trusting select() readiness (always 0 for a plain connection).

  • tunnel_close(conn) - shut the connection down and free the handle.

SEE ALSO

Fetch::Response, Fetch::Headers, Fetch::CookieJar, Fetch::WebSocket, Fetch::Future, Fetch::Loop and the loop adapters Fetch::Loop::Standalone, Fetch::Loop::IOAsync, Fetch::Loop::AnyEvent, Fetch::Loop::Hyperman.

AUTHOR

LNATION <email@lnation.org>

LICENSE AND COPYRIGHT

This software is Copyright (c) 2026 by LNATION.

This is free software, licensed under the Artistic License 2.0.