NAME
Database::BI::SafeUA - Fetch a URL only if it leads to the public internet
VERSION
0.023.1
SYNOPSIS
my $ua = Database::BI::SafeUA->new(private_hosts => ['intranet.example.com']);
my $res = $ua->get('https://example.com/table.html');
# http://localtest.me/ -> 403 "Refused: localtest.me is 127.0.0.1, which is this machine"
DESCRIPTION
An LWP::UserAgent for url: sources and /import. Every request, including each redirect (at most 5), is resolved, refused with a 403 response if any address is on this machine, a private or shared network, link-local (cloud metadata services), multicast or reserved, and then sent to the address that was checked. The name stays in the Host header, the TLS server name (SNI) and the certificate check, so DNS that answers differently on a second lookup (rebinding) changes nothing. An IPv4 address inside IPv6 (mapped, NAT64, compatible, 6to4) is checked as that IPv4 address. Only http and https are fetched.
METHODS
new
API SPECIFICATION
INPUT
private_hosts: arrayref of host names, optional; names (or *.suffix) that
may resolve to private addresses (an intranet). An address
written as such is never one of them.
resolve: code ref, optional; host -> list of address strings (tests)
anything else: as LWP::UserAgent->new (timeout 30, max_redirect 5 by default)
OUTPUT
The user agent.
send_request
As "send_request" in LWP::UserAgent, with the checks above. LWP calls it for every request and each redirect; callers use get and the rest as usual.
API SPECIFICATION
INPUT
$request: HTTP::Request
$arg, $size: as LWP::UserAgent (content file or callback, read size)
OUTPUT
An HTTP::Response whose request is the one given (not the pinned copy). A refused request gets 403 Refused: <why> and Client-Warning: Internal response, and nothing is sent: a scheme other than http or https, no host, a name that does not resolve, or any address blocked_address refuses (unless private_host_allowed allows the name).
private_host_allowed
$ua->private_host_allowed('reports.corp.example'); # 1 with private_hosts => ['*.corp.example']
API SPECIFICATION
INPUT
$host: a host name (case does not matter)
OUTPUT
1 when private_hosts names it, or a *.suffix entry ends it (*.corp.example matches a.corp.example, not corp.example or evilcorp.example), else 0. send_request asks this only for a name, never for an address written as one.
FUNCTIONS
blocked_address
my $why = Database::BI::SafeUA::blocked_address('10.1.2.3'); # 'a private network'
API SPECIFICATION
INPUT
$ip: an IPv4 or IPv6 address as text (brackets and a zone index are allowed)
OUTPUT
Why the address is refused, or undef when it is public. Text that is not an address is refused.
numeric_host
numeric_host('2130706433'); # '127.0.0.1'
numeric_host('example.com'); # undef
API SPECIFICATION
INPUT
$host: a URL's host
OUTPUT
The address the host is when it is written as one, in any form the resolver reads (127.1, 0x7f000001, 0177.0.0.1, [::1]), else undef.
resolve_host
API SPECIFICATION
INPUT
$host: a host name
OUTPUT
Its addresses (getaddrinfo), or an empty list.
AUTHOR
Nigel Horne <njh@nigelhorne.com>
LICENCE AND COPYRIGHT
Copyright 2026 Nigel Horne.
Usage is subject to the GPL2 licence terms.