NAME

Mojo::UserAgent - Non-blocking I/O HTTP and WebSocket user agent

SYNOPSIS

  use Mojo::UserAgent;

  # Fine grained response handling (dies on connection errors)
  my $ua  = Mojo::UserAgent->new;
  my $res = $ua->get('docs.mojolicious.org')->result;
  if    ($res->is_success)  { say $res->body }
  elsif ($res->is_error)    { say $res->message }
  elsif ($res->code == 301) { say $res->headers->location }
  else                      { say 'Whatever...' }

  # Say hello to the Unicode snowman and include an Accept header
  say $ua->get('www.☃.net?hello=there' => {Accept => '*/*'})->result->body;

  # Extract data from HTML and XML resources with CSS selectors
  say $ua->get('www.perl.org')->result->dom->at('title')->text;

  # Scrape the latest headlines from a news site
  say $ua->get('blogs.perl.org')->result->dom->find('h2 > a')->map('text')->join("\n");

  # IPv6 PUT request with Content-Type header and content
  my $tx = $ua->put('[::1]:3000' => {'Content-Type' => 'text/plain'} => 'Hi!');

  # Quick JSON API request with Basic authentication
  my $url = Mojo::URL->new('https://example.com/test.json')->userinfo('sri:☃');
  my $value = $ua->get($url)->result->json;

  # JSON POST (application/json) with TLS certificate authentication
  my $tx = $ua->cert('tls.crt')->key('tls.key')->post('https://example.com' => json => {top => 'secret'});

  # Form POST (application/x-www-form-urlencoded)
  my $tx = $ua->post('https://metacpan.org/search' => form => {q => 'mojo'});

  # Search DuckDuckGo anonymously through Tor
  $ua->proxy->http('socks://127.0.0.1:9050');
  say $ua->get('api.3g2upl4pq6kufc4m.onion/?q=mojolicious&format=json')->result->json('/Abstract');

  # GET request via UNIX domain socket "/tmp/myapp.sock" (percent encoded slash)
  say $ua->get('http+unix://%2Ftmp%2Fmyapp.sock/test')->result->body;

  # Follow redirects to download Mojolicious from GitHub
  $ua->max_redirects(5)
    ->get('https://www.github.com/mojolicious/mojo/tarball/main')
    ->result->save_to('/home/sri/mojo.tar.gz');

  # Non-blocking request
  $ua->get('mojolicious.org' => sub ($ua, $tx) { say $tx->result->dom->at('title')->text });
  Mojo::IOLoop->start unless Mojo::IOLoop->is_running;

  # Concurrent non-blocking requests (synchronized with promises)
  my $mojo_promise = $ua->get_p('mojolicious.org');
  my $cpan_promise = $ua->get_p('cpan.org');
  Mojo::Promise->all($mojo_promise, $cpan_promise)->then(sub ($mojo, $cpan) {
    say $mojo->[0]->result->dom->at('title')->text;
    say $cpan->[0]->result->dom->at('title')->text;
  })->wait;

  # WebSocket connection sending and receiving JSON via UNIX domain socket
  $ua->websocket('ws+unix://%2Ftmp%2Fmyapp.sock/echo.json' => sub ($ua, $tx) {
    say 'WebSocket handshake failed!' and return unless $tx->is_websocket;
    $tx->on(json => sub ($tx, $hash) {
      say "WebSocket message via JSON: $hash->{msg}";
      $tx->finish;
    });
    $tx->send({json => {msg => 'Hello World!'}});
  });
  Mojo::IOLoop->start unless Mojo::IOLoop->is_running;

DESCRIPTION

Mojo::UserAgent is a full featured non-blocking I/O HTTP and WebSocket user agent, with IPv6, TLS, SNI, IDNA, HTTP/SOCKS5 proxy, UNIX domain socket, Comet (long polling), Promises/A+, keep-alive, connection pooling, timeout, cookie, multipart, gzip compression and multiple event loop support.

All connections will be reset automatically if a new process has been forked, this allows multiple processes to share the same Mojo::UserAgent object safely.

For better scalability (epoll, kqueue) and to provide non-blocking name resolution, SOCKS5 as well as TLS support, the optional modules EV (4.32+), Net::DNS::Native (0.15+), IO::Socket::Socks (0.64+) and IO::Socket::SSL (2.009+) will be used automatically if possible. Individual features can also be disabled with the MOJO_NO_NNR, MOJO_NO_SOCKS and MOJO_NO_TLS environment variables.

See "USER AGENT" in Mojolicious::Guides::Cookbook for more.

EVENTS

Mojo::UserAgent inherits all events from Mojo::EventEmitter and can emit the following new ones.

prepare

  $ua->on(prepare => sub ($ua, $tx) {...});

Emitted whenever a new transaction is being prepared, before relative URLs are rewritten and cookies added. This includes automatically prepared proxy CONNECT requests and followed redirects.

  $ua->on(prepare => sub ($ua, $tx) {
    $tx->req->url(Mojo::URL->new('/mock-mojolicious')) if $tx->req->url->host eq 'mojolicious.org';
  });

start

  $ua->on(start => sub ($ua, $tx) {...});

Emitted whenever a new transaction is about to start. This includes automatically prepared proxy CONNECT requests and followed redirects.

  $ua->on(start => sub ($ua, $tx) {
    $tx->req->headers->header('X-Bender' => 'Bite my shiny metal ass!');
  });

ATTRIBUTES

Mojo::UserAgent implements the following attributes.

ca

  my $ca = $ua->ca;
  $ua    = $ua->ca('/etc/tls/ca.crt');

Path to TLS certificate authority file used to verify the peer certificate, defaults to the value of the MOJO_CA_FILE environment variable.

  # Show certificate authorities for debugging
  IO::Socket::SSL::set_defaults(SSL_verify_callback => sub { say "Authority: $_[2]" and return $_[0] });

cert

  my $cert = $ua->cert;
  $ua      = $ua->cert('/etc/tls/client.crt');

Path to TLS certificate file, defaults to the value of the MOJO_CERT_FILE environment variable.

connect_timeout

  my $timeout = $ua->connect_timeout;
  $ua         = $ua->connect_timeout(5);

Maximum amount of time in seconds establishing a connection may take before getting canceled, defaults to the value of the MOJO_CONNECT_TIMEOUT environment variable or 10.

  my $cookie_jar = $ua->cookie_jar;
  $ua            = $ua->cookie_jar(Mojo::UserAgent::CookieJar->new);

Cookie jar to use for requests performed by this user agent, defaults to a Mojo::UserAgent::CookieJar object.

  # Ignore all cookies
  $ua->cookie_jar->ignore(sub { 1 });

  # Ignore cookies for public suffixes
  my $ps = IO::Socket::SSL::PublicSuffix->default;
  $ua->cookie_jar->ignore(sub ($cookie) {
    return undef unless my $domain = $cookie->domain;
    return ($ps->public_suffix($domain))[0] eq '';
  });

  # Add custom cookie to the jar
  $ua->cookie_jar->add(
    Mojo::Cookie::Response->new(
      name   => 'foo',
      value  => 'bar',
      domain => 'docs.mojolicious.org',
      path   => '/Mojolicious'
    )
  );

inactivity_timeout

  my $timeout = $ua->inactivity_timeout;
  $ua         = $ua->inactivity_timeout(15);

Maximum amount of time in seconds a connection can be inactive before getting closed, defaults to the value of the MOJO_INACTIVITY_TIMEOUT environment variable or 40. Setting the value to 0 will allow connections to be inactive indefinitely.

insecure

  my $bool = $ua->insecure;
  $ua      = $ua->insecure($bool);

Do not require a valid TLS certificate to access HTTPS/WSS sites, defaults to the value of the MOJO_INSECURE environment variable.

  # Disable TLS certificate verification for testing
  say $ua->insecure(1)->get('https://127.0.0.1:3000')->result->code;

ioloop

  my $loop = $ua->ioloop;
  $ua      = $ua->ioloop(Mojo::IOLoop->new);

Event loop object to use for blocking I/O operations, defaults to a Mojo::IOLoop object.

key

  my $key = $ua->key;
  $ua     = $ua->key('/etc/tls/client.crt');

Path to TLS key file, defaults to the value of the MOJO_KEY_FILE environment variable.

max_connections

  my $max = $ua->max_connections;
  $ua     = $ua->max_connections(5);

Maximum number of keep-alive connections that the user agent will retain before it starts closing the oldest ones, defaults to 5. Setting the value to 0 will prevent any connections from being kept alive.

max_redirects

  my $max = $ua->max_redirects;
  $ua     = $ua->max_redirects(3);

Maximum number of redirects the user agent will follow before it fails, defaults to the value of the MOJO_MAX_REDIRECTS environment variable or 0.

max_response_size

  my $max = $ua->max_response_size;
  $ua     = $ua->max_response_size(16777216);

Maximum response size in bytes, defaults to the value of "max_message_size" in Mojo::Message::Response. Setting the value to 0 will allow responses of indefinite size. Note that increasing this value can also drastically increase memory usage, should you for example attempt to parse an excessively large response body with the methods "dom" in Mojo::Message or "json" in Mojo::Message.

proxy

  my $proxy = $ua->proxy;
  $ua       = $ua->proxy(Mojo::UserAgent::Proxy->new);

Proxy manager, defaults to a Mojo::UserAgent::Proxy object.

  # Detect proxy servers from environment
  $ua->proxy->detect;

  # Manually configure HTTP proxy (using CONNECT for HTTPS/WebSockets)
  $ua->proxy->http('http://127.0.0.1:8080')->https('http://127.0.0.1:8080');

  # Manually configure Tor (SOCKS5)
  $ua->proxy->http('socks://127.0.0.1:9050')->https('socks://127.0.0.1:9050');

  # Manually configure UNIX domain socket (using CONNECT for HTTPS/WebSockets)
  $ua->proxy->http('http+unix://%2Ftmp%2Fproxy.sock') ->https('http+unix://%2Ftmp%2Fproxy.sock');

request_timeout

  my $timeout = $ua->request_timeout;
  $ua         = $ua->request_timeout(5);

Maximum amount of time in seconds establishing a connection, sending the request and receiving a whole response may take before getting canceled, defaults to the value of the MOJO_REQUEST_TIMEOUT environment variable or 0. Setting the value to 0 will allow the user agent to wait indefinitely. The timeout will reset for every followed redirect.

  # Total limit of 5 seconds, of which 3 seconds may be spent connecting
  $ua->max_redirects(0)->connect_timeout(3)->request_timeout(5);

server

  my $server = $ua->server;
  $ua        = $ua->server(Mojo::UserAgent::Server->new);

Application server relative URLs will be processed with, defaults to a Mojo::UserAgent::Server object.

  # Mock web service
  $ua->server->app(Mojolicious->new);
  $ua->server->app->routes->get('/time' => sub ($c) {
    $c->render(json => {now => time});
  });
  my $time = $ua->get('/time')->result->json->{now};

  # Change log level
  $ua->server->app->log->level('fatal');

  # Port currently used for processing relative URLs blocking
  say $ua->server->url->port;

  # Port currently used for processing relative URLs non-blocking
  say $ua->server->nb_url->port;

socket_options

  my $options = $ua->socket_options;
  $ua         = $ua->socket_options({LocalAddr => '127.0.0.1'});

Additional options for IO::Socket::IP when opening new connections.

tls_options

  my $options = $ua->tls_options;
  $ua         = $ua->tls_options({SSL_cipher_list => 'DEFAULT:!DH@SECLEVEL=1'});

Additional options for IO::Socket::SSL when opening new connections.

transactor

  my $t = $ua->transactor;
  $ua   = $ua->transactor(Mojo::UserAgent::Transactor->new);

Transaction builder, defaults to a Mojo::UserAgent::Transactor object.

  # Change name of user agent
  $ua->transactor->name('MyUA 1.0');

  # Disable compression
  $ua->transactor->compressed(0);

METHODS

Mojo::UserAgent inherits all methods from Mojo::EventEmitter and implements the following new ones.

build_tx

  my $tx = $ua->build_tx(GET => 'example.com');
  my $tx = $ua->build_tx(PUT => 'http://example.com' => {Accept => '*/*'} => 'Content!');
  my $tx = $ua->build_tx(PUT => 'http://example.com' => {Accept => '*/*'} => form => {a => 'b'});
  my $tx = $ua->build_tx(PUT => 'http://example.com' => {Accept => '*/*'} => json => {a => 'b'});

Generate Mojo::Transaction::HTTP object with "tx" in Mojo::UserAgent::Transactor.

  # Request with custom cookie
  my $tx = $ua->build_tx(GET => 'https://example.com/account');
  $tx->req->cookies({name => 'user', value => 'sri'});
  $tx = $ua->start($tx);

  # Deactivate gzip compression
  my $tx = $ua->build_tx(GET => 'example.com');
  $tx->req->headers->remove('Accept-Encoding');
  $tx = $ua->start($tx);

  # Interrupt response by raising an error
  my $tx = $ua->build_tx(GET => 'http://example.com');
  $tx->res->on(progress => sub ($res) {
    return unless my $server = $res->headers->server;
    $res->error({message => 'Oh noes, it is IIS!'}) if $server =~ /IIS/;
  });
  $tx = $ua->start($tx);

build_websocket_tx

  my $tx = $ua->build_websocket_tx('ws://example.com');
  my $tx = $ua->build_websocket_tx( 'ws://example.com' => {DNT => 1} => ['v1.proto']);

Generate Mojo::Transaction::HTTP object with "websocket" in Mojo::UserAgent::Transactor.

  # Custom WebSocket handshake with cookie
  my $tx = $ua->build_websocket_tx('wss://example.com/echo');
  $tx->req->cookies({name => 'user', value => 'sri'});
  $ua->start($tx => sub ($ua, $tx) {
    say 'WebSocket handshake failed!' and return unless $tx->is_websocket;
    $tx->on(message => sub ($tx, $msg) {
      say "WebSocket message: $msg";
      $tx->finish;
    });
    $tx->send('Hi!');
  });
  Mojo::IOLoop->start unless Mojo::IOLoop->is_running;

delete

  my $tx = $ua->delete('example.com');
  my $tx = $ua->delete('http://example.com' => {Accept => '*/*'} => 'Content!');
  my $tx = $ua->delete('http://example.com' => {Accept => '*/*'} => form => {a => 'b'});
  my $tx = $ua->delete('http://example.com' => {Accept => '*/*'} => json => {a => 'b'});

Perform blocking DELETE request and return resulting Mojo::Transaction::HTTP object, takes the same arguments as "tx" in Mojo::UserAgent::Transactor (except for the DELETE method, which is implied). You can also append a callback to perform requests non-blocking.

  $ua->delete('http://example.com' => json => {a => 'b'} => sub ($ua, $tx) { say $tx->result->body });
  Mojo::IOLoop->start unless Mojo::IOLoop->is_running;

delete_p

  my $promise = $ua->delete_p('http://example.com');

Same as "delete", but performs all requests non-blocking and returns a Mojo::Promise object instead of accepting a callback.

  $ua->delete_p('http://example.com' => json => {a => 'b'})->then(sub ($tx) {
    say $tx->result->body;
  })->catch(sub ($err) {
    warn "Connection error: $err";
  })->wait;

get

  my $tx = $ua->get('example.com');
  my $tx = $ua->get('http://example.com' => {Accept => '*/*'} => 'Content!');
  my $tx = $ua->get('http://example.com' => {Accept => '*/*'} => form => {a => 'b'});
  my $tx = $ua->get('http://example.com' => {Accept => '*/*'} => json => {a => 'b'});

Perform blocking GET request and return resulting Mojo::Transaction::HTTP object, takes the same arguments as "tx" in Mojo::UserAgent::Transactor (except for the GET method, which is implied). You can also append a callback to perform requests non-blocking.

  $ua->get('http://example.com' => json => {a => 'b'} => sub ($ua, $tx) { say $tx->result->body });
  Mojo::IOLoop->start unless Mojo::IOLoop->is_running;

get_p

  my $promise = $ua->get_p('http://example.com');

Same as "get", but performs all requests non-blocking and returns a Mojo::Promise object instead of accepting a callback.

  $ua->get_p('http://example.com' => json => {a => 'b'})->then(sub ($tx) {
    say $tx->result->body;
  })->catch(sub ($err) {
    warn "Connection error: $err";
  })->wait;
  my $tx = $ua->head('example.com');
  my $tx = $ua->head('http://example.com' => {Accept => '*/*'} => 'Content!');
  my $tx = $ua->head('http://example.com' => {Accept => '*/*'} => form => {a => 'b'});
  my $tx = $ua->head('http://example.com' => {Accept => '*/*'} => json => {a => 'b'});

Perform blocking HEAD request and return resulting Mojo::Transaction::HTTP object, takes the same arguments as "tx" in Mojo::UserAgent::Transactor (except for the HEAD method, which is implied). You can also append a callback to perform requests non-blocking.

  $ua->head('http://example.com' => json => {a => 'b'} => sub ($ua, $tx) { say $tx->result->body });
  Mojo::IOLoop->start unless Mojo::IOLoop->is_running;

head_p

  my $promise = $ua->head_p('http://example.com');

Same as "head", but performs all requests non-blocking and returns a Mojo::Promise object instead of accepting a callback.

  $ua->head_p('http://example.com' => json => {a => 'b'})->then(sub ($tx) {
    say $tx->result->body;
  })->catch(sub ($err) {
    warn "Connection error: $err";
  })->wait;

options

  my $tx = $ua->options('example.com');
  my $tx = $ua->options('http://example.com' => {Accept => '*/*'} => 'Content!');
  my $tx = $ua->options('http://example.com' => {Accept => '*/*'} => form => {a => 'b'});
  my $tx = $ua->options('http://example.com' => {Accept => '*/*'} => json => {a => 'b'});

Perform blocking OPTIONS request and return resulting Mojo::Transaction::HTTP object, takes the same arguments as "tx" in Mojo::UserAgent::Transactor (except for the OPTIONS method, which is implied). You can also append a callback to perform requests non-blocking.

  $ua->options('http://example.com' => json => {a => 'b'} => sub ($ua, $tx) { say $tx->result->body });
  Mojo::IOLoop->start unless Mojo::IOLoop->is_running;

options_p

  my $promise = $ua->options_p('http://example.com');

Same as "options", but performs all requests non-blocking and returns a Mojo::Promise object instead of accepting a callback.

  $ua->options_p('http://example.com' => json => {a => 'b'})->then(sub ($tx) {
    say $tx->result->body;
  })->catch(sub ($err) {
    warn "Connection error: $err";
  })->wait;

patch

  my $tx = $ua->patch('example.com');
  my $tx = $ua->patch('http://example.com' => {Accept => '*/*'} => 'Content!');
  my $tx = $ua->patch('http://example.com' => {Accept => '*/*'} => form => {a => 'b'});
  my $tx = $ua->patch('http://example.com' => {Accept => '*/*'} => json => {a => 'b'});

Perform blocking PATCH request and return resulting Mojo::Transaction::HTTP object, takes the same arguments as "tx" in Mojo::UserAgent::Transactor (except for the PATCH method, which is implied). You can also append a callback to perform requests non-blocking.

  $ua->patch('http://example.com' => json => {a => 'b'} => sub ($ua, $tx) { say $tx->result->body });
  Mojo::IOLoop->start unless Mojo::IOLoop->is_running;

patch_p

  my $promise = $ua->patch_p('http://example.com');

Same as "patch", but performs all requests non-blocking and returns a Mojo::Promise object instead of accepting a callback.

  $ua->patch_p('http://example.com' => json => {a => 'b'})->then(sub ($tx) {
    say $tx->result->body;
  })->catch(sub ($err) {
    warn "Connection error: $err";
  })->wait;

post

  my $tx = $ua->post('example.com');
  my $tx = $ua->post('http://example.com' => {Accept => '*/*'} => 'Content!');
  my $tx = $ua->post('http://example.com' => {Accept => '*/*'} => form => {a => 'b'});
  my $tx = $ua->post('http://example.com' => {Accept => '*/*'} => json => {a => 'b'});

Perform blocking POST request and return resulting Mojo::Transaction::HTTP object, takes the same arguments as "tx" in Mojo::UserAgent::Transactor (except for the POST method, which is implied). You can also append a callback to perform requests non-blocking.

  $ua->post('http://example.com' => json => {a => 'b'} => sub ($ua, $tx) { say $tx->result->body });
  Mojo::IOLoop->start unless Mojo::IOLoop->is_running;

post_p

  my $promise = $ua->post_p('http://example.com');

Same as "post", but performs all requests non-blocking and returns a Mojo::Promise object instead of accepting a callback.

  $ua->post_p('http://example.com' => json => {a => 'b'})->then(sub ($tx) {
    say $tx->result->body;
  })->catch(sub ($err) {
    warn "Connection error: $err";
  })->wait;

put

  my $tx = $ua->put('example.com');
  my $tx = $ua->put('http://example.com' => {Accept => '*/*'} => 'Content!');
  my $tx = $ua->put('http://example.com' => {Accept => '*/*'} => form => {a => 'b'});
  my $tx = $ua->put('http://example.com' => {Accept => '*/*'} => json => {a => 'b'});

Perform blocking PUT request and return resulting Mojo::Transaction::HTTP object, takes the same arguments as "tx" in Mojo::UserAgent::Transactor (except for the PUT method, which is implied). You can also append a callback to perform requests non-blocking.

  $ua->put('http://example.com' => json => {a => 'b'} => sub ($ua, $tx) { say $tx->result->body });
  Mojo::IOLoop->start unless Mojo::IOLoop->is_running;

put_p

  my $promise = $ua->put_p('http://example.com');

Same as "put", but performs all requests non-blocking and returns a Mojo::Promise object instead of accepting a callback.

  $ua->put_p('http://example.com' => json => {a => 'b'})->then(sub ($tx) {
    say $tx->result->body;
  })->catch(sub ($err) {
    warn "Connection error: $err";
  })->wait;

start

  my $tx = $ua->start(Mojo::Transaction::HTTP->new);

Perform blocking request for a custom Mojo::Transaction::HTTP object, which can be prepared manually or with "build_tx". You can also append a callback to perform requests non-blocking.

  my $tx = $ua->build_tx(GET => 'http://example.com');
  $ua->start($tx => sub ($ua, $tx) { say $tx->result->body });
  Mojo::IOLoop->start unless Mojo::IOLoop->is_running;

start_p

  my $promise = $ua->start_p(Mojo::Transaction::HTTP->new);

Same as "start", but performs all requests non-blocking and returns a Mojo::Promise object instead of accepting a callback.

  my $tx = $ua->build_tx(GET => 'http://example.com');
  $ua->start_p($tx)->then(sub ($tx) {
    say $tx->result->body;
  })->catch(sub ($err) {
    warn "Connection error: $err";
  })->wait;

websocket

  $ua->websocket('ws://example.com' => sub {...});
  $ua->websocket('ws://example.com' => {DNT => 1} => ['v1.proto'] => sub {...});

Open a non-blocking WebSocket connection with transparent handshake, takes the same arguments as "websocket" in Mojo::UserAgent::Transactor. The callback will receive either a Mojo::Transaction::WebSocket or Mojo::Transaction::HTTP object, depending on if the handshake was successful.

  $ua->websocket('wss://example.com/echo' => ['v1.proto'] => sub ($ua, $tx) {
    say 'WebSocket handshake failed!' and return unless $tx->is_websocket;
    say 'Subprotocol negotiation failed!' and return unless $tx->protocol;
    $tx->on(finish => sub ($tx, $code, $reason) { say "WebSocket closed with status $code." });
    $tx->on(message => sub ($tx, $msg) {
      say "WebSocket message: $msg";
      $tx->finish;
    });
    $tx->send('Hi!');
  });
  Mojo::IOLoop->start unless Mojo::IOLoop->is_running;

You can activate permessage-deflate compression by setting the Sec-WebSocket-Extensions header, this can result in much better performance, but also increases memory usage by up to 300KiB per connection.

  $ua->websocket('ws://example.com/foo' => {
    'Sec-WebSocket-Extensions' => 'permessage-deflate'
  } => sub {...});

websocket_p

  my $promise = $ua->websocket_p('ws://example.com');

Same as "websocket", but returns a Mojo::Promise object instead of accepting a callback.

  $ua->websocket_p('wss://example.com/echo')->then(sub ($tx) {
    my $promise = Mojo::Promise->new;
    $tx->on(finish => sub { $promise->resolve });
    $tx->on(message => sub ($tx, $msg) {
      say "WebSocket message: $msg";
      $tx->finish;
    });
    $tx->send('Hi!');
    return $promise;
  })->catch(sub ($err) {
    warn "WebSocket error: $err";
  })->wait;

DEBUGGING

You can set the MOJO_CLIENT_DEBUG environment variable to get some advanced diagnostics information printed to STDERR.

  MOJO_CLIENT_DEBUG=1

SEE ALSO

Mojolicious, Mojolicious::Guides, https://mojolicious.org.