The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.
 / 
HTTP-Request-FromCurl-0.32
River stage zero No dependents
3 ++
/ HTTP::Request::FromWget

NAME

HTTP::Request::FromWget - create a HTTP::Request from a wget command line

SYNOPSIS

    my $req = HTTP::Request::FromWget->new(
        # Note - wget itself may not appear
        argv => ['https://example.com'],
    );

    my $req = HTTP::Request::FromWget->new(
        command => 'https://example.com',
    );

    my $req = HTTP::Request::FromWget->new(
        command_wget => 'wget -A mywget/1.0 https://example.com',
    );

    my @requests = HTTP::Request::FromWget->new(
        command_wget => 'wget -A mywget/1.0 https://example.com https://www.example.com',
    );
    # Send the requests
    for my $r (@requests) {
        $ua->request( $r->as_request )
    }

RATIONALE

wget command lines are found everywhere in documentation. The Firefox developer tools can also copy network requests as wget command lines from the network panel. This module enables converting these to Perl code.

METHODS

->new

    my $req = HTTP::Request::FromWget->new(
        # Note - wget itself may not appear
        argv => ['--user-agent', 'myscript/1.0', 'https://example.com'],
    );

    my $req = HTTP::Request::FromWget->new(
        # Note - wget itself may not appear
        command => '--user-agent myscript/1.0 https://example.com',
    );

The constructor returns one or more HTTP::Request::CurlParameters objects that encapsulate the parameters. If the command generates multiple requests, they will be returned in list context. In scalar context, only the first request will be returned.

    my $req = HTTP::Request::FromWget->new(
        command => '--post-file /etc/passwd https://example.com',
        read_files => 1,
    );

Options

argv

An arrayref of commands as could be given in @ARGV .

command

A scalar in a command line, excluding the wget command

command_wget

A scalar in a command line, including the wget command

read_files

Do read in the content of files specified with (for example) --data=@/etc/passwd. The default is to not read the contents of files specified this way.

GLOBAL VARIABLES

%default_headers

Contains the default headers added to every request

@option_spec

Contains the Getopt::Long specification of the recognized command line parameters.

The following wget options are recognized but largely ignored:

verbose
quiet
auth-no-challenge
output-document
debug

If you want to keep session cookies between subsequent requests, you need to provide a cookie jar in your user agent.

METHODS

->squash_uri( $uri )

    my $uri = HTTP::Request::FromWget->squash_uri(
        URI->new( 'https://example.com/foo/bar/..' )
    );
    # https://example.com/foo/

Helper method to clean up relative path elements from the URI the same way that wget does.

LIVE DEMO

https://corion.net/curl2lwp.psgi

KNOWN DIFFERENCES

Until somebody writes a robust Netscape cookie file parser and proper loading and storage for HTTP::CookieJar, this module will not be able to load and save files in the format that wget uses.

You're expected to instruct your UA to load/save cookie jars:

    use Path::Tiny;
    use HTTP::CookieJar::LWP;

    if( my $cookies = $r->cookie_jar ) {
        $ua->cookie_jar( HTTP::CookieJar::LWP->new()->load_cookies(
            path($cookies)->lines
        ));
    };

Different Content-Length for POST requests

Different delimiter for form data

The delimiter is built by HTTP::Message, and wget uses a different mechanism to come up with a unique data delimiter. This results in differences in the raw body content and the Content-Length header.

MISSING FUNCTIONALITY

  • File uploads / content from files

    While file uploads and reading POST data from files are supported, the content is slurped into memory completely. This can be problematic for large files and little available memory.

SEE ALSO

HTTP::Request::AsCurl - for the inverse function

The module HTTP::Request::AsCurl likely also implements a much better version of ->as_curl than this module.

REPOSITORY

The public repository of this module is http://github.com/Corion/HTTP-Request-FromWget.

SUPPORT

The public support forum of this module is https://perlmonks.org/.

BUG TRACKER

Please report bugs in this module via the Github bug queue at https://github.com/Corion/HTTP-Request-FromWget/issues

AUTHOR

Max Maischein corion@cpan.org

COPYRIGHT (c)

Copyright 2018-2021 by Max Maischein corion@cpan.org.

LICENSE

This module is released under the same terms as Perl itself.