Catmandu::Importer::getJSON - Load JSON-encoded data from a server using a GET HTTP request
version 0.40
The following three examples are equivalent:
Catmandu::Importer::getJSON->new( file => \"http://example.org/alice.json\nhttp://example.org/bob.json" )->each(sub { my ($record) = @_; ... ); Catmandu::Importer::getJSON->new( url => "http://example.org", file => \"/alice.json\n/bob.json" )->each(sub { my ($record) = @_; ... ); Catmandu::Importer::getJSON->new( url => "http://example.org/{name}.json", file => \"{\"name\":\"alice\"}\n{\"name\":\"bob\"}" )->each(sub { my ($record) = @_; ... );
For more convenience the catmandu command line client can be used:
echo http://example.org/alice.json | catmandu convert getJSON to YAML catmandu convert getJSON --from http://example.org/alice.json to YAML catmandu convert getJSON --dry 1 --url http://{domain}/robots.txt < domains
This Catmandu::Importer performs a HTTP GET request to load JSON-encoded data from a server. The importer expects a line-separated input. Each line corresponds to a HTTP request that is mapped to a JSON-record on success. The following input formats are accepted:
A line that starts with "http://" or "https://" is used as plain URL.
http://
https://
A line that starts with "/" is appended to the configured url parameter.
/
A JSON object with variables to be used with an URL template or as HTTP query parameters. For instance the input line {"name":"Karl Marx"} with URL http://api.lobid.org/person or the input line {"entity":"person","name":"Karl Marx"} with URL template http://api.lobid.org/{entity}{?id}{?name}{?q} are both expanded to http://api.lobid.org/person?name=Karl+Marx.
{"name":"Karl Marx"}
http://api.lobid.org/person
{"entity":"person","name":"Karl Marx"}
http://api.lobid.org/{entity}{?id}{?name}{?q}
If the JSON data returned in a HTTP response is a JSON array, its elements are imported as multiple items. If a JSON object is returned, it is imported as one item.
An URI or an URI templates (URI::Template) as defined by RFC 6570 to load JSON from. If no url is configured, plain URLs must be provided as input or option from must be used instead.
from
A plain URL to load JSON without reading any input lines.
Optional HTTP client settings.
Instance of a Furl HTTP client to perform requests with.
Don't do any HTTP requests but return URLs that data would be queried from.
Input to read lines from (see Catmandu::Importer). Defaults to STDIN.
An optional fix to be applied on every item (see Catmandu::Fix).
Number of seconds to wait between requests.
Returns the UNIX timestamp right before the last request. This can be used for instance to add timestamps or the measure how fast requests were responded.
This importer provides two methods to filter requests and responses, respectively. See Catmandu::Importer::Wikidata for an example.
Gets a whitespace-trimmed input line and is expected to return an unblessed object or an URL.
Gets the queried response object and is expected to return an object.
URLs are emitted before each request on DEBUG log level.
Error handling is very limited.
Future versions of this module may also support asynchronous HTTP fetching modules such as HTTP::Async, for retrieving multiple URLs at the same time..
Catmandu::Fix::get_json provides this importer as fix function.
Jakob Voß
This software is copyright (c) 2014 by Jakob Voß.
This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.
To install Catmandu::Importer::getJSON, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Catmandu::Importer::getJSON
CPAN shell
perl -MCPAN -e shell install Catmandu::Importer::getJSON
For more information on module installation, please visit the detailed CPAN module installation guide.