JSON::Tokenize - tokenize a string containing JSON
use JSON::Tokenize ':all'; my $input = '{"tuttie":["fruity", true, 100]}'; my $token = tokenize_json ($input); print_tokens ($token, 0); sub print_tokens { my ($token, $depth) = @_; while ($token) { my $start = tokenize_start ($token); my $end = tokenize_end ($token); my $type = tokenize_type ($token); print " " x $depth; my $value = substr ($input, $start, $end - $start); print ">>$value<< has type $type\n"; my $child = tokenize_child ($token); if ($child) { print_tokens ($child, $depth+1); } my $next = tokenize_next ($token); $token = $next; } }
This outputs
>>{"tuttie":["fruity", true, 100]}<< has type object >>"tuttie"<< has type string >>:<< has type colon >>["fruity", true, 100]<< has type array >>"fruity"<< has type string >>,<< has type comma >>true<< has type literal >>,<< has type comma >>100<< has type number
This documents version 0.55 of JSON::Tokenize corresponding to git commit 739c1e12e85756c703242229caef685615ba77ca released on Tue Oct 24 07:11:47 2017 +0900.
This is a module for tokenizing a JSON string. It breaks the string into individual tokens without creating any Perl structures. Thus it can be used for tasks such as picking out or searching through parts of a large JSON structure without storing each part of the entire structure as individual Perl variables in memory.
This module is an experimental part of JSON::Parse and its interface is likely to change. The tokenizing functions are currently written in a very primitive way.
my $token = tokenize_json ($json);
my $next = tokenize_next ($token);
Walk the tree of tokens.
my $child = tokenize_child ($child);
my $start = tokenize_start ($token);
Get the start of the token as a byte offset from the start of the string. Note this is a byte offset not a character offset.
my $end = tokenize_end ($token);
Get the end of the token as a byte offset from the start of the string. Note this is a byte offset not a character offset.
my $type = tokenize_type ($token);
Get the type of the token as a string. The possible return values are
"invalid", "initial state", "string", "number", "literal", "object", "array", "unicode escape"
my $text = tokenize_text ($json, $token);
Given a token $token from this parsing and the JSON in $json, return the text which corresponds to the token. This is a convenience function written in Perl which uses "tokenize_start" and "tokenize_end" and substr to get the string from $json.
$token
$json
substr
Ben Bullock, <bkb@cpan.org>
This package and associated files are copyright (C) 2016-2017 Ben Bullock.
You can use, copy, modify and redistribute this package and associated files under the Perl Artistic Licence or the GNU General Public Licence.
To install JSON::Parse, copy and paste the appropriate command in to your terminal.
cpanm
cpanm JSON::Parse
CPAN shell
perl -MCPAN -e shell install JSON::Parse
For more information on module installation, please visit the detailed CPAN module installation guide.