PHP::Decode::Tokenizer
# Create an instance package SymTokenizer; use base 'PHP::Decode::Tokenizer'; my @tok; sub new { my ($class, %args) = @_; return $class->SUPER::new(%args); } sub add { my ($tab, $sym) = @_; push(@tok, $sym); } package main; my $parser = SymTokenizer->new(); # Tokenize functions my $line = '<?php echo "test"; ?>'; my $quote = $parser->tokenize_line($line); printf "tokens: %s\n", join(' ', @$tok);
The PHP::Decode::Tokenizer module is the tokenizer base class for a php parser.
Create a PHP::Decode::Tokenizer object. Arguments are passed in key => value pairs.
$parser = PHP::Decode::Tokenizer->new(%args)
The tokenizer class is designed to be used as base class for a parser which overrides at least the basic add() method.
The accepted arguments are:
Tokenize a php code string.
my $quote = $parser->tokenize_line($line, $quote);
The tokenize_line method can be called once for a complete php code string, or incremental for consecutive parts of a script. Internally the tokenizer keeps the 'inscript' and 'quote' state between subsequent calls. The initial 'inscript state can be set when 'new' is called. If the tokenizer should start in quoted mode, an optional opening 'quote' can be passed to the call.
The tokenize_line method returns undef if not in quoted mode after the input was processed, or the type of 'quote' to indicate quoted mode.
The tokenizer will call the add() methods of the parser sub-class for each token. If one of the following add-methods is not overridden, the tokenizer will call $self->add($sym) as a default:
Barnim Dzwillo @ Strato AG
To install PHP::Decode, copy and paste the appropriate command in to your terminal.
cpanm
cpanm PHP::Decode
CPAN shell
perl -MCPAN -e shell install PHP::Decode
For more information on module installation, please visit the detailed CPAN module installation guide.