Moot::Waste::Lexer - libmoot : WASTE tokenizer : mid-level lexer
use Moot::Waste::Lexer; ##===================================================================== ## Usage $wl = Moot::Waste::Lexer->new(); ##-- create a new lexer $wl->scanner($scanner); ##-- set low-level TokenReader object (e.g. Moot::WasteScanner) $wl->scanner(); ##-- get underlying scanner or undef $wl->close(); ##-- close current input source (unsets scanner) $wl->dehyphenate($bool); ##-- enable/disable automatic dehyhpenation $tok = $wl->get_token(); ##-- read next token $buf = $wl->get_sentence(); ##-- read all remaining tokens as a list #... or (almost) any other Moot::TokenReader method ##===================================================================== ## Lexica (see Moot::Waste::Lexicon) $lex = $wl->stopwords(); $lex = $wl->abbrevs(); $lex = $wl->conjunctions();
The Moot::Waste::Lexer module provides an object-oriented interface to the WASTE tokenization system's mid-level rule-based segment classification stage.
Moot(3perl), Moot::Waste(3perl), Moot::Waste::Scanner(3perl), waste(1), perl(1).
Bryan Jurish <moocow@cpan.org>
Copyright (C) 2013 by Bryan Jurish
This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.14.2 or, at your option, any later version of Perl 5 you may have available.
To install Moot, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Moot
CPAN shell
perl -MCPAN -e shell install Moot
For more information on module installation, please visit the detailed CPAN module installation guide.