NAME

Moot::Waste::Decoder - libmoot : WASTE tokenizer : post-Viterbi decoder

SYNOPSIS

  use Moot::Waste::Decoder;

  ##=====================================================================
  ## Usage

  $wd = Moot::Waste::Decoder->new();    ##-- create a new decoder

  $wd->sink($writer);                   ##-- set low-level TokenWriter object (e.g. Moot::TokenWriterNative)
  $wd->sink();                          ##-- get underlying TokenWriter or undef
  $wd->close();                         ##-- close current output channel (unsets sink, clears buffer)

  $wd->put_token($tok);                 ##-- decode next token

  #... or (almost) any other Moot::TokenWriter method

  ##=====================================================================
  ## Buffer Access (e.g. with no TokenWriter sink)

  $bool = $wd->buffer_empty();          ##-- check whether token-buffer is empty
  $size = $wd->buffer_size();           ##-- number of buffered tokens, O(N)

  $tok  = $wd->buffer_peek();           ##-- peek at first token of buffer
  $bool = $wd->buffer_can_shift();      ##-- true iff first token is safe to shift
  undef = $wd->buffer_shift();          ##-- shifts first element off of buffer
  $toks = $wd->buffer_flush($force=0);  ##-- get all (safe) tokens from buffer

DESCRIPTION

The Moot::Waste::Decoder module provides an object-oriented interface to the WASTE tokenization system's rule-based post-Viterbi decoder stage.

SEE ALSO

Moot(3perl), Moot::Waste(3perl), Moot::Waste::Scanner(3perl), Moot::Waste::Lexer(3perl), waste(1), perl(1).

AUTHOR

Bryan Jurish <moocow@cpan.org>

COPYRIGHT AND LICENSE

Copyright (C) 2013 by Bryan Jurish

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.14.2 or, at your option, any later version of Perl 5 you may have available.