The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.

Changes for version 0.05

  • added support for whitespace tokens, feature requested by Stephan Tobias.
    • added tests for this
    • added documenation for this
  • fixed bug where the '0' token got ignored, also thanks to Stephan Tobias for this.

Modules

A simple string tokenizer.

Provides

in lib/String/Tokenizer.pm