Author image ☼ 林永忠 ☼

NAME

Plucene::Analysis::CJKAnalyzer - Analyzer for CJK texts

SYNOPSIS

        # isa Plucene::Analysis::Analyzer

        my Plucene::Analysis::CJKAnalyzer $wt 
                = Plucene::Analysis::CJKAnalyzer->new(@args);
                

DESCRIPTION

This is a text analyzer for analyzing CJK texts. Plucene does not support CJK texts natively. This module encodes terms in MIME::Base64 format to get around this problem. Texts are assumbed to be in UTF-8 encoding.

See "cjk.t" in t for more details.

METHODS

tokenstream

        my Plucene::Analysis::CJKAnalyzer Plucene::Analysis::ChineseAnalyzer->new(@args);

Creates a TokenStream which tokenizes all the text in the provided Reader.

SEE ALSO

Plucene

MIME::Base64

Plucene::Analysis::CJKTokenizer

COPYRIGHT

Copyright (C) 2006 by Yung-chung Lin (a.k.a. xern) <xern@cpan.org>

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself