KinoSearch::Analysis::Stopalizer - Suppress a "stoplist" of common words.
my $stopalizer = KinoSearch::Analysis::Stopalizer->new( language => 'fr', ); my $polyanalyzer = KinoSearch::Analysis::PolyAnalyzer->new( analyzers => [ $case_folder, $tokenizer, $stopalizer, $stemmer ], );
This class uses Lingua::StopWords for its default stoplists, so it supports the same set of languages.
A "stoplist" is collection of "stopwords": words which are common enough to be of little value when determining search results. For example, so many documents in English contain "the", "if", and "maybe" that it may improve both performance and relevance to block them.
Before filtering stopwords:
( "i", "am", "the", "walrus" )
After filtering stopwords:
( "walrus" );
Stopalizer provides default stoplists for several languages, courtesy of the Snowball project (<http://snowball.tartarus.org>), or you may supply your own.
my $stopalizer = KinoSearch::Analysis::Stopalizer->new( language => 'de', ); # or... my $stopalizer = KinoSearch::Analysis::Stopalizer->new( stoplist => \%stoplist, );
stoplist - A hash with stopwords as the keys.
language - The ISO code for a supported language.
KinoSearch::Analysis::Stopalizer isa KinoSearch::Analysis::Analyzer isa KinoSearch::Obj.
Copyright 2005-2009 Marvin Humphrey
See KinoSearch version 0.30.
To install KSx::Simple, copy and paste the appropriate command in to your terminal.
cpanm
cpanm KSx::Simple
CPAN shell
perl -MCPAN -e shell install KSx::Simple
For more information on module installation, please visit the detailed CPAN module installation guide.