The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.

NAME

Lucy::Analysis::SnowballStopFilter - Suppress a "stoplist" of common words.

SYNOPSIS

    my $stopfilter = Lucy::Analysis::SnowballStopFilter->new(
        language => 'fr',
    );
    my $polyanalyzer = Lucy::Analysis::PolyAnalyzer->new(
        analyzers => [ $case_folder, $tokenizer, $stopfilter, $stemmer ],
    );

DESCRIPTION

A "stoplist" is collection of "stopwords": words which are common enough to be of little value when determining search results. For example, so many documents in English contain "the", "if", and "maybe" that it may improve both performance and relevance to block them.

Before filtering stopwords:

    ("i", "am", "the", "walrus")

After filtering stopwords:

    ("walrus")

SnowballStopFilter provides default stoplists for several languages, courtesy of the Snowball project (<http://snowball.tartarus.org>), or you may supply your own.

    |-----------------------|
    | ISO CODE | LANGUAGE   |
    |-----------------------|
    | da       | Danish     |
    | de       | German     |
    | en       | English    |
    | es       | Spanish    |
    | fi       | Finnish    |
    | fr       | French     |
    | hu       | Hungarian  |
    | it       | Italian    |
    | nl       | Dutch      |
    | no       | Norwegian  |
    | pt       | Portuguese |
    | sv       | Swedish    |
    | ru       | Russian    |
    |-----------------------|

CONSTRUCTORS

new( [labeled params] )

    my $stopfilter = Lucy::Analysis::SnowballStopFilter->new(
        language => 'de',
    );
    
    # or...
    my $stopfilter = Lucy::Analysis::SnowballStopFilter->new(
        stoplist => \%stoplist,
    );
  • stoplist - A hash with stopwords as the keys.

  • language - The ISO code for a supported language.

INHERITANCE

Lucy::Analysis::SnowballStopFilter isa Lucy::Analysis::Analyzer isa Lucy::Object::Obj.