The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.
 / 
KinoSearch-0.30_101
River stage two • 9 direct dependents • 10 total dependents
1 ++
/ KinoSearch

NAME

KinoSearch - Search engine library.

VERSION

0.30_101

SYNOPSIS

First, plan out your index structure, create the index, and add documents:

    # indexer.pl
    
    use KinoSearch::Index::Indexer;
    use KinoSearch::Plan::Schema;
    use KinoSearch::Analysis::PolyAnalyzer;
    use KinoSearch::Plan::FullTextType;
    
    # Create a Schema which defines index fields.
    my $schema = KinoSearch::Plan::Schema->new;
    my $polyanalyzer = KinoSearch::Analysis::PolyAnalyzer->new( 
        language => 'en',
    );
    my $type = KinoSearch::Plan::FullTextType->new(
        analyzer => $polyanalyzer,
    );
    $schema->spec_field( name => 'title',   type => $type );
    $schema->spec_field( name => 'content', type => $type );
    
    # Create the index and add documents.
    my $indexer = KinoSearch::Index::Indexer->new(
        schema => $schema,   
        index  => '/path/to/index',
        create => 1,
    );
    while ( my ( $title, $content ) = each %source_docs ) {
        $indexer->add_doc({
            title   => $title,
            content => $content,
        });
    }
    $indexer->commit;

Then, search the index:

    # search.pl
    
    use KinoSearch::Search::IndexSearcher;
    
    my $searcher = KinoSearch::Search::IndexSearcher->new( 
        index => '/path/to/index' 
    );
    my $hits = $searcher->hits( query => "foo bar" );
    while ( my $hit = $hits->next ) {
        print "$hit->{title}\n";
    }

DESCRIPTION

KinoSearch is a high-performance, modular search engine library.

Features

  • Extremely fast. A single machine can handle millions of documents.

  • Scalable to multiple machines.

  • Incremental indexing (addition/deletion of documents to/from an existing index).

  • UTF-8 support.

  • Support for boolean operators AND, OR, and AND NOT; parenthetical groupings; and prepended +plus and -minus.

  • Algorithmic selection of relevant excerpts and highlighting of search terms within excerpts.

  • Highly customizable query and indexing APIs.

  • Customizable sorting.

  • Phrase matching.

  • Stemming.

  • Stoplists.

Getting Started

KSx::Simple provides a stripped down API which may suffice for many tasks.

KinoSearch::Docs::Tutorial demonstrates how to build a basic CGI search application.

The tutorial spends most of its time on these five classes:

Supported Languages and Encodings

As of version 0.20, KinoSearch supports Unicode in addition to Latin-1. All output strings use Perl's internal Unicode encoding. For use of KinoSearch with non-Latin-1 material, see Encode.

KinoSearch provides "native support" for 15 languages, meaning that PolyAnalyzer supports them.

  • Danish

  • Dutch

  • English

  • Finnish

  • French

  • German

  • Hungarian

  • Italian

  • Norwegian

  • Portuguese

  • Romanian

  • Russian

  • Spanish

  • Swedish

  • Turkish

Delving Deeper

KinoSearch::Docs::CookBook augments the tutorial with more advanced recipes.

For creating complex queries, see KinoSearch::Search::Query and its subclasses TermQuery, PhraseQuery, ANDQuery, ORQuery, NOTQuery, RequiredOptionalQuery, MatchAllQuery, and NoMatchQuery, plus KinoSearch::Search::QueryParser.

For distributed searching, see KSx::Remote::SearchServer, KSx::Remote::SearchClient, and KinoSearch::Search::PolySearcher.

Backwards Compatibility Policy

KinoSearch spins off stable forks into new namespaces periodically. As of this release, the latest is "KinoSearch1", forked from version 0.165. Users who require strong backwards compatibility should use a stable fork.

The main namespace, "KinoSearch", is an unstable development branch (as hinted at by its version number). Superficial API changes are frequent. Hard file format compatibility breaks which require reindexing are rare, as we generally try to provide continuity across multiple releases, but they happen every once in a while.

CLASS METHODS

The KinoSearch module itself does not do much.

error

    my $instream = $folder->open_in( file => 'foo' ) or die KinoSearch->error;

Access a shared variable which is set by some routines on failure. It will always be either a KinoSearch::Object::Err or undef.

SEE ALSO

The KinoSearch homepage, where you'll find links to the mailing list and so on, is http://www.rectangular.com/kinosearch.

The Lucene homepage is http://lucene.apache.org.

History

Search::Kinosearch 0.02x, now dead and removed from CPAN, was this suite's forerunner. Plucene is a pure-Perl port of Lucene 1.3. KinoSearch is a from-scratch project which attempts to draws on the lessons of both.

KinoSearch is named for Kino, the main character in John Steinbeck's novella, "The Pearl".

SUPPORT

Please direct support questions to the kinosearch mailing list: subscription information at http://www.rectangular.com/kinosearch.

BUGS

Not thread-safe.

Some exceptions leak memory.

Won't work on esoteric architectures where a char is more than one byte, or where floats don't conform to IEEE 754.

Please report any other bugs or feature requests to bug-kinosearch@rt.cpan.org, or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=KinoSearch.

COPYRIGHT & LICENSE

Copyright 2005-2010 Marvin Humphrey

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

Terms of usage for Apache Lucene, from which portions of KinoSearch are derived, are spelled out in the Apache License: see the file "ApacheLicense2.0.txt".

DISCLAIMER OF WARRANTY

BECAUSE THIS SOFTWARE IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE SOFTWARE, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE SOFTWARE "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE SOFTWARE IS WITH YOU. SHOULD THE SOFTWARE PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR, OR CORRECTION.

IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE SOFTWARE AS PERMITTED BY THE ABOVE LICENCE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE SOFTWARE (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE SOFTWARE TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.