The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.

NAME

Lucy::Search::Compiler - Query-to-Matcher compiler.

SYNOPSIS

    # (Compiler is an abstract base class.)
    package MyCompiler;
    use base qw( Lucy::Search::Compiler );

    sub make_matcher {
        my $self = shift;
        return MyMatcher->new( @_, compiler => $self );
    }

DESCRIPTION

The purpose of the Compiler class is to take a specification in the form of a Query object and compile a Matcher object that can do real work.

The simplest Compiler subclasses – such as those associated with constant-scoring Query types – might simply implement a make_matcher() method which passes along information verbatim from the Query to the Matcher’s constructor.

However it is common for the Compiler to perform some calculations which affect it’s “weight” – a floating point multiplier that the Matcher will factor into each document’s score. If that is the case, then the Compiler subclass may wish to override get_weight(), sum_of_squared_weights(), and apply_norm_factor().

Compiling a Matcher is a two stage process.

The first stage takes place during the Compiler’s construction, which is where the Query object meets a Searcher object for the first time. Searchers operate on a specific document collection and they can tell you certain statistical information about the collection – such as how many total documents are in the collection, or how many documents in the collection a particular term is present in. Lucy’s core Compiler classes plug this information into the classic TF/IDF weighting algorithm to adjust the Compiler’s weight; custom subclasses might do something similar.

The second stage of compilation is make_matcher(), method, which is where the Compiler meets a SegReader object. SegReaders are associated with a single segment within a single index on a single machine, and are thus lower-level than Searchers, which may represent a document collection spread out over a search cluster (comprising several indexes and many segments). The Compiler object can use new information supplied by the SegReader – such as whether a term is missing from the local index even though it is present within the larger collection represented by the Searcher – when figuring out what to feed to the Matchers’s constructor, or whether make_matcher() should return a Matcher at all.

CONSTRUCTORS

new

    my $compiler = MyCompiler->SUPER::new(
        parent     => $my_query,
        searcher   => $searcher,
        similarity => $sim,        # default: undef
        boost      => undef,       # default: see below
    );

Abstract constructor.

  • parent - The parent Query.

  • searcher - A Lucy::Search::Searcher, such as an IndexSearcher.

  • similarity - A Similarity.

  • boost - An arbitrary scoring multiplier. Defaults to the boost of the parent Query.

ABSTRACT METHODS

make_matcher

    my $matcher = $compiler->make_matcher(
        reader     => $reader,      # required
        need_score => $need_score,  # required
    );

Factory method returning a Matcher.

  • reader - A SegReader.

  • need_score - Indicate whether the Matcher must implement score().

Returns: a Matcher, or undef if the Matcher would have matched no documents.

METHODS

get_weight

    my $float = $compiler->get_weight();

Return the Compiler’s numerical weight, a scoring multiplier. By default, returns the object’s boost.

get_similarity

    my $similarity = $compiler->get_similarity();

Accessor for the Compiler’s Similarity object.

get_parent

    my $query = $compiler->get_parent();

Accessor for the Compiler’s parent Query object.

sum_of_squared_weights

    my $float = $compiler->sum_of_squared_weights();

Compute and return a raw weighting factor. (This quantity is used by normalize()). By default, simply returns 1.0.

apply_norm_factor

    $compiler->apply_norm_factor($factor);

Apply a floating point normalization multiplier. For a TermCompiler, this involves multiplying its own weight by the supplied factor; combining classes such as ORCompiler would apply the factor recursively to their children.

The default implementation is a no-op; subclasses may wish to multiply their internal weight by the supplied factor.

  • factor - The multiplier.

normalize

    $compiler->normalize();

Take a newly minted Compiler object and apply query-specific normalization factors. Should be invoked by Query subclasses during make_compiler() for top-level nodes.

For a TermQuery, the scoring formula is approximately:

    (tf_d * idf_t / norm_d) * (tf_q * idf_t / norm_q)

normalize() is theoretically concerned with applying the second half of that formula to a the Compiler’s weight. What actually happens depends on how the Compiler and Similarity methods called internally are implemented.

INHERITANCE

Lucy::Search::Compiler isa Lucy::Search::Query isa Clownfish::Obj.