NAME

Data::Stream::Bulk - N at a time iteration api

SYNOPSIS

        # get a bulk stream from somewere
        my $s = Data::Stream::Bulk::Foo->new( ... );

        # can be used like this:
        until ( $s->is_done ) {
                foreach my $item ( $s->items ) {
                        process($item);
                }
        }

        # or like this:
        while( my $block = $s->next ) {
                foreach my $item ( @$block ) {
                        process($item);
                }
        }

DESCRIPTION

This module tries to find middle ground between one at a time and all at once processing of data sets.

The purpose of this module is to avoid the overhead of implementing an iterative api when this isn't necessary, without breaking forward compatibility in case that becomes necessary later on.

The API optimizes for when a data set typically fits in memory and is returned as an array, but the consumer cannot assume that the data set is bounded.

The API is destructive in order to minimize the chance that resultsets are leaked due to improper usage.

API

Required Methods

The API requires two methods to be implemented:

is_done

Should return true if the stream is exhausted.

As long as this method returns a false value (not done) next could potentially return another block.

next

Returns the next block.

Note that next is not guaranteed to return an array reference, even if is_done returned false prior to calling it.

Convenience Methods

items

This method calls next and dereferences the result if there are pending items.

CLASSES

Data::Stream::Bulk::Array

This class is not a stream at all, but just one block. When the data set easily fits in memory this class can be used, while retaining forward compatibility with larger data sets.

Data::Stream::Bulk::Callback

Callback driven iteration.

Data::Stream::Bulk::DBI

Bulk fetching of data from DBI statement handles.

Data::Stream::Bulk::DBIC

DBIx::Class::ResultSet iteration.

Data::Stream::Bulk::Nil

An empty result set.

VERSION CONTROL

This module is maintained using Darcs. You can get the latest version from http://nothingmuch.woobling.org/code, and use darcs send to commit changes.

AUTHOR

Yuval Kogman <nothingmuch@woobling.org>

COPYRIGHT

        Copyright (c) 2008 Yuval Kogman. All rights reserved
        This program is free software; you can redistribute
        it and/or modify it under the same terms as Perl itself.