Hadoop-Streaming

Changes for version 0.110030 - 2011-01-03

switch build from AutoPrereq to AutoPrereqs, as per RJBS the former is EOL. Correct Synopsis section in Hadoop::Streaming parent doc, thanks to error report from Frank S Fejes III

Modules

Hadoop::Streaming

Contains Mapper, Combiner and Reducer roles to simplify writing Hadoop Streaming jobs

Hadoop::Streaming::Combiner

Simplify writing Hadoop Streaming jobs. Combiner follows the same interface as Reducer. Requires a combine() function which will be called for each line of combiner data. Combiners are run on the same machine as the mapper as a pre-reduce reduction step.

Hadoop::Streaming::Mapper

Simplify writing Hadoop Streaming Mapper jobs. Write a map() function and let this role handle the Stream interface.

Hadoop::Streaming::Reducer

Simplify writing Hadoop Streaming jobs. Write a reduce() function and let this role handle the Stream interface. This Reducer roll provides an iterator over the multiple values for a given key.

Hadoop::Streaming::Reducer::Input

Parse input stream for reducer

Hadoop::Streaming::Reducer::Input::Iterator

Collects values for each key together with an iterator interface

Hadoop::Streaming::Reducer::Input::ValuesIterator

Role providing access to values for a given key.

Hadoop::Streaming::Role::Emitter

Role to provide emit, counter, and status interaction with Hadoop::Streaming.

Hadoop::Streaming::Role::Iterator

Role to require has_next and next