The London Perl and Raku Workshop takes place on 26th Oct 2024. If your company depends on Perl, please consider sponsoring and/or attending.
 / 
Net-Amazon-EMR-0.10
River stage zero No dependents
/ Net::Amazon::EMR

NAME

Net::Amazon::EMR - API for Amazon's Elastic Map-Reduce service

SYNOPSIS

  use Net::Amazon::EMR;

  my $emr = Net::Amazon::EMR->new(
    AWSAccessKeyId  => $AWS_ACCESS_KEY_ID,
    SecretAccessKey => $SECRET_ACCESS_KEY,
    ssl             => 1,
    );

  my $id = $emr->run_job_flow(Name => "Example Job",
                              Instances => {
                                  Ec2KeyName => 'myKeyId',
                                  InstanceCount => 10,
                                  KeepJobFlowAliveWhenNoSteps => 1,
                                  MasterInstanceType => 'm1.small',
                                  Placement => { AvailabilityZone => 'us-east-1a' },
                                  SlaveInstanceType => 'm1.small',
                              },
    );

  print "Job flow id = " . $id->JobFlowId . "\n";

  # Get details of just-launched job
  $result = $emr->describe_job_flows(JobFlowIds => [ $id->JobFlowId ]);

  # or get details of all jobs created after a given time
  $result = $emr->describe_job_flows(CreatedAfter => '2012-12-17T07:19:57Z');

  # or use DateTime
  $result = $emr->describe_job_flows(CreatedAfter => DateTime->new(year => 2012, month => 12, day => 17));

  # See the details of the typed result
  use Data::Dumper; print Dumper($result);

  # or dispense with types and see the details as a perl hash
  use Data::Dumper; print Dumper($result->as_hash);

  # Flexible Booleans - 1, 0, undef, 'true', 'false'
  $emr->set_visible_to_all_users(JobFlowIds => $id, VisibleToAllUsers => 1);
  $emr->set_termination_protection(JobFlowIds => [ $id->JobFlowId ], TerminationProtected => 'false');

DESCRIPTION

This is an implementation of the Amazon Elastic Map-Reduce API.

CONSTRUCTOR

new(%options)

This is the constructor. Options are as follows:

  • AWSAccessKeyId (required)

    Your AWS access key.

  • SecretAccessKey (required)

    Your secret key.

  • ssl (optional)

    If set to a true value, the base_url will use https:// instead of http://. Defaults to true.

METHODS

Detailed information on each of the methods can be found in the Amazon EMR API documentation. Each method takes a hash of parameters using the names given in the documentation. Parameter passing uses the following rules:

  • Array inputs such as InstanceGroups.member.N use their primary name and a Perl ArrayRef, i.e. InstanceGroups => [ ... ] in this example.

  • Either hashes or object instances may be passed in; e.g both of the following forms are acceptable:

        $emr->run_job_flow(Name => "API Test Job",
                                Instances => {
                                    Ec2KeyName => 'xxx',
                                    InstanceCount => 1,
                                },
        );

    $emr->run_job_flow(Name => "API Test Job",
                                Instances => Net::Amazon::EMR::JobFlowInstancesConfig->new(
                                    Ec2KeyName => 'xxx',
                                    InstanceCount => 1,
                                ),
        );

  • Otherwise, the names of parameters are exactly as found in the Amazon documentation for API version 2009-03-31.

add_instance_groups(%params)

AddInstanceGroups adds an instance group to a running cluster. Returns a Net::Amazon::EMR::AddInstanceGroupsResult object.

add_job_flow_steps(%params)

AddJobFlowSteps adds new steps to a running job flow. Returns 1 on success.

describe_job_flows(%params)

Returns a Net::Amazon::EMR::RunJobFlowResult that describes the job flows that match all of the supplied parameters.

modify_instance_groups(%params)

Modifies the number of nodes and configuration settings of an instance group. Returns 1 on success.

run_job_flow(%params)

Creates and starts running a new job flow. Returns a Net::Amazon::EMR::RunJobFlowResult object that contains the job flow ID.

set_termination_protection(%params)

Locks a job flow so the Amazon EC2 instances in the cluster cannot be terminated by user intervention, an API call, or in the event of a job-flow error. Returns 1 on success.

set_visible_to_all_users(%params)

Sets whether all AWS Identity and Access Management (IAM) users under your account can access the specifed job flows. Returns 1 on success.

terminate_job_flows(%params)

Terminates a list of job flows. Returns 1 on success.

ERROR HANDLING

If an error occurs in any of the methods, the error will be logged and an Exception::Class exception of type Net::Amazon::EMR::Exception will be thrown.

ERROR LOGGING

Logging uses Log::Log4perl. You should initialise Log::Log4perl at the beginning of your program to suit your needs. The simplest way to enable debugging output to STDERR is to call

  use Log::Log4perl qw/:easy/;
  Log::Log4perl->easy_init($DEBUG);

AUTHOR

Jon Schutz

http://notes.jschutz.net

BUGS

Please report any bugs or feature requests to bug-net-amazon-emr at rt.cpan.org, or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=Net-Amazon-EMR. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.

SUPPORT

You can find documentation for this module with the perldoc command.

    perldoc Net::Amazon::EMR

You can also look for information at:

ACKNOWLEDGEMENTS

The core interface code was adapted from Net::Amazon::EC2.

LICENSE AND COPYRIGHT

Copyright 2012 Jon Schutz.

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

See http://dev.perl.org/licenses/ for more information.

SEE ALSO

Amazon EMR API: http://http://docs.amazonwebservices.com/ElasticMapReduce/latest/APIReference/Welcome.html