NAME

Argon

RATIONALE

Argon is a distributed processing platform built for Perl. It is built to provide a simple system for building radically scalable software while at the same time ensuring a high level of robustness and redundancy.

MANAGERS

Managers are entry points into the distributed network. They accept tasks from clients and route them to workers, then deliver the results back to the client.

Managers keep track of the nodes available on the network, ensuring that work is distributed in a balanced and efficient manner to achieve the highest throughput. If a worker becomes unavailable, the load is automatically shifted to the rest of the network. If the worker becomes available again, it will be picked up and the manager will start shifting load to it as necessary.

Managers are started with argon:

    argon --manager --port 8000 --host mgrhost

See "argon" in bin.

WORKERS

Workers are essentially a managed pool of Perl processes. Managers route tasks to workers, who distribute them among their pool of Perl processes, then return the results to the manager (who in turn ensures it gets back to the client).

Once started, the worker notifies the manager that it is available and can immediately start handling tasks as needed. If for any reason the worker loses its connection to the manager, it will attempt to reestablish the connection until it is again in contact with its manager.

Argon workers are uniform. There are no "channels" for individual types of tasks. All workers can handle any type of task. This ensures that no classes of task are starved of resources while other types have underutilized workers.

Workers are started with argon:

    argon --worker --port 8001 --host workerhost --manager somehost:8000

By default, a worker will start a number of Perl processes that correlates to the number of CPUs on the system. This can be overridden with the --workers option.

    argon --worker --port 8001 --host workerhost --manager somehost:8000 --workers 8

See "argon" in bin.

CLIENTS

Clients connect to the manager (or, if desired, directly to a "stand-alone" worker that was started without the --manager option). Tasks can be launched in two different ways.

The first method is to send a task and wait for the results. Note that Argon uses Coro, so "waiting" for the result means that the current thread of execution yields until the result is ready, at which point it is awoken.

    use Argon::Client;

    my $client = Argon::Client->new(host => "mgrhost", port => 8000);
    my $result = $client->queue(
        # Code to execute
        sub {
            my ($x, $y) = @_;
            return $x + $y;
        },
        # Arguments to pass that code
        [4, 7],
    );

Tasks can also be sent off to the network in the background, allowing the thread of execution to continue until a point where synchronization is required.

    use Argon::Client;

    my $client = Argon::Client->new(host => "mgrhost", port => 8000);

    # Ship the task off and get a function that, when called, waits for
    # the result and returns it.
    my $deferred = $client->defer(
        # Code to execute
        sub {
            my ($x, $y) = @_;
            return $x + $y;
        },
        # Arguments to pass that code
        [4, 7],
    );

    # Synchronize to get the result
    my $result = $deferred->();

Errors thrown in the execution of the task are trapped and re-thrown by the client when the result is returned. In the case of queue that is done when call returns. In the case of defer, it happens when the deferred result is synchronized.

See Argon::Client.

SCALABILITY

Argon is designed to make scalability simple and easy. Simply add more workers to boost the resources available to all applications utilizing the network.

Because Argon workers are all uniform, adding a new worker node guarantees a linear boost in resources available to all client applications. For example, given identical tasks on a network with two worker nodes, each running the same number of processes, adding another worker would increase throughput by 50%. Doubling the number of workers would increase throughput by 100%.

AUTHOR

Jeff Ober <jeffober@gmail.com>

cpanm Argon

perl -MCPAN -e shell
install Argon