The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.
 / 
Patro-0.16
River stage zero No dependents
++
/ Patro

NAME

Patro - proxy access to remote objects

VERSION

0.16

SYNOPSIS

    # on machine 1 (server)
    use Patro;
    my $obj = ...
    $config = patronize($obj);
    $config->to_file( 'config_file' );


    # on machines 2 through n (clients)
    use Patro;
    my ($proxy) = Patro->new( 'config_file' )->getProxies;
    ...
    $proxy->{key} = $val;         # updates $obj->{key} for obj on server
    $val = $proxy->method(@args); # calls $obj->method for obj on server

DESCRIPTION

Patro is a mechanism for making any Perl reference in one Perl program accessible is other processes, even processes running on different hosts. The "proxy" references have the same look and feel as the native references in the original process, and any manipulation of the proxy reference will have an effect on the original reference.

Some important features:

  • Hash members and array elements

    Accessing or updating hash values or array values on a remote reference is done with the same syntax as with the local reference:

        # host 1
    use Patro;
    my $hash1 = { abc => 123, def => [ 456, { ghi => "jkl" }, "mno" ] };
    my $config = patronize($hash1);
    ...

    # host 2
    use Patro;
    my $hash2 = Patro->new($config)->getProxies;
    print $hash2->{abc};                # "123"
    $hash2->{def}[2] = "pqr";           # updates $hash1 on host 1
    print delete $hash2->{def}[1]{ghi}; # "jkl", updates $hash1 on host1

  • Remote method calls

    Method calls on the proxy object are propagated to the original object, affecting the remote object and returning the result of the call.

        # host 1
    use Patro;
    sub Foofie::new { bless \$_[1],'Foofie' }
    sub Foofie::blerp { my $self=shift; wantarray ? (5,6,7,$$self) : ++$$self }
    patronize(Foofie->new(17))->to_file('/config/file');
    ...

    # host 2
    use Patro;
    my $foo = Patro->new('/config/file')->getProxies;
    my @x = $foo->blerp;           # (5,6,7,17)
    my $x = $foo->blerp;           # 18

  • Overloaded operators

    Any overloaded operations on the original object are supported on the remote object.

        # host 1
    use Patro;
    my $obj = Barfie->new(2,5);
    $config = patronize($obj);
    $config->to_file( 'config' );
    package Barfie;
    use overload '+=' => sub { $_ += $_[1] for @{$_[0]->{vals}};$_[0] },
         fallback => 1;
    sub new {
        my $pkg = shift;
        bless { vals => [ @_ ] }, $pkg;
    }
    sub prod { my $self = shift; my $z=1; $z*=$_ for @{$_[0]->{vals}}; $z }

    # host 2
    use Patro;
    my $proxy = getProxies('config');
    print $proxy->prod;      # calls Barfie::prod($obj) on host1, 2 * 5 => 10
    $proxy += 4;             # calls Barfie '+=' sub on host1
    print $proxy->prod;      # 6 * 9 => 54

  • Code references

    Patro supports sharing code references and data structures that contain code references (think dispatch tables). Proxies to these code references can invoke the code, which will then run on the server.

        # host 1
    use Patro;
    my $foo = sub { $_[0] + 42 };
    my $d = {
        f1 => sub { $_[0] + $_[1] },
        f2 => sub { $_[0] * $_[1] },
        f3 => sub { int( $_[0] / ($_[1] || 1) ) },
        g1 => sub { $_[0] += $_[1]; 18 },
    };
    patronize($foo,$d)->to_file('config');
    ...

    # host 2
    use Patro;
    my ($p_foo, $p_d) = getProxies('config');
    print $p_foo->(17);        # "59"   (42+17)
    print $p_d->{f1}->(7,33);  # "40"   (7+33)
    print $p_d->{f3}->(33,7);  # "4"    int(33/7)
    ($x,$y) = (5,6);
    $p_d->{g1}->($x,$y);
    print $x;                  # "11"   ($x:6 += 5)

  • filehandles

    Filehandles can also be shared through the Patro framework.

        # host 1
    use Patro;
    open my $fh, '>', 'host1.log';
    patronize($fh)->to_file('config');
    ...

    # host 2
    use Patro;
    my $ph = getProxies('config');
    print $ph "A log message for the server\n";

    Calling open through a proxy filehandle presents some security concerns. A client could read or write any file on the server host visible to the server's user id. Or worse, a client could open a pipe through the handle to run an arbitrary command on the server. open and close operations on proxy filehandles will not be allowed unless the process running the Patro server imports Patro with the :insecure tag. See "SERVER OPTIONS" in Patro::Server for more information.

FUNCTIONS

patronize

    CONFIG = patronize(@REFS)

Creates a server on the local machine that provides proxy access to the given list of references. It returns an object with information about how to connect to the server.

The returned object has to_string and to_file methods to store the configuration where it can be read by other processes. Either the object, its string representation, or the filename containing config information may be used as input to the "getProxies" function to retrieve proxies to the shared references.

getProxies

    PROXIES = getProxies(CONFIG)
    PROXIES = getProxies(STRING)
    PROXIES = getProxies(FILENAME)

Connects to a server on another machine, specified in the CONFIG string, and returns proxies to the list of references that are served. In scalar context, returns a proxy to the first reference that is served.

See the "PROXIES" section below for what you can do with the output of this function.

ref

    TYPE = Patro::ref(PROXY)

For the given proxy object, returns the ref type of the remote object being served. If the input is not a proxy, returns undef. See also "reftype".

reftype

    TYPE = Patro::reftype(PROXY)

Returns the simple reference type (e.g., ARRAY) of the remote object associated with the given proxy, as if we were calling Scalar::Util::reftype on the remote object. Returns undef if the input is not a proxy object.

client

    CLIENT = Patro::client(PROXY)

Returns the IPC client object used by the given proxy to communicate with the remote object server. The client object contains information about how to communicate with the server and other connection configuration.

Also see the functions related to remote resource synchronization in the "SYNCHRONIZATION" section below.

PROXIES

Proxy references, as returned by the "getProxies" function above, or sometimes returned in other calls to the server, are designed to look and feel as much as possible as the real references on the remote server that they provide access to, so any operation or expression with the proxy on the local machine should evaluate to the same value(s) as the same operation or expression with the real object/reference on the remote server. When the server if using threads and is sharing the served objects between threads, an update to the proxy object will affect the remote object, and vice versa.

Example 1: network file synchronization

Network file systems are notoriously flaky when it comes to synchronizing files that are being written to by processes on many different hosts [citation needed]. Patro provides a workaround, in that every machine can hold to a proxy to an object that writes to a file, with the object running on a single machine.

    # master
    package SyncWriter;
    use Fcntl qw(:flock SEEK_END);
    sub new {
        my ($pkg,$filename) = @_;
        open my $fh, '>', $filename;
        bless { fh => $fh }, $pkg;
    }
    sub print {
        my $self = shift;
        flock $self->{fh}, LOCK_EX;
        seek $self->{fh}, 0, SEEK_END;
        print {$self->{fh}} @_;
        flock $self->{fh}, LOCK_UN;
    }
    sub printf { ... }

    use Patro;
    my $writer = SyncWriter->new("file.log");
    my $cfg = patronize($writer);
    open my $fh,'>','/network/accessible/file';
    print $fh $cfg;
    close $fh;
    ...

    # slaves
    use Patro;
    open my $fh, '<', '/network/accessible/file';
    my $cfg = <$fh>;
    close $fh;
    my $writer = Patro->new($cfg)->getProxies;
    ...
    # $writer->print with a proxy $writer
    # invokes $writer->print on the host. Since all
    # the file operations are done on a single machine,
    # there are no network synchronization issues
    $writer->print("a log message\n");
    ...

Example 2: Distributed queue

A program that distributes tasks to several threads or several child processes can be extended to distribute tasks to several machines.

    # master
    use Patro;
    my $queue = [ 'job 1', 'job 2', ... ];
    patronize($queue)->to_file('/network/accessible/file');
    ...

    # slaves
    use Patro;
    my $queue = Patro->new('/network/accessible/file')->getProxies;

    while (my $task = shift @$queue) {
        ... do task ...
    }

(This example will not work without threads. For a more robust network-safe queue that will run with forks, see Forks::Queue)

Example 3: Keep your code secret

If you distribute your Perl code for others to use, it is very difficult to keep others from being able to see (and potentially steal) your code. Obfuscators are penetrable by any determined reverse engineer. Most other suggestions for keeping your code secret revolve around running your code on a server, and having your clients send input and receive output through a network service.

The Patro framework can make this service model easier to use. Design a small set of objects that can execute your code, provide your clients with a public API for those objects, and make proxies to your objects available through Patro.

    # code to run on client machine
    use Patro;
    my $cfg = ...    # provided by you
    my ($obj1,$obj2) = Patro->new($cfg)->getProxies;
    $result = $obj1->method($arg1,$arg2);
    ...

In this model, the client can use the objects and methods of your code, and inspect the members of your objects through the proxies, but the client cannot see the source code.

SYNCHRONIZATION

A Patro server may make the same reference available to more than one client. If the server is running in "threads" mode, each client will have an instance of the reference in a different thread, and the reference will be shared between threads. For thread safety, we will want threads and processes to have exclusive access to the reference. This also applies to client processes that have a proxy to a remote resource.

Patro provides a few functions to request and manage exclusive access to remote references. Like most such "locks" in Perl, these locks are "advisory", meaning clients that do not use this synchronization scheme may still manipulate remote references that are locked with this scheme by other clients.

synchronize

@list = Patro::synchronize $proxy, BLOCK [, options]

$list = Patro::synchronize $proxy, BLOCK [, options]

Requests exclusive access to the underlying remote object that the $proxy refers to. When access is granted, executes the given BLOCK in either list or scalar context. When the code BLOCK is finished, relinquish control of the remote resource and return the results of the code.

    use Patro;
    $proxy = getProxies('config/file');
    Patro::synchronize $proxy, sub {
        $proxy->method_call_that_needs_thread_safety()
    };

options may be a hash or reference to a hash, with these two key-value pairs recognized:

timeout => $seconds

Waits at least $seconds seconds until the lock for the remote resource is available, and gives up after that. Using a negative value for $seconds has the semantics of a non-blocking lock call. If $seconds is negative and the lock is not acquired on the first attempt, the synchronize call does not make any further attempts. If the lock is not acquired, synchronize will return the empty list and set $!.

steal => $bool

If true, and if allowed by the server, acquires the lock for the remote resource even if it is held by another process. If timeout is also specified, waits until the timeout expires before stealing the lock.

Whether one monitor may steal the lock from another monitor is a setting on the server. If stealing is not allowed and if this call can not acquire the lock conventionally, the synchronize call returns the empty list and sets $!.

lock

unlock

$status = Patro::lock($proxy [, $timeout [, $steal]])

$status = Patro::unlock($proxy [,$count])

An alternative to the Patro::synchronize($proxy, $code) syntax is to use Patro::lock and Patro::unlock explicitly.

Patro::lock attempts to acquire an exclusive (but advisory) lock on the remote resource referred to by the $proxy. It returns true if the lock was successfully acquired, and returns false and sets $! if there was an issue acquiring the lock. As in the options to Patro::synchronize, you may specify a timeout -- a maximum number of seconds to wait to acquire the lock, and/or set the steal flag, which will always acquire the lock even if it is held by another monitor (if the server allows stealing).

Patro::lock may be called on a proxy that already possesses the lock on its remote resource. Successive lock calls "stack" so that you must call Patro::unlock on the proxy the same number of times that you call Patro::lock (or provide a $count argument to Patro::unlock, see below) before the lock on the remote resource will be released.

Patro::unlock release the (a) lock on the remote resource referred to by the $proxy. Returns true if the lock was successfully removed. A false return value generally means that the given $proxy was not in possession of the remote resource's lock at the time the function was called.

Since lock calls "stack", a proxy may hold the lock on a remote resource more than one time. If a $count argument is provided to Patro::unlock, more than one of those stacked locks can be released. If $count is positive, Patro::unlock will release up to $count locks held by the proxy. If $count is negative, all locks will be released and the lock on the remote resource will become available.

wait

$status = Patro::wait($proxy [, $timeout])

For a $proxy that possesses the lock on its remote resource, releases the lock and puts the resource monitor into a "wait" state. The monitor will remain in that state until another monitor on the same resource issues a notify call (see "notify" below). After the monitor receives a "notify" call, it will attempt to reacquire the lock before resuming execution. Returns true if the monitor successfully releases the lock, waits for a notify call, and reacquires the lock.

If a $timeout is specified, the Patro::wait call will return after $timeout seconds whether or not the monitor has been notified and the lock has been reacquired. Patro::wait will also return false if $proxy is not currently in possession of its remote resource's lock.

notify

$status = Patro::notify($proxy [,$count])

For a $proxy that possess the lock on its remote resource, move one or more other monitors on the resource that are currently in a "wait" state into a "notify" state, so that those other monitor will attempt to acquire the lock to the remote resource. If $count is provided and is positive, this function will notify up to $count other monitors. If $count is negative, this function will notify all waiting monitors.

Returns false if the $proxy is not in possession of the lock on the remote resource when the function call is made. Otherwise, returns the number of monitors notified, or "0 but true" if there were no monitors to notify.

Note that a Patro::notify call does not release the remote resource. The notified monitors would still have to wait for the monitor that called notify to release the lock on the remote resource.

lock_state

$state = lock_state($proxy)

Returns a code indicating the status of the proxy's monitor on the lock of its remote resource. The return values will be one of

    0 - NULL - the monitor does not possess the lock
    1 - WAIT - the monitor has received a wait call since it
               last possessed the lock
    2 - NOTIFY - the monitor has received a notify call since
               it last possessed the lock
    3 - STOLEN - the monitor possessed the lock, but it was
               stolen by another monitor
  >=4 - LOCK - the monitor possesses the lock. Values larger than
               4 indicate that the monitor has stacked lock calls

Patro does not export these synchronization functions because lock and wait are also names for important Perl builtins.

As of v0.16, these synchronization features require that the server be run on a system that has the /dev/shm shared memory virtual filesystem.

EXPORTS

Patro exports the "patronize" function, to be used by a server, and "getProxies", to be used by a client.

The :insecure tag configures the server to allow insecure operations through a proxy. As of v0.16, this includes calling open and close on a proxy filehandle, and stealing a lock (see "lock", above) on a remote reference from another thread. This tag only affects programs that are serving remote objects. You can not disable this security, such as it as, in the server by applying the :insecure tag in a client program.

ENVIRONMENT

Patro pays attention to the following environment variables.

PATRO_THREADS

If the environment variable PATRO_THREADS is set, Patro will use it to determine whether to use a forked server or a threaded server to provide proxy access to objects. If this variable is not set, Patro will use threads if the threads module can be loaded.

LIMITATIONS

The -X file test operations on a proxy filehandle depend on the file test implementation in overload, which is available only in Perl v5.12 or better.

When the server uses forks (because threads are unavailable or because "PATRO_THREADS" was set to a false value), it is less practical to share variables between processes. When you manipulate a proxy reference, you are manipulating the copy of the reference running in a different process than the remote server. So you will not observe a change in the reference on the server (unless you use a class that does not save state in local memory, like Forks::Queue).

The synchronization functions Patro::wait and Patro::notify seem to require at least version 2.45 of the Storable module.

DOCUMENTATION AND SUPPORT

Up-to-date (blead version) sources for Patro are on github at https://github.com/mjob/p5-patro

You can find documentation for this module with the perldoc command.

    perldoc Patro

You can also look for information at:

LICENSE AND COPYRIGHT

MIT License

Copyright (c) 2017, Marty O'Brien

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.