NAME
Shared::Arena - memory two processes can both read, without a syscall
VERSION
Version 0.01
SYNOPSIS
use Shared::Arena;
# in the parent, before the fork
my $arena = Shared::Arena->create(size => 8 * 1024 * 1024);
my $ring = $arena->ring('events', slots => 4096, slot_size => 512);
my $cursor = $ring->cursor;
if (fork() == 0) {
$ring->publish('hello', 'from the child');
exit 0;
}
for my $rec ($cursor->drain) {
my ($topic, $payload, $seq) = @$rec;
}
# or by name, from a process that is not a child at all
my $arena = Shared::Arena->attach('my-app');DESCRIPTION
Two processes on one machine that want to share a structure have, in Perl, a pipe, a socket, a file or a database. Each costs a syscall and at least one copy per message. The thing that removes both, a region of memory every process maps and reads directly, is ordinary elsewhere and absent from CPAN.
Shared::Arena is that region. It is mapped once, carved into named sub-regions, and read by every process that maps it without any of them copying, locking or calling into the kernel.
Eight things come with it to put in one.
Shared::Arena::Ring - a record queue that many processes write at once and many read, each at its own pace.
Shared::Arena::Map - a fixed-capacity table with lock-free reads and counters that increment in a single atomic instruction.
Shared::Arena::Cache - a map that evicts rather than refusing, so one pool of workers warms one cache instead of N.
Shared::Arena::Bloom - a set that answers "no" exactly and "yes" probably, and holds no keys at all.
Shared::Arena::CountMin - counts how often each key has been seen without storing any of them, which is how you find the loud one when you cannot bound how many there are.
Shared::Arena::Histogram - a distribution every process adds to at once, with no merge step and a bounded error.
Shared::Arena::Rate - a token bucket per key, so a limit enforced across a pool is the limit you wrote rather than that limit times the number of workers.
Shared::Arena::Frozen - one structure, published whole and read where it lies: nested data every worker reads without any of them rebuilding it.
The first seven store opaque bytes, or in the case of the filter and the sketch no keys at all, so a nested structure has to be flattened going in and rebuilt coming out. Shared::Arena::Frozen is the one that does not rebuild, and it is the reason Frozen is a prerequisite.
Two ways in
An anonymous arena is mapped before a fork and inherited. It has no name, needs no cleanup, and disappears when the last process holding it exits.
A named arena can be attached by any process that knows the name, related or not. Creating a name that already exists attaches to it instead, so every process can run the same setup code and exactly one of them will turn out to be the creator. Ask created which one that was.
Nothing inside an arena is a pointer
Every reference from one part of an arena to another is an offset. That is what lets two processes map the same arena at two different addresses and read the same structure, and it is the difference between something that can be attached and something that can only be inherited.
Allocation is a bump, and there is no free
A sub-region is carved by advancing a high-water mark and lives as long as the arena.
This is a limit on purpose. A free list that several processes share is corrupted permanently by one of them dying between two writes, and the next allocation then hands out memory somebody else is already using: a failure with no symptom until it has a bad one. Nothing that wants an arena needs to release a sub-region. A caller who genuinely needs reuse can do it inside its own carved region, where getting it wrong costs one feature rather than everything.
Sizing one
size is the usable bytes and the bookkeeping is added to it, so asking for eight megabytes gives eight megabytes to spend. A ring costs slots * slot_size for its life. Everything is reserved when the arena is created and nothing grows afterwards, so a carve that does not fit is refused rather than served slowly.
METHODS
create
my $arena = Shared::Arena->create(%opts);size- usable bytes, default one megabyte.name- makes it a named arena other processes can attach to.regions- how many sub-regions may be carved, default 64.
Croaks if the arena cannot be created, saying what was wrong. Creating a name that already exists attaches to it.
attach
my $arena = Shared::Arena->attach($name);Attaches to an existing named arena, or returns undef. It never creates one: attaching to a name nobody has used is a question with an answer, not an instruction.
region
my ($off, $len) = $arena->region($name, size => $bytes);
my ($off, $len) = $arena->region($name); # find, do not carveCarves a named sub-region, or returns the one already carved under that name so every process can ask for it the same way. An empty list if it does not exist, the arena is full, or the name is unusable: names are one to thirty-one bytes.
poke, peek
$arena->poke($name, $offset, $bytes);
my $bytes = $arena->peek($name, $offset, $length);Raw bytes in and out of a carved region, for a caller with a structure of its own design. Croaks if the access would run past the region's end.
These do no locking of any kind. Two processes writing the same bytes get what they deserve; use a ring, or coordinate.
ring
my $ring = $arena->ring($name, slots => 4096, slot_size => 512);A Shared::Arena::Ring in this arena, created on first use. Every process may call this with the same arguments.
map
my $map = $arena->map($name, slots => 4096, slot_size => 512);A Shared::Arena::Map in this arena, created on first use. Every process may call this with the same arguments.
bloom
my $b = $arena->bloom($name, capacity => 1_000_000, fp_rate => 0.001);A Shared::Arena::Bloom in this arena, created on first use. Every process may call this with the same arguments.
histogram
my $h = $arena->histogram($name, max => 60_000_000, sigbits => 5);A Shared::Arena::Histogram in this arena, created on first use. Every process may call this with the same arguments.
cache
my $c = $arena->cache($name, capacity => 4096, entry_size => 4096);A Shared::Arena::Cache in this arena, created on first use. Every process may call this with the same arguments.
frozen
my $conf = $arena->frozen($name, size => 256 * 1024, slots => 4);A Shared::Arena::Frozen in this arena, created on first use. Every process may call this with the same arguments.
size is the largest block it will carry and the region costs size * slots, because a publish never writes where a reader is reading.
rate
my $rl = $arena->rate($name, limit => 100, window => 60, slots => 4096);A Shared::Arena::Rate in this arena, created on first use. Every process may call this with the same arguments.
limit is the burst: the most one key may spend at once, and also how much a full refill puts back. window is how long that refill takes, in seconds, so the two together are the sustained rate. slots is how many distinct keys the table holds; it is rounded up to a power of two.
A limiter is per policy rather than per call, so two routes with different limits are two carves. That is deliberate: a table whose limit is whatever the last caller passed is a table two callers can disagree about.
$rl->allow($ip) or return $c->status(429);
$rl->allow($ip, 10) or ...; # an expensive route costs more
$c->header('X-RateLimit-Remaining' => int $rl->remaining($ip));
$c->header('Retry-After' => int $rl->retry_after($ip) + 1);A limit enforced per worker is not the limit you wrote. Four workers each keeping their own counter turn 100/min into 400/min, and the number changes when the pool is resized. This keeps one bucket per key in memory every worker already shares, so the limit is the limit whatever the pool does.
It is a token bucket, not a fixed window, because a fixed window has a hole at the boundary: a caller may spend its whole allowance at 11:59:59 and its whole allowance again at 12:00:00, which is twice the limit in one second. A bucket refills continuously and has no boundary to stand on.
countmin
my $cms = $arena->countmin($name, error => 0.001, confidence => 0.99);A Shared::Arena::CountMin in this arena, created on first use. Every process may call this with the same arguments.
How often a key has been seen, in space that does not grow with the number of keys. It answers high and never low, so a heavy hitter can never hide, and the error is a fraction of the total rather than of the key's own count: it finds the client making a million requests and says nothing useful about one that made three.
my $seen = $cms->add($ip);
warn "$ip is loud" if $seen > 10_000;regions
my @names = $arena->regions;size, created
my $bytes = $arena->size;
my $is_mine = $arena->created;base
my $address = $arena->base;Where this process mapped the arena. Of no use except to prove that two mappings of one arena are at different addresses, which is what a test wants.
destroy
Shared::Arena->destroy($name);Removes a named arena's name. Existing mappings stay valid until their last user exits, which is a feature and a leak at once and cannot be one without the other: a creator that crashes leaves the arena behind, so a restarting server reattaches to a live one and its readers never noticed, but nothing removes a name nobody will open again. Call it when the arena's life is over.
have_atomics
Shared::Arena::have_atomics() or fall_back();Whether this build has the atomic operations everything here rests on. When it does not, create refuses rather than pretending, and a caller is expected to degrade. See "ATOMICS".
WAKING A READER
Draining a ring never blocks, so a reader either polls or waits to be told. To wait to be told, ask for wakeups before the fork:
my $arena = Shared::Arena->create(size => 8 * 1024 * 1024);
$arena->wakers(16); # before any fork
# in each process that will read
$arena->waker;
my $fd = $arena->waker_fd;
# and in its event loop
vec(my $bits, $fd, 1) = 1;
while (select(my $r = $bits, undef, undef, undef) > 0) {
$arena->drained; # before draining the ring
my @recs = $cursor->drain;
}A wakeup means there is something to read: the notification follows the record, never precedes it. Many records in quick succession cost one wakeup rather than one each.
Only processes that inherited the arena can be woken. The mechanism is a pipe created before the fork, and a process that attached to a named arena inherited nothing. There, waker_fd returns -1 and the reader polls.
wakers
$arena->wakers($count);Creates the wakeup channels. Must run before the fork.
waker
my $index = $arena->waker;Claims one for this process, after the fork. Returns its index, or -1 when there are none left, in which case the caller polls.
waker_fd
my $fd = $arena->waker_fd;The descriptor to select on, or -1 when this process has no wakeup channel.
drained
$arena->drained;Call after a wakeup and before draining the ring.
WHEN A PROCESS DIES
A publisher that is killed part-way through writing a record leaves a hole. Every reader that reaches it faces a question with two wrong answers: skipping a record that is merely late throws away live data, and waiting for one that will never arrive stalls the reader for ever. A timeout picks one of those and is wrong the rest of the time, because a live publisher on a loaded machine can lose the CPU for longer than any bound worth setting.
So a reader does not wait on a clock. It asks whether the process that claimed the record still exists, and requires two independent answers to agree: that the process is gone, and that it has made no progress at all for a grace period. Both must hold. A process that is alive but wedged fails the second; a process whose identifier has been reused passes the first, and the disagreement costs one stalled record rather than a lost one.
Once a hole is proven abandoned it is filled, not skipped, so every other reader passes it by reading rather than by waiting. The record is counted as abandoned and never as lapped, because a crash and a slow reader are different diagnoses.
peers
my %p = $arena->peers; # used, live, reapedProcesses registered with this arena. reaped is how many were found dead holding an unfinished record.
ATOMICS
Everything here rests on atomic loads, stores and compare-and-swap. Where the compiler provides none, create refuses instead of pretending, and have_atomics reports which build this is. That is a supported configuration rather than a broken one: a caller is expected to fall back to whatever it did before.
THE HOT METHODS ARE OPCODES
The doors called in a loop are compiled to run without a subroutine call: get and set on a cache, fetch, store, exists and incr on a map, add and check on a filter, add and estimate on a sketch, record on a histogram, publish on a ring, and allow on a limiter.
Nothing needs doing to get this. The ordinary method call is the fast path, there is no second API to migrate to, and the answers are identical either way, including which of them return an empty list for a miss.
Measured on an M-series Mac, nanoseconds per call, ordinary against compiled:
cache->get 60.2 -> 55.1
cache->set 37.4 -> 30.2
map->fetch 54.1 -> 42.6
map->exists 30.9 -> 23.8
map->incr 37.6 -> 24.4
bloom->check 31.9 -> 22.8
bloom->add 30.6 -> 21.1
countmin->add 30.2 -> 24.9
hist->record 25.1 -> 13.1Anything that would change the answer takes the ordinary path instead: a subclass that overrides the method, a replaced subroutine, an argument list of a width the call site was not compiled with, an object whose handle has been released. So a debugger, a profiler and local *Some::Method = sub {...} all work the way they did.
cache->get gains least, and the reason is worth knowing: Frozen compiles its own ->get the same way, and two of these cannot own one call site. Whichever got there first keeps it and the other takes what is left, which here is about half. Both remain correct and neither is slowed by the other. It only affects a name both dists compile, which today is get.
What it costs a program that does not use it
The compiler hook sees every call site in the process with one of those names, not only this module's. A call on some other class runs a guard, is declined, and takes its ordinary path: 2.5ns per call, measured at 39.6 against 42.1.
If a program makes millions of those and few of these, set
SHARED_ARENA_NO_XOP=1in the environment before it starts. Every door then goes through the ordinary subroutine, and nothing else changes. It is read once, when the module loads, because the rewriting happens at compile time.
CAVEATS
One machine. An arena is memory, not a protocol. Nothing here crosses a network, and nothing is written to disk.
Not durable. A record lives until the ring wraps or the last process exits. Anything that must survive a machine restart belongs in a database.
No security boundary. Every process that can map the arena can read and write all of it. A named arena is created with owner-only permissions, and that is the whole of the protection.
Trusted contents. The arena is written by programs you run, not by strangers. Structural checks refuse an arena this build cannot read, and a reader will not follow a length past the end of a mapping, but a process with write access can make another read nonsense.
SEE ALSO
Shared::Arena::Frozen, Shared::Arena::Frozen::View, Frozen, Shared::Arena::Ring, Shared::Arena::Ring::Cursor, Shared::Arena::Map, Shared::Arena::Bloom, Shared::Arena::Histogram, Shared::Arena::Cache, Shared::Arena::Rate, Shared::Arena::CountMin.
AUTHOR
LNATION, <email at lnation.org>
BUGS
Please report any bugs or feature requests to bug-shared-arena at rt.cpan.org, or through the web interface at https://rt.cpan.org/NoAuth/ReportBug.html?Queue=Shared-Arena.
LICENSE AND COPYRIGHT
This software is Copyright (c) 2026 by LNATION.
This is free software, licensed under the Artistic License 2.0.