NAME
List::Gen - provides functions for generating lists
VERSION
version 0.976
SYNOPSIS
this module provides higher order functions, list comprehensions, generators, iterators, and other utility functions for working with lists. walk lists with any step size you want, create lazy ranges and arrays with a map like syntax that generate values on demand. there are several other hopefully useful functions, and all functions from List::Util are available.
use List::Gen;
print "@$_\n" for every 5 => 1 .. 15;
# 1 2 3 4 5
# 6 7 8 9 10
# 11 12 13 14 15
print mapn {"$_[0]: $_[1]\n"} 2 => %myhash;
my $ints = <0..>;
my $squares = gen {$_**2} $ints;
say "@$squares[2 .. 6]"; # 4 9 16 25 36
$ints->zip('.', -$squares)->say(6); # 0-0 1-1 2-4 3-9 4-16 5-25
list(1, 2, 3)->gen('**2')->say; # 1 4 9
my $fib = ([0, 1] + iterate {fib($_, $_ + 1)->sum})->rec('fib');
my $fac = iterate {$_ < 2 or $_ * self($_ - 1)}->rec;
say "@$fib[0 .. 15]"; # 0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610
say "@$fac[0 .. 10]"; # 1 1 2 6 24 120 720 5040 40320 362880 3628800
say <0, 1, * + * ...>->take(10)->str; # 0 1 1 2 3 5 8 13 21 34
say <[..*] 1, 1..>->str(8); # 1 1 2 6 24 120 720 5040
<**2 for 1..10 if even>->say; # 4 16 36 64 100
<1..>->map('**2')->grep(qr/1/)->say(5); # 1 16 81 100 121
EXPORT
use List::Gen; # is the same as
use List::Gen qw/mapn by every range gen cap \ filter cache apply zip
min max reduce glob iterate list/;
the following export tags are available:
:utility mapn by every apply min max reduce mapab
mapkey d deref slide curse remove
:source range glob makegen list array vecgen repeat file
:modify gen cache expand contract collect slice flip overlay
test recursive sequence scan scan_stream == scanS
cartesian transpose stream strict
:zip zip zipgen tuples zipwith zipwithab unzip unzipn
zipmax zipgenmax zipwithmax
:iterate iterate
iterate_multi == iterateM
iterate_stream == iterateS
iterate_multi_stream == iterateMS
:gather gather
gather_stream == gatherS
gather_multi == gatherM
gather_multi_stream == gatherMS
:mutable mutable done done_if done_unless
:filter filter
filter_stream == filterS
filter_ # non-lookahead version
:while take_while == While
take_until == Until
while_ until_ # non-lookahead versions
drop_while drop_until
:numeric primes
:deprecated genzip
:List::Util first max maxstr min minstr reduce shuffle sum
use List::Gen '*'; # everything
use List::Gen 0; # everything
use List::Gen ':all'; # everything
use List::Gen ':base'; # same as 'use List::Gen;'
use List::Gen (); # no exports
FUNCTIONS
- mapn
{CODE} NUM LIST
-
this function works like the builtin
map
but takesNUM
sized steps over the list, rather than one element at a time. inside theCODE
block, the current slice is in@_
and$_
is set to$_[0]
. slice elements are aliases to the original list. ifmapn
is called in void context, theCODE
block will be executed in void context for efficiency.print mapn {$_ % 2 ? "@_" : " [@_] "} 3 => 1..20; # 1 2 3 [4 5 6] 7 8 9 [10 11 12] 13 14 15 [16 17 18] 19 20 print "student grades: \n"; mapn { print shift, ": ", &sum / @_, "\n"; } 5 => qw { bob 90 80 65 85 alice 75 95 70 100 eve 80 90 80 75 };
- by
NUM LIST
- every
NUM LIST
-
by
andevery
are exactly the same, and allow you to add variable step size to any other list control structure with whichever reads better to you.for (every 2 => @_) {do something with pairs in @$_} grep {do something with triples in @$_} by 3 => @list;
the functions generate an array of array references to
NUM
sized slices ofLIST
. the elements in each slice are aliases to the original list.in list context, returns a real array. in scalar context, returns a generator.
my @slices = every 2 => 1 .. 10; # real array my $slices = every 2 => 1 .. 10; # generator for (every 2 => 1 .. 10) { ... } # real array for (@{every 2 => 1 .. 10}) { ... } # generator
if you plan to use all the slices, the real array is probably better. if you only need a few, the generator won't need to compute all of the other slices.
print "@$_\n" for every 3 => 1..9; # 1 2 3 # 4 5 6 # 7 8 9 my @a = 1 .. 10; for (every 2 => @a) { @$_[0, 1] = @$_[1, 0] # flip each pair } print "@a"; # 2 1 4 3 6 5 8 7 10 9 print "@$_\n" for grep {$$_[0] % 2} by 3 => 1 .. 9; # 1 2 3 # 7 8 9
- apply
{CODE} LIST
-
apply a function that modifies
$_
to a shallow copy ofLIST
and returns the copyprint join ", " => apply {s/$/ one/} "this", "and that"; > this one, and that one
- zip
LIST
-
zip
takes a list of array references and generators. it interleaves the elements of the passed in sequences to create a new list.zip
continues until the end of the shortest sequence.LIST
can be any combination of array references and generators.%hash = zip [qw/a b c/], [1..3]; # same as %hash = (a => 1, b => 2, c => 3);
in scalar context,
zip
returns a generator, produced byzipgen
if the first argument to
zip
is not an array or generator, it is assumed to be code or a code like string. that code will be used to join the elements from the remaining arguments.my $gen = zip sub {$_[0] . $_[1]}, [1..5], <a..>; # or = zip '.' => [1..5], <a..>; # or = zipwith {$_[0] . $_[1]} [1..5], <a..>; $gen->str; # '1a 2b 3c 4d 5e'
- zipmax
LIST
-
interleaves the passed in lists to create a new list.
zipmax
continues until the end of the longest list,undef
is returned for missing elements of shorter lists.LIST
can be any combination of array references and generators.%hash = zipmax [qw/a b c d/], [1..3]; # same as %hash = (a => 1, b => 2, c => 3, d => undef);
in scalar context,
zipmax
returns a generator, produced byzipgenmax
zipmax
provides the same functionality aszip
did in versions before 0.90 - tuples
LIST
-
interleaves the passed in lists to create a new list of arrays.
tuples
continues until the end of the shortest list.LIST
can be any combination of array references and generators.@list = tuples [qw/a b c/], [1..3]; # same as @list = ([a => 1], [b => 2], [c => 3]);
in scalar context,
tuples
returns a generator:tuples(...) ~~ zipwith {\@_} ...
- cap
LIST
-
cap
captures a list, it is exactly the same assub{\@_}->(LIST)
note that this method of constructing an array ref from a list is roughly 40% faster than
[ LIST ]
, but with the caveat and feature that elements are aliases to the original list &\(LIST)
-
a synonym for
cap
, the symbols&\(...)
will perform the same action. it could be read as taking the subroutine style reference of a list. like all symbol variables, once imported,&\
is global across all packages.my $capture = & \(my $x, my $y); # a space between & and \ is fine # and it looks a bit more syntactic ($x, $y) = (1, 2); say "@$capture"; # 1 2
generators
in this document, a generator is an object similar to an array that generates its elements on demand. generators can be used as iterators in perl's list control structures such as for/foreach
and while
. generators, like programmers, are lazy. unless they have to, they will not calculate or store anything. this laziness allows infinite generators to be created. you can choose to explicitly cache a generator, and several generators have implicit caches for efficiency.
there are source generators, which can be numeric ranges, arrays, or iterative subroutines. these can then be modified by wrapping each element with a subroutine, filtering elements, or combining generators with other generators. all of this behavior is lazy, only resolving generator elements at the latest possible time.
all generator functions return a blessed and overloaded reference to a tied array. this may sound a bit magical, but it just means that you can access the generator in a variety of ways, all which remain lazy.
given the generator:
my $gen = gen {$_**2} range 0, 100;
or gen {$_**2} 0, 100;
or range(0, 100)->map(sub {$_**2});
or <0..100>->map('**2');
or <**2 for 0..100>;
which describes the sequence of n**2 for n from 0 to 100 by 1
:
0 1 4 9 16 25 ... 9604 9801 10000
the following lines are equivalent (each prints '25'
):
say $gen->get(5);
say $gen->(5);
say $gen->[5];
say $gen->drop(5)->head;
say $gen->('5..')->head;
as are these (each printing '25 36 49 64 81 100'
):
say "@$gen[5 .. 10]";
say join ' ' => $gen->slice(5 .. 10);
say join ' ' => $gen->(5 .. 10);
say join ' ' => @$gen[5 .. 10];
say $gen->slice(range 5 => 10)->str;
say $gen->drop(5)->take(6)->str;
say $gen->(<5..10>)->str;
say $gen->('5..10')->str;
generators as arrays
you can access generators as if they were array references. only the requested indicies will be generated.
my $range = range 0, 1_000_000, 0.2;
# will produce 0, 0.2, 0.4, ... 1000000
say "@$range[10 .. 15]"; # calculates 6 values: 2 2.2 2.4 2.6 2.8 3
my $gen = gen {$_**2} $range; # attaches a generator function to a range
say "@$gen[10 .. 15]"; # '4 4.84 5.76 6.76 7.84 9'
for (@$gen) {
last if $_ > some_condition;
# the iteration of this loop is lazy, so when exited
# with `last`, no extra values are generated
...
}
generators in loops
evaluation in each of these looping examples remains lazy. using last
to escape from the loop early will result in some values never being generated.
... for @$gen;
for my $x (@$gen) {...}
... while <$gen>;
while (my ($next) = $gen->()) {...}
there are also looping methods, which take a subroutine. calling last
from the subroutine works the same as in the examples above.
$gen->do(sub {...}); or ->each
For {$gen} sub {
... # indirect object syntax
};
there is also a user space subroutine named &last
that is installed into the calling namespace during the execution of the loop. calling it without arguments has the same function as the builtin last
. calling it with an argument will still end the looping construct, but will also cause the loop to return the argument. the done ...
exception also works the same way as &last(...)
my $first = $gen->do(sub {&last($_) if /something/});
# same as: $gen->first(qr/something/);
you can use generators as file handle iterators:
local $_;
while (<$gen>) { # calls $gen->next internally
# do something with $_
}
generators as objects
all generators have the following methods by default
iteration:
$gen->next # iterates over generator ~~ $gen->get($gen->index++) $gen->() # same. iterators return () when past the end $gen->more # test if $gen->index not past end $gen->reset # reset iterator to start $gen->reset(4) # $gen->next returns $$gen[4] $gen->index # fetches the current position $gen->index = 4 # same as $gen->reset(4) $gen->nxt # next until defined $gen->iterator # returns the $gen->next coderef iterator
indexing:
$gen->get(index) # returns $$gen[index] $gen->(index) # same $gen->slice(4 .. 12) # returns @$gen[4 .. 12] $gen->(4 .. 12) # same $gen->size # returns 'scalar @$gen' $gen->all # same as list context '@$gen' but faster $gen->list # same as $gen->all
printing:
$gen->join(' ') # join ' ', $gen->all $gen->str # join $", $gen->all (recursive with nested generators) $gen->str(10) # limits generators to 10 elements $gen->perl # serializes the generator in array syntax (recursive) $gen->perl(9) # limits generators to 9 elements $gen->perl(9, '...') # prints ... at the end of each truncated generator $gen->print(...); # print $gen->str(...) $gen->say(...); # print $gen->str(...), $/ $gen->say(*FH, ...) # print FH $gen->str(...), $/ $gen->dump(...) # print $gen->perl(...), $/ $gen->debug # carps debugging information $gen->watch(...) # prints ..., value, $/ each time a value is requested
eager looping:
$gen->do(sub {...}) # for (@$gen) {...} # but faster $gen->each(sub{...}) # same
slicing:
$gen->head # $gen->get(0) $gen->tail # $gen->slice(<1..>) # lazy slices $gen->drop(2) # $gen->slice(<2..>) $gen->take(4) # $gen->slice(<0..3>) $gen->x_xs # ($gen->head, $gen->tail)
accessors:
$gen->range # range(0, $gen->size - 1) $gen->keys # same as $gen->range, but a list in list context $gen->values # same as $gen, but a list in list context $gen->kv # zip($gen->range, $gen) $gen->pairs # same as ->kv, but each pair is a tuple (array ref)
randomization:
$gen->pick # return a random element from $gen $gen->pick(n) # return n random elements from $gen $gen->roll # same as pick $gen->roll(n) # pick and replace $gen->shuffle # a lazy shuffled generator $gen->random # an infinite generator that returns random elements
searching:
$gen->first(sub {$_ > 5}) # first {$_ > 5} $gen->all # but faster $gen->first('>5') # same $gen->last(...) # $gen->reverse->first(...) $gen->first_idx(...) # same as first, but returns the index $gen->last_idx(...)
sorting:
$gen->sort # sort $gen->all $gen->sort(sub {$a <=> $b}) # sort {$a <=> $b} $gen->all $gen->sort('<=>') # same $gen->sort('uc', 'cmp') # does: map {$$_[0]} # sort {$$a[1] cmp $$b[1]} # map {[$_ => uc]} $gen->all
reductions:
$gen->reduce(sub {$a + $b}) # reduce {$a + $b} $gen->all $gen->reduce('+') # same $gen->sum # $gen->reduce('+') $gen->product # $gen->reduce('*') $gen->scan('+') # [$$gen[0], sum(@$gen[0..1]), sum(@$gen[0..2]), ...] $gen->min # min $gen->all $gen->max # max $gen->all
transforms:
$gen->cycle # infinite repetition of a generator $gen->rotate(1) # [$gen[1], $gen[2] ... $gen[-1], $gen[0]] $gen->rotate(-1) # [$gen[-1], $gen[0], $gen[1] ... $gen[-2]] $gen->uniq # $gen->filter(do {my %seen; sub {not $seen{$_}++}}) $gen->deref # tuples($a, $b)->deref ~~ zip($a, $b)
combinations:
$gen->zip($gen2, ...) # takes any number of generators or array refs $gen->cross($gen2) # cross product $gen->cross2d($gen2) # returns a 2D generator containing the same # elements as the flat ->cross generator $gen->tuples($gen2) # tuples($gen, $gen2)
the
zip
and thecross
methods all use the comma operator (','
) by default to join their arguments. if the first argument to any of these methods is code or a code like string, that will be used to join the arguments. more detail in the overloaded operators section below$gen->zip(',' => $gen2) # same as $gen->zip($gen2) $gen->zip('.' => $gen2) # $gen[0].$gen2[0], $gen[1].$gen2[1], ...
introspection:
$gen->type # returns the package name of the generator $gen->is_mutable # can the generator change size?
utility:
$gen->apply # causes a mutable generator to determine its true size $gen->clone # copy a generator, resets the index $gen->copy # copy a generator, preserves the index $gen->purge # purge any caches in the source chain
traversal:
$gen->leaves # returns a coderef iterator that will perform a depth first # traversal of the edge nodes in a tree of nested generators. # a full run of the iterator will ->reset all of the internal # generators
while:
$gen->while(...) # While {...} $gen $gen->take_while(...) # same $gen->drop_while(...) # $gen->drop( $gen->first_idx(sub {...}) ) $gen->span # collects $gen->next calls until one # returns undef, then returns the collection. # ->span starts from and moves the ->index $gen->span(sub{...}) # span with an argument splits the list when the code # returns false, it is equivalent to but more efficient # than ($gen->take_while(...), $gen->drop_while(...)) $gen->break(...) # $gen->span(sub {not ...})
tied vs methods:
the methods duplicate and extend the tied functionality and are necessary when working with indices outside of perl's array limit
(0 .. 2**31 - 1)
or when fetching a list return value (perl clamps the return to a scalar with the array syntax). in all cases, they are also faster than the tied interface.functions as methods:
most of the functions in this package are also methods of generators, including by, every, mapn, gen, map (alias of gen), filter, grep (alias of filter), test, cache, flip, reverse (alias of flip), expand, collect, overlay, mutable, while, until, recursive, rec (alias of recursive).
my $gen = (range 0, 1_000_000)->gen(sub{$_**2})->filter(sub{$_ % 2}); #same as: filter {$_ % 2} gen {$_**2} 0, 1_000_000;
dwim code:
when a method takes a code ref, that code ref can be specified as a string containing an operator and an optional curried argument (on either side)
my $gen = <0 .. 1_000_000>->map('**2')->grep('%2'); # same as above
you can prefix
!
ornot
to negate the operator:my $even = <1..>->grep('!%2'); # sub {not $_ % 2}
you can even use a typeglob to specify an operator when the method expects a binary subroutine:
say <1 .. 10>->reduce(*+); # 55 # and saves a character over '+'
or a regex ref:
<1..30>->grep(qr/3/)->say; # 3 13 23 30
you can flip the arguments to a binary operator by prefixing it with
R
or by applying the~
operator to it:say <a..d>->reduce('R.'); # 'dcba' # lowercase r works too say <a..d>->reduce(~'.'); # 'dcba' say <a..d>->reduce(~*.); # 'dcba'
methods without return values:
the methods that do not have a useful return value, such as
->say
, return the same generator they were called with. this lets you easily insert these methods at any point in a method chain for debugging.
predicates
several predicates are available to use with the filtering methods:
<1..>->grep('even' )->say(5); # 2 4 6 8 10
<1..>->grep('odd' )->say(5); # 1 3 5 7 9
<1..>->grep('prime')->say(5); # 2 3 5 7 11
<1.. if prime>->say(5); # 2 3 5 7 11
others are: defined, true, false
lazy slices
if you call the slice
method with a range
or other numeric generator as its argument, the method will return a generator that will perform the slice
my $gen = gen {$_ ** 2};
my $slice = $gen->slice(range 100 => 1000); # nothing calculated
say "@$slice[5 .. 10]"; # 6 values calculated
or using the glob syntax:
my $slice = $gen->slice(<100 .. 1000>);
infinite slices are fine:
my $tail = $gen->slice(<1..>);
lazy slices also work with the dwim code-deref syntax:
my $tail = $gen->(<1..>);
stacked continuous lazy slices collapse into a single composite slice for efficiency
my $slice = $gen->(<1..>)->(<1..>)->(<1..>);
$slice == $gen->(<3..>);
if you choose not to import the glob
function, you can still write ranges succinctly as strings, when used as arguments to slice:
my $tail = $gen->('1..');
my $tail = $gen->slice('1..');
dwim code dereference
when dereferenced as code, a generator decides what do do based on the arguments it is passed.
$gen->() ~~ $gen->next
$gen->(1) ~~ $gen->get(1) or $$gen[1]
$gen->(1, 2, ...) ~~ $gen->slice(1, 2, ...) or @$gen[1, 2, ...]
$gen->(<1..>) ~~ $gen->slice(<1..>) or $gen->tail
if passed a code ref or regex ref, ->map
will be called with the argument, if passed a reference to a code ref or regex ref, ->grep
will be called.
my $pow2 = <0..>->(sub {$_**2}); # calls ->map(sub{...})
my $uc = $gen->(\qr/[A-Z]/); # calls ->grep(qr/.../)
you can lexically enable code coercion from strings (experimental):
local $List::Gen::DWIM_CODE_STRINGS = 1;
my $gen = <0 .. 1_000_000>->('**2')(\'%2');
^map ^grep
due to some scoping issues, if you want to install this dwim coderef into a subroutine, the reliable way is to call the ->code
method:
*fib = <0, 1, *+*...>->code; # rather than *fib = \&{<0, 1, *+*...>}
overloaded operators
to make the usage of generators a bit more syntactic the following operators are overridden:
$gen1 x $gen2 ~~ $gen1->cross($gen2)
$gen1 x'.'x $gen2 ~~ $gen1->cross('.', $gen2)
or $gen1->cross(sub {$_[0].$_[1]}, $gen2)
$gen1 x sub{$_[0].$_[1]} x $gen2 # same as above
$gen1 + $gen2 ~~ sequence $gen1, $gen2
$g1 + $g2 + $g3 ~~ sequence $g1, $g2, $g3 # or more
$gen1 | $gen2 ~~ $gen1->zip($gen2)
$gen1 |'+'| $gen2 ~~ $gen1->zip('+', $gen2)
or $gen1->zip(sub {$_[0] + $_[1]}, $gen2)
$gen1 |sub{$_[0]+$_[1]}| $gen2 # same as above
$x | $y | $z ~~ $x->zip($y, $z)
$w | $x | $y | $z ~~ $w->zip($x, $y, $z) # or more
if the first argument to a ->zip
or ->cross
method is not an array or generator, it is assumed to be a subroutine and the corresponding ->(zip|cross)with
method is called:
$gen1->zipwith('+', $gen2) ~~ $gen1->zip('+', $gen2);
hyper operators:
not quite as elegant as perl6's hyper operators, but the same idea. these are similar to zipwith
but with more control over the length of the returned generator. all of perl's non-mutating binary operators are available to use as strings, or you can use a subroutine.
$gen1 <<'.'>> $gen2 # longest list
$gen1 >>'+'<< $gen2 # equal length lists or error
$gen1 >>'-'>> $gen2 # length of $gen2
$gen1 <<'=='<< $gen2 # length of $gen1
$gen1 <<sub{...}>> $gen2
$gen1 <<\&some_sub>> $gen2
my $x = <1..> <<'.'>> 'x';
$x->say(5); # '1x 2x 3x 4x 5x'
in the last example, a bare string is the final element, and precedence rules keep everything working. however, if you want to use a non generator as the first element, a few parens are needed to force the evaluation properly:
my $y = 'y' <<('.'>> <1..>);
$y->say(5); # 'y1 y2 y3 y4 y5'
otherwise 'y' << '.'
will run first without overloading, which will be an error. since that is a bit awkward, where you can specify an operator string, you can prefix R
or r
to indicate that the arguments to the operator should be reversed.
my $y = <1..> <<'R.'>> 'y';
$y->say(5); # 'y1 y2 y3 y4 y5'
just like in perl6, hyper operators are recursively defined for multi dimensional generators.
say +(list(<1..>, <2..>, <3..>) >>'*'>> -1)->perl(4, '...')
# [[-1, -2, -3, -4, ...], [-2, -3, -4, -5, ...], [-3, -4, -5, -6, ...]]
hyper operators currently do not work with mutable generators. this will be addressed in a future update.
you can also specify the operator in a hyper-operator as a typeglob:
my $xs = <1..> >>*.>> 'x'; # *. is equivalent to '.'
$xs->say(5); # 1x 2x 3x 4x 5x
my $negs = <0..> >>*-; # same as: <0..> >>'-'
$negs->say(5); # 0 -1 -2 -3 -4
hyper also works as a method:
<1..>->hyper('<<.>>', 'x')->say(5); # '1x 2x 3x 4x 5x'
# defaults to '<<...>>'
<1..>->hyper('.', 'x')->say(5); # '1x 2x 3x 4x 5x'
hyper negation can be done directly with the prefix minus operator:
-$gen ~~ $gen >>'-' ~~ $gen->hyper('-')
mutable generators
mutable generators (those returned from mutable, filter, While, Until, and iterate_multi) are generators with variable length. in addition to all normal methods, mutable generators have the following methods:
$gen->when_done(sub {...}) # schedule a method to be called when the
# generator is exhausted
# when_done can be called multiple times to
# schedule multiple end actions
$gen->apply; # causes the generator to evaluate all of its elements in
# order to find out its true size. it is a bad idea to call
# ->apply on an infinite generator
due to the way perl processes list operations, when perl sees an expression like:
print "@$gen\n"; # or
print join ' ' => @$gen;
it calls the internal FETCHSIZE
method only once, before it starts getting elements from the array. this is fine for immutable generators. however, since mutable generators do not know their true size, perl will think the array is bigger than it really is, and will most likely run off the end of the list, returning many undefined elements, or throwing an exception.
the solution to this is to call $gen->apply
first, or to use the $gen->all
method with mutable generators instead of @$gen
, since the ->all
method understands how to deal with arrays that can change size while being read.
perl's for/foreach
loop is a bit smarter, so just like immutable generators, the mutable ones can be dereferenced as the loop argument with no problem:
... foreach @$mutable_generator; # works fine
stream generators
the generators filter
, scan
, and iterate
(all of its flavors) have internal caches that allow random access within the generator. some algorithms only need monotonically increasing access to the generator (all access via repeated calls to $gen->next
for example), and the cache could become a performance/memory problem.
the *_stream
family of generators do not maintain an internal cache, and are subsequently unable to fulfill requests for indicies lower than or equal to the last accessed index. they will however be faster and use less memory than their non-stream counterparts when monotonically increasing access is all that an algorithm needs.
stream generators can be thought of as traditional subroutine iterators that also have generator methods. it is up to you to ensure that all operations and methods follow the monotonically increasing index rule. you can determine the current position of the stream iterator with the $gen->index
method.
my $nums = iterate_stream{2*$_}->from(1);
say $nums->(); # 1
say $nums->(); # 2
say $nums->(); # 4
say $nums->index; # 3
say $nums->drop( $nums->index )->str(5); # '8 16 32 64 128'
say $nums->index; # 8
the $gen->drop( $gen->index )->method
pattern can be shortened to $gen->idx->method
say $nums->idx->str(5); # '256 512 1024 2048 4096'
the $gen->index
method of stream generators is read only. calling $gen->reset
on a stream generator will throw an error.
stream generators are experimental and may change in future versions.
threads
generators have the following multithreaded methods:
$gen->threads_blocksize(3) # sets size to divide work into
$gen->threads_cached; # implements a threads::shared cache
$gen->threads_cached(10) # as normal, then calls threads_start with arg
$gen->threads_start; # creates 4 worker threads
$gen->threads_start(2); # or however many you want
# if you don't call it, threads_slice will
my @list = $gen->threads_slice(0 .. 1000); # sends work to the threads
my @list = $gen->threads_all;
$gen->threads_stop; # or let the generator fall out of scope
all threads are local to a particular generator, they are not shared. if the passed in generator was cached (at the top level) that cache is shared and used automatically. this includes most generators with implicit caches. threads_slice and threads_all can be called without starting the threads explicitly. in that case, they will start with default values.
the threaded methods only work in perl versions 5.10.1 to 5.12.x, patches to support other versions are welcome.
source generators
- range
SIZE
-
returns a generator from
0
toSIZE - 1
my $range = range 10; say $range->str; # 0 1 2 3 4 5 6 7 8 9 say $range->size; # 10
- range
START STOP [STEP]
-
returns a generator for values from
START
toSTOP
bySTEP
, inclusive.STEP
defaults to 1 but can be fractional and negative. depending on your choice ofSTEP
, the last value returned may not always beSTOP
.range(0, 3, 0.4) will return (0, 0.4, 0.8, 1.2, 1.6, 2, 2.4, 2.8) print "$_ " for @{range 0, 1, 0.1}; # 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 print "$_ " for @{range 5, 0, -1}; # 5 4 3 2 1 0 my $nums = range 0, 1_000_000, 2; print "@$nums[10, 100, 1000]"; # gets the tenth, hundredth, and thousandth numbers in the range # without calculating any other values
range
also accepts character strings instead of numbers. it will behave the same way as perl's internal..
operator, except it will be lazy.say range('a', 'z')->str; # 'a b c d e f g ... x y z' range('a', 'zzz', 2)->say; # 'a c e g i k m ... zzu zzw zzy' say <A .. ZZ>->str; # 'A B C D E ... ZX ZY ZZ' <1..>->zip(<a..>)->say(10); # '1 a 2 b 3 c 4 d 5 e'
to specify an infinite range, you can pass
range
an infinite value (9**9**9
works well), or the glob**
, or the string'*'
range(1, 9**9**9) ~~ range(1, **) ~~ range(1, '*') ~~ <1..*> ~~ <1..>
ranges only store their endpoints, and ranges of all sizes take up the same amount of memory.
- gen
{CODE} GENERATOR
- gen
{CODE} ARRAYREF
- gen
{CODE} SIZE
- gen
{CODE} [START STOP [STEP]]
- gen
{CODE} GLOBSTRING
-
gen
is the equivalent ofmap
for generators. it returns a generator that will apply theCODE
block to its source when accessed.gen
takes a generator, array ref, glob-string, or suitable arguments forrange
as its source. with no arguments,gen
uses the range0 .. infinity
.my @result = map {slow($_)} @source; # slow() called @source times my $result = gen {slow($_)} \@source; # slow() not called my ($x, $y) = @$result[4, 7]; # slow() called twice my $lazy = gen {slow($_)} range 1, 1_000_000_000; same: gen {slow($_)} 1, 1_000_000_000; print $$lazy[1_000_000]; # slow() only called once
gen {...} list LIST
is a replacement for[ map {...} LIST ]
.gen
provides the functionality of the identical->gen(...)
and->map(...)
methods.note that while effort has gone into making generators as fast as possible there is overhead involved with lazy generation. simply replacing all calls to
map
withgen
will almost certainly slow down your code. use these functions in situations where the time / memory required to completely generate the list is unacceptable.gen
and other similarly argumented functions in this package can also accept a string suitable for the<glob>
syntax:my $square_of_nats = gen {$_**2} '1..'; my $square_of_fibs = gen {$_**2} '0, 1, *+*'; # no need for '...' with '*'
which is the same as the following if
glob
is imported:my $square_of_nats = gen {$_**2} <1..>; my $square_of_fibs = gen {$_**2} <0, 1, *+* ...>; # still need dots here
- makegen
ARRAY
-
makegen
converts an array to a generator. this is normally not needed as most generator functions will call it automatically if passed an array referencemakegen
considers the length ofARRAY
to be immutable. changing the length of an array after passing it tomakegen
(or togen
and like argumented subroutines) will result in undefined behavior. this is done for performance reasons. if you need a length mutable array, use thearray
function. changing the value of a cell in the array is fine, and will be picked up by a generator (of course if the generator uses a cache, the value won't change after being cached).you can assign to the generator returned by
makegen
, provided the assignment does not lengthen the array.my $gen = makegen @array; $$gen[3] = 'some value'; # now $array[3] is 'some value'
- list
LIST
-
list
converts a list to a generator. it is a thin wrapper aroundmakegen
that simply passes its@_
tomakegen
. that means the values in the returned generator are aliases tolist
's arguments.list(2, 5, 8, 11)->map('*2')->say; # '4 10 16 22'
is the same as writing:
(gen {$_*2} cap 2, 5, 8, 11)->say;
in the above example,
list
can be used in place ofcap
and has exactly the same functionality:(gen {$_*2} list 2, 5, 8, 11)->say;
- array
[ARRAY]
-
array
is similar tomakegen
except the array is considered a mutable data source. because of this, certain optimizations are not possible, and the generator returned will be a bit slower than the one created bymakegen
in most conditions (increasing as generator functions are stacked).it is ok to modify
ARRAY
after creating the generator. it is also possible to use normal array modification functions such aspush
,pop
,shift
,unshift
, andsplice
on the generator. all changes will translate back to the source array.you can think of
array
as converting an array to an array reference that is also a generator.my @src = 1..5; my $gen = array @src; push @$gen, 6; $$gen[6] = 7; # assignment is ok too say $gen->size; # 7 say shift @$gen; # 1 say $gen->size; # 6 say $gen->str; # 2 3 4 5 6 7 say "@src"; # 2 3 4 5 6 7 my $array = array; # no args creates an empty array
- file
FILE [OPTIONS]
-
file
creates anarray
generator from a file name or file handle usingTie::File
.OPTIONS
are passed toTie::File
my $gen = file 'some_file.txt'; my $uc_file = $gen->map('uc'); my $with_line_numbers = <1..>->zip('"$a: $b"', $gen);
- repeat
SCALAR [SIZE]
-
an infinite generator that returns
SCALAR
for every position. it is equivalent togen {SCALAR}
but a little faster. - iterate
{CODE} [LIMIT|GENERATOR]
-
iterate
returns a generator that is created iteratively.iterate
implicitly caches its values, this allows random access normally not possible with an iterative algorithm. LIMIT is an optional number of times to iterate. normally, inside the CODE block,$_
is set to the current iteration number. if passed a generator instead of a limit,$_
will be set to sequential values from that generator.my $fib = do { my ($x, $y) = (0, 1); iterate { my $return = $x; ($x, $y) = ($y, $x + $y); $return } };
generators produced by
iterate
have an extra method,->from(LIST)
. the method must be called before values are accessed from the generator. the passedLIST
will be the first values returned by the generator. the method also changes the behavior of$_
inside the block.$_
will contain the previous value generated by the iterator. this allowsiterate
to behave the same way as the like named haskell function.haskell: take 10 (iterate (2*) 1) perl: iterate{2*$_}->from(1)->take(10) <1, 2 * * ... 10> <1,2**...10>
which all return
[1, 2, 4, 8, 16, 32, 64, 128, 256, 512]
- iterate_stream
{CODE} [LIMIT]
-
iterate_stream
is a version ofiterate
that does not cache the generated values. because of this, access to the returned generator must be monotonically increasing (such as repeated calls to$gen->next
). - iterate_multi
{CODE} [LIMIT]
-
the same as
iterate
, except CODE can return a list of any size. inside CODE,$_
is set to the position in the returned generator where the block's returned list will be placed.the returned generator from
iterate_multi
can be modified withpush
,pop
,shift
,unshift
, andsplice
like a normal array. it is up to you to ensure that the iterative algorithm will still work after modifying the array.the
->from(...)
method can be called on the returned generator. seeiterate
for the rules and effects of this. - iterate_multi_stream
{CODE} [LIMIT]
-
iterate_multi_stream
is a version ofiterate_multi
that does not cache the generated values. because of this, access to the returned generator must be monotonically increasing (such as repeated calls to$gen->next
).keyword modification of a stream iterator (with
push
,shift
, ...) is not supported. - gather
{CODE} [LIMIT]
-
gather
returns a generator that is created iteratively. rather than returning a value, you calltake($return_value)
within theCODE
block. note that since perl5 does not have continuations,take(...)
does not pause execution of the block. rather, it stores the return value, the block finishes, and then the generator returns the stored value.you can not import the
take(...)
function from this module.take(...)
will be installed automatically into your namespace during the execution of theCODE
block. because of this, you must always calltake(...)
with parenthesis.take
returns its argument unchanged.gather implicitly caches its values, this allows random access normally not possible with an iterative algorithm. the algorithm in
iterate
is a bit cleaner here, butgather
is slower thaniterate
, so benchmark if speed is a concernmy $fib = do { my ($x, $y) = (0, 1); gather { ($x, $y) = ($y, take($x) + $y) } };
a non-cached version
gather_stream
is also available, seeiterate_stream
- gather_multi
{CODE} [LIMIT]
-
the same as
gather
except you cantake(...)
multiple times, and each can take a list.gather_multi_stream
is also available. - stream
{CODE}
-
in the
CODE
block, calls to functions or methods with stream versions will be replaced by those versions. this applies also to functions that are called internally byList::Gen
(such as in the glob syntax).stream
returns whatCODE
returns.say iterate{}->type; # List::Gen::Iterate say iterate_stream{}->type; # List::Gen::Iterate_Stream stream { say iterate{}->type; # List::Gen::Iterate_Stream }; say stream{iterate{}}->type; # List::Gen::Iterate_Stream say stream{<1.. if even>}->type; # List::Gen::Filter_Stream
placing code inside a
stream
block is exactly the same as placinglocal $List::Gen::STREAM = 1;
at the top of a block. - glob
STRING
- <list comprehension>
-
by default, this module overrides perl's default
glob
function. this is because theglob
function provides the behavior of the angle bracket delimited<*.ext>
operator, which is a nice place for inserting list comprehensions into perl's syntax. the override causesglob()
and the<*.ext>
operator to have a few special cases overridden, but any case that is not overridden will be passed to perl's internalglob
function (my @files = <*.txt>;
works as normal).there are several types of overridden operations:
range: < [prefix,] low .. [high] [by step] > iterate: < [prefix,] code ... [size] > list comprehension: < [code for] (range|iterate) [if code] [while code] > reduction: < \[op|name\] (range|iterate|list comprehension) >
range strings match the following pattern:
(prefix,)? number .. number? ((by | += | -= | [-+]) number)?
here are a few examples of valid ranges:
<1 .. 10> ~~ range 1, 10 <0 .. > ~~ range 0, 9**9**9 <0 .. *> ~~ range 0, 9**9**9 <1 .. 10 by 2> ~~ range 1, 10, 2 <10 .. 1 -= 2> ~~ range 10, 1, -2 <a .. z> ~~ range 'a', 'z' <A .. ZZ> ~~ range 'A', 'ZZ' <a..> ~~ range 'a', 9**9**9 <a.. += b> ~~ range 'a', 9**9**9, 2 <0, 0..> ~~ [0] + range 0, 9**9**9 <'a','ab', 0..> ~~ ['a','ab'] + range 0, 9**9**9 <qw(a ab), 0..> ~~ [qw(a ab)] + range 0, 9**9**9
iterate strings match the following pattern:
(.+? ,)+ (.*[*].* | \{ .+ }) ... number?
such as:
my $fib = <0, 1, * + * ... *>;
which means something like:
my $fib = do { my @pre = (0, 1); my $self; $self = iterate { @pre ? shift @pre : $self->get($_ - 2) + $self->get($_ - 1) } 9**9**9 };
a few more examples:
my $fib = <0, 1, {$^a + $^b} ... *>; my $fac = <1, * * _ ... *>; my $int = <0, * + 1 ... *>; my $fib = <0,1,*+*...>; # ending star is optional
list comprehension strings match:
( .+ (for | [:|]) )? (range | iterate) ( (if | unless | [?,]) .+ )? ( (while | until ) .+ )?
examples:
<**2: 1 .. 10> ~~ gen {$_**2} range 1, 10 <**2: 1 .. 10 ? %2> ~~ gen {$_**2} filter {$_ % 2} range 1, 10 <sin: 0 .. 3.14 += 0.01> ~~ gen {sin} range 0, 3.14, 0.01 <1 .. 10 if % 2> ~~ filter {$_ % 2} range 1, 10 <sin for 0 .. 10 by 3 if /5/> ~~ gen {sin} filter {/5/} range 0, 10, 3 <*3 for 0 .. 10 unless %3> ~~ gen {$_ * 3} filter {not $_ % 3} 0, 10 <0 .. 100 while \< 10> ~~ While {$_ < 10} range 0, 100 <*2 for 0.. if %2 while \<10> ~~ <0..>->grep('%2')->while('<10')->map('*2')
there are three delimiter types available for basic list comprehensions:
terse: <*2: 1.. ?%3> haskell: <*2| 1.., %3> verbose: <*2 for 1.. if %3>
you can mix and match
<*2 for 1.., %3>
,<*2| 1.. ?%3>
in the above examples, most of the code areas are using abbreviated syntax. here are a few equivalencies:
<*2:1..?%3> ~~ <*2 for 1.. if %3> ~~ <\$_ * 2 for 1 .. * if \$_ % 3> <1.. if even> ~~ <1.. if not %2> ~~ <1..?!%2> ~~ <1.. if not _ % 2> ~~ <1.. unless %2> ~~ <1..* if not \$_ % 2> <1.. if %2> ~~ <1.. if _%2> ~~ <1..* ?odd> ~~ <1.. ? \$_ % 2>
reduction strings match:
\[operator | function_name\] (range | iterate | list comp)
examples:
say <[+] 1..10>; # prints 55
pre/post fixing the operator with '..' uses the
scan
function instead ofreduce
my $fac = <[..*] 1..>; # read as "a running product of one to infinity" my $sum = <[+]>; # no argument returns the reduction function say $sum->(1 .. 10); # 55 say $sum->(<1..10>); # 55 my $rev_cat = <[R.]>; # prefix the operator with `R` to reverse it say $rev_cat->(1 .. 9); # 987654321
all of these features can be used together:
<[+..] *2 for 0 .. 100 by 2 unless %3 >
which is the same as:
range(0, 100, 2)->grep('not %3')->map('*2')->scan('+')
when multiple features are used together, the following construction order is used:
1. prefix 2. range or iterate 3. if / unless (grep) 4. while / until (while) 5. for (map) 6. reduce / scan ([prefix] + (range|iterate))->grep(...)->while(...)->map(...)->reduce(...)
bignums
when run in perl 5.9.4+, glob strings will honor the lexical pragmas
bignum
,bigint
, andbigrat
.*factorial = do {use bigint; <[..*] 1, 1..>->code}; say factorial(25); # 15511210043330985984000000
special characters
since the angle brackets (
<
and>
) are used as delimiters of the glob string, they both must be escaped with\
if used in the<...>
construct.<1..10 if \< 5>->say; # 1 2 3 4
due to
<...>
being aqq{}
string, in the code areas if you need to write$_
write it without the sigil as_
<1 .. 10 if _**2 \> 40>->say; # 7 8 9 10
it can be escaped
\$_
as well.neither of these issues apply to calling glob directly with a single quoted string:
glob('1..10 if $_ < 5')->say; # 1 2 3 4
- List::Gen
...
-
the subroutine
Gen
in the packageList::
is a dwimmy function that produces a generator from a variety of sources. sinceList::Gen
is a fully qualified name, it is available from all packages without the need to import it.if given only one argument, the following table describes what is done:
array ref: List::Gen \@array ~~ makegen @array code ref: List::Gen sub {$_**2} ~~ <0..>->map(sub {$_**2}) scalar ref: List::Gen \'*2' ~~ <0..>->map('*2') glob string: List::Gen '1.. by 2' ~~ <1.. by 2> glob string: List::Gen '0, 1, *+*' ~~ <0, 1, *+*...> file handle: List::Gen $fh ~~ file $fh
if the argument does not match the table, or the method is given more than one argument, the list is converted to a generator with
list(...)
List::Gen(1, 2, 3)->map('2**')->say; # 2 4 8
since it results in longer code than any of the equivalent constructs, it is mostly for if you have not imported anything:
use List::Gen ();
- vecgen
[BITS] [SIZE] [DATA]
-
vecgen
wraps a bit vector in a generator. BITS defaults to 8. SIZE defaults to infinite. DATA defaults to an empty string.cells of the generator can be assigned to using array dereferencing:
my $vec = vecgen; $$vec[3] = 5;
or with the
->set(...)
method:$vec->set(3, 5);
- primes
-
utilizing the same mechanism as the
<1..>->grep('prime')
construct, theprimes
function returns an equivalent, but more efficiently constructed generator.If the module
Math::Prime::Util
is found in@INC
, the functionsnext_prime
andis_prime
thereof are used for generating and testing of primes. By setting$List::Gen::FORCE_PRIME
to true the below described internal implementation can be forced.Otherwise prime numbers below 1e7 are tested with a sieve of eratosthenes and should be reasonably efficient. beyond that, simple trial division is used.
primes
always returns the same generator.
modifying generators
- slice
SOURCE_GEN RANGE_GEN
-
slice
usesRANGE_GEN
to generate the indices used to take a lazy slice ofSOURCE_GEN
.my $gen = gen {$_ ** 2}; my $s1 = slice $gen, range 1, 9**9**9; my $s2 = slice $gen, <1..>; my $s3 = $gen->slice(<1..>); my $s4 = $gen->(<1..>); $s1 ~~ $s2 ~~ $s3 ~~ $s4 ~~ $gen->tail
slice
will perform some optimizations if it detects thatRANGE_GEN
is sufficiently simple (something likerange $x, $y, 1
). also, stacked simple slices will collapse into a single slice, which turns repeated tailing of a generator into a relatively efficient operation.$gen->(<1..>)->(<1..>)->(<1..>) ~~ $gen->(<3..>) ~~ $gen->tail->tail->tail
- test
{CODE} [ARGS_FOR_GEN]
-
test
attaches a code block to a generator. it takes arguments suitable for thegen
function. accessing an element of the returned generator will call the code block first with the element in$_
, and if it returns true, the element is returned, otherwise an empty list (undef in scalar context) is returned.when accessing a slice of a tested generator, if you use the
->(x .. y)
syntax, the the empty lists will collapse and you may receive a shorter slice. an array dereference slice will always be the size you ask for, and will have undef in each failed slotthe
$gen->nxt
method is a version of$gen->next
that continues to call->next
until a call returns a value, or the generator is exhausted. this makes the->nxt
method the easiest way to iterate over only the passing values of a tested generator. - cache
{CODE}
- cache
GENERATOR
- cache
list => ...
-
cache
will return a cached version of the generators returned by functions in this package. when passed a code reference, cache returns a memoized code ref (arguments joined with$;
). when in 'list' mode, the source is in list context, otherwise scalar context is used.my $gen = cache gen {slow($_)} \@source; # calls = 0 print $gen->[123]; # calls += 1 ... print @$gen[123, 456] # calls += 1
- flip
GENERATOR
-
flip
isreverse
for generators. the->apply
method is called onGENERATOR
.$gen->flip
and$gen->reverse
do the same thing.flip gen {$_**2} 0, 10 ~~ gen {$_**2} 10, 0, -1
- expand
GENERATOR
- expand
SCALE GENERATOR
-
expand
scales a generator with elements that return equal sized lists. it can be passed a list length, or will automatically determine it from the length of the list returned by the first element of the generator.expand
implicitly caches its returned generator.my $multigen = gen {$_, $_/2, $_/4} 1, 10; # each element returns a list say join ' '=> $$multigen[0]; # 0.25 # only last element say join ' '=> &$multigen(0); # 1 0.5 0.25 # works say scalar @$multigen; # 10 say $multigen->size; # 10 my $expanded = expand $multigen; say join ' '=> @$expanded[0 .. 2]; # 1 0.5 0.25 say join ' '=> &$expanded(0 .. 2); # 1 0.5 0.25 say scalar @$expanded; # 30 say $expanded->size; # 30 my $expanded = expand gen {$_, $_/2, $_/4} 1, 10; # in one line
expand
can also scale a generator that returns array references:my $refs = gen {[$_, $_.$_]} 3; say $refs->join(', '); # ARRAY(0x272514), ARRAY(0x272524), ARRAY(0x272544) say $refs->expand->join(', '); # 0, 00, 1, 11, 2, 22
expand
in array ref mode is the same as calling the->deref
method. - contract
SCALE GENERATOR
-
contract
is the inverse ofexpand
also called
collect
- scan
{CODE} GENERATOR
- scan
{CODE} LIST
-
scan
is areduce
that builds a list of all the intermediate values.scan
returns a generator, and is the function behind the<[..+]>
globstring reduction operator.(scan {$a * $b} <1, 1..>)->say(8); # 1 1 2 6 24 120 720 5040 40320 say <[..*] 1, 1..>->str(8); # 1 1 2 6 24 120 720 5040 40320 say <1, 1..>->scan('*')->str(8); # 1 1 2 6 24 120 720 5040 40320 say <[..*]>->(1, 1 .. 7)->str; # 1 1 2 6 24 120 720 5040 40320
you can even use the
->code
method to tersely define a factorial function:*factorial = <[..*] 1, 1..>->code; say factorial(5); # 120
a stream version
scan_stream
is also available. - overlay
GENERATOR PAIRS
-
overlay allows you to replace the values of specific generator cells. to set the values, either pass the overlay constructor a list of pairs in the form
index => value, ...
, or assign values to the returned generator using normal array ref syntaxmy $fib; $fib = overlay gen {$$fib[$_ - 1] + $$fib[$_ - 2]}; @$fib[0, 1] = (0, 1); # or my $fib; $fib = gen {$$fib[$_ - 1] + $$fib[$_ - 2]} ->overlay( 0 => 0, 1 => 1 ); print "@$fib[0 .. 15]"; # '0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610'
- recursive
[NAME] GENERATOR
-
recursive
defines a subroutine namedself(...)
orNAME(...)
during generator execution. when called with no arguments it returns the generator. when called with one or more numeric arguments, it fetches those indices from the generator. when called with a generator, it returns a lazy slice from the source generator. since the subroutine created byrecursive
is installed at runtime, you must call the subroutine with parenthesis.my $fib = gen {self($_ - 1) + self($_ - 2)} ->overlay( 0 => 0, 1 => 1 ) ->cache ->recursive; print "@$fib[0 .. 15]"; # '0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610'
when used as a method,
$gen->recursive
can be shortened to$gen->rec
.my $fib = ([0, 1] + iterate {sum fib($_, $_ + 1)})->rec('fib'); print "@$fib[0 .. 15]"; # '0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610'
of course the fibonacci sequence is better written with the glob syntax as
<0, 1, *+*...>
which is compiled into something similar to the example withiterate
above.
mutable generators
- filter
{CODE} [ARGS_FOR_GEN]
-
filter
is a lazy version ofgrep
which attaches a code block to a generator. it returns a generator that will test elements with the code block on demand.filter
processes its argument list the same waygen
does.filter
provides the functionality of the identical->filter(...)
and->grep(...)
methods.normal generators, such as those produced by
range
orgen
, have a fixed length, and that is used to allow random access within the range. however, there is no way to know how many elements will pass a filter. because of this, random access within the filter is not alwaysO(1)
.filter
will attempt to be as lazy as possible, but to access the 10th element of a filter, the first 9 passing elements must be found first. depending on the coderef and the source, the filter may need to process significantly more elements from its source than just 10.in addition, since filters don't know their true size, entire filter arrays do not expand to the correct number of elements in list context. to correct this, call the
->apply
method which will test the filter on all of its source elements. after that, the filter will return a properly sized array. calling->apply
on an infinite (or very large) range wouldn't be a good idea. if you are using->apply
frequently, you should probably just be usinggrep
. you can call->apply
on any stack of generator functions, it will start from the deepest filter and move up.the method
->all
will first call->apply
on itself and then return the complete listfilters implicitly cache their values. accessing any element below the highest element already accessed is
O(1)
.accessing individual elements or slices works as you would expect.
my $filter = filter {$_ % 2} 0, 100; say $#$filter; # incorrectly reports 100 say "@$filter[5 .. 10]"; # reads the source range up to element 23 # prints 11 13 15 17 19 21 say $#$filter; # reports 88, closer but still wrong $filter->apply; # reads remaining elements from the source say $#$filter; # 49 as it should be
note:
filter
now reads one element past the last element accessed, this allows filters to behave properly when dereferenced in a foreach loop (without having to call->apply
). if you prefer the old behavior, set$List::Gen::LOOKAHEAD = 0
or usefilter_ ...
- filter_stream
{CODE} ...
-
as
filter
runs, it builds up a cache of the elements that pass the filter. this enables efficient random access in the returned generator. sometimes this caching behavior causes certain algorithms to use too much memory.filter_stream
is a version offilter
that does not maintain a cache.normally, access to
*_stream
iterators must be monotonically increasing since their source can only produce values in one direction. filtering is a reversible algorithm, and subsequently filter streams are able to rewind themselves to any previous index. however, unlikefilter
, thefilter_stream
generator must test previously tested elements to rewind. things probably wont end well if the test code is non-deterministic or if the source values are changing.when used as a method, it can be spelled
$gen->filter_stream(...)
or$gen->grep_stream(...)
- While
{CODE} GENERATOR
- Until
{CODE} GENERATOR
-
While / ->while(...)
returns a new generator that will end when its passed in subroutine returns false. theuntil
pair ends when the subroutine returns true.if
$List::Gen::LOOKAHEAD
is true (the default), each reads one element past its requested element, and saves this value only until the next call for efficiency, no other values are saved. each supports random access, but is optimized for sequential access.these functions have all of the caveats of
filter
, should be considered experimental, and may change in future versions. the generator returned should only be dereferenced in aforeach
loop, otherwise, just like afilter
perl will expand it to the wrong size.the generator will return undef the first time an access is made and the check code indicates it is past the end.
the generator will throw an error if accessed beyond its dynamically found limit subsequent times.
my $pow = While {$_ < 20} gen {$_**2}; <0..>->map('**2')->while('< 20') say for @$pow;
prints:
0 1 4 9 16
in general, it is faster to write it this way:
my $pow = gen {$_**2}; $gen->do(sub { last if $_ > 20; say; });
- mutable
GENERATOR
$gen->mutable
-
mutable
takes a single fixed size (immutable) generator, such as those produced bygen
and converts it into a variable size (mutable) generator, such as those returned byfilter
.as with filter, it is important to not use full array dereferencing (
@$gen
) with mutable generators, since perl will expand the generator to the wrong size. to access all of the elements, use the$gen->all
method, or call$gen->apply
before@$gen
. using a slice@$gen[5 .. 10]
is always ok, and does not require calling->apply
.mutable generators respond to the
List::Gen::Done
exception, which can be produced with eitherdone
,done_if
, ordone_unless
. when the exception is caught, it causes the generator to set its size, and it also triggers any->when_done
actions.my $gen = mutable gen {done if $_ > 5; $_**2}; say $gen->size; # inf say $gen->str; # 0 1 4 9 16 25 say $gen->size; # 6
generators returned from
mutable
have a->set_size(int)
method that will set the generator's size and then trigger any->when_done(sub{...})
methods. - done
[LAST_RETURN_VALUE]
-
throws an exception that will be caught by a mutable generator indicating that the generator should set its size. if a value is passed to done, that will be the final value returned by the generator, otherwise, the final value will be the value returned on the previous call.
- done_if
COND VALUE
- done_unless
COND VALUE
-
these are convenience functions for throwing
done
exceptions. if the condition does not indicatedone
then the function returnsVALUE
- strict
{CODE}
-
in the
CODE
block, calls to functions or methods are subject to the following localizations:local $List::Gen::LOOKAHEAD = 0;
the functions
filter
,While
and their various forms normally stay an element ahead of the last requested element so that an array dereference in aforeach
loop ends properly. this localization disables this behavior, which might be needed for certain algorithms. it is therefore important to never write code like:for(@$strict_filtered){...}
, instead write$strict_filtered->do(sub{...})
which is faster as well. the following code illustrates the difference in behavior:my $test = sub { my $loud = filter {print "$_, "; $_ % 2}; print "($_:", $loud->next, '), ' for 0 .. 2; print $/; }; print 'normal: '; $test->(); print 'strict: '; strict {$test->()}; normal: 0, 1, 2, 3, (0:1), 4, 5, (1:3), 6, 7, (2:5), strict: 0, 1, (0:1), 2, 3, (1:3), 4, 5, (2:5),
local $List::Gen::DWIM_CODE_STRINGS = 0;
in the dwim
$gen->(...)
code deref syntax, if$DWIM_CODE_STRINGS
has been set to a true value, bare strings that look like code will be interpreted as code and passed togen
(string refs tofilter
). since this behavior is fun for golf, but potentially error prone, it is off by default.strict
turns it back off if it had been turned on.
strict
returns whatCODE
returns.strict
may have additional restrictions added to it in the future.
combining generators
- sequence
LIST
-
string generators, arrays, and scalars together.
sequence
provides the functionality of the overloaded+
operator on generators:my $seq = <1 .. 10> + <20 .. 30> + <40 .. 50>;
is exactly the same as:
my $seq = sequence <1 .. 10>, <20 .. 30>, <40 .. 50>;
you can even write things like:
my $fib; $fib = [0, 1] + iterate {sum $fib->($_, $_ + 1)}; say "@$fib[0 .. 10]"; # 0 1 1 2 3 5 8 13 21 34 55
- zipgen
LIST
-
zipgen
is a lazy version ofzip
. it takes any combination of generators and array refs and returns a generator. it is called automatically whenzip
is used in scalar context.zipgen
can be spelledgenzip
- unzip
LIST
-
unzip
is the opposite ofzip src1, src2
. unzip returns 2 generators, the first returning src1, the second, src2. ifLIST
is a single element, and is a generator, that generator will be unzipped. - unzipn
NUMBER LIST
-
unzipn
is the n-dimentional precursor ofunzip
. assuming a zipped list produced byzip
withn
elements,unzip n list
returnsn
lists corresponding to the lists originally passed tozip
. ifLIST
is a single element, and is a generator, that generator will be unzipped. if only passed 1 argument,unzipn
will return a curried version of itself:*unzip3 = unzipn 3; my $zip3 = zip <1..>, <2..>, <3..>; my ($x, $y, $z) = unzip3($zip3); # $x == <1..>, $y == <2..>, $z == <3..>;
- zipgenmax
LIST
-
zipgenmax
is a lazy version ofzipmax
. it takes any combination of generators and array refs and returns a generator. - zipwith
{CODE} LIST
-
zipwith
takes a code block and a list. theLIST
is zipped together and each sub-list is passed toCODE
when requested.zipwith
produces a generator with the same length as its shortest source list.my $triples = zipwith {\@_} <1..>, <20..>, <300..>; say "@$_" for @$triples[0 .. 3]; 1 20 300 # the first element of each list 2 21 301 # the second 3 22 302 # the third 4 23 303 # the fourth
- zipwithab
{AB_CODE} $gen1, $gen2
-
The zipwithab function takes a function which uses
$a
and$b
, as well as two lists and returns a list analogous to zipwith. - zipwithmax
{CODE} LIST
-
zipwithmax
is a version ofzipwith
that has the ending conditions ofzipgenmax
. - transpose
MULTI_DIMENSIONAL_ARRAY
- transpose
LIST
-
transpose
computes the 90 degree rotation of its arguments, which must be a single multidimensional array or generator, or a list of 1+ dimensional structures.say transpose([[1, 2, 3]])->perl; # [[1], [2], [3]] say transpose([[1, 1], [2, 2], [3, 3]])->perl; # [[1, 2, 3], [1, 2, 3]] say transpose(<1..>, <2..>, <3..>)->take(5)->perl; # [[1, 2, 3], [2, 3, 4], [3, 4, 5], [4, 5, 6], [5, 6, 7]]
- cartesian
{CODE} LIST
-
cartesian
computes the cartesian product of any number of array refs or generators, each which can be any size. returns a generatormy $product = cartesian {$_[0] . $_[1]} [qw/a b/], [1, 2]; @$product == qw( a1 a2 b1 b2 );
misc functions
- mapkey
{CODE} KEY LIST
-
this function is syntactic sugar for the following idiom
my @cartesian_product = map { my $first = $_; map { my $second = $_; map { $first . $second . $_ } 1 .. 3 } qw/x y z/ } qw/a b c/; my @cartesian_product = mapkey { mapkey { mapkey { $_{first} . $_{second} . $_{third} } third => 1 .. 3 } second => qw/x y z/ } first => qw/a b c/;
- mapab
{CODE} PAIRS
-
this function works like the builtin
map
but consumes a list in pairs, rather than one element at a time. inside theCODE
block, the variables$a
and$b
are aliased to the elements of the list. ifmapab
is called in void context, theCODE
block will be executed in void context for efficiency. ifmapab
is passed an uneven length list, in the final iteration,$b
will beundef
my %hash = (a => 1, b => 2, c => 3); my %reverse = mapab {$b, $a} %hash;
- slide
{CODE} WINDOW LIST
-
slides a
WINDOW
sized slice overLIST
, callingCODE
for each slice and collecting the resultas the window reaches the end, the passed in slice will shrink
print slide {"@_\n"} 2 => 1 .. 4 # 1 2 # 2 3 # 3 4 # 4 # only one element here
- remove
{CODE} ARRAY|HASH
-
remove
removes and returns elements from its source whenCODE
returns true. in the code block, if the source is an array,$_
is aliased to its elements. if the source is a hash,$_
is aliased to its keys (and a list of the removedkey => value
pairs are returned).my @array = (1, 7, 6, 3, 8, 4); my @removed = remove {$_ > 5} @array; say "@array"; # 1 3 4 say "@removed"; # 7 6 8
in list context,
remove
returns the list of removed elements/pairs. in scalar context, it returns the number of removals.remove
will not build a return list in void context for efficiency. - d
[SCALAR]
- deref
[SCALAR]
-
dereference a
SCALAR
,ARRAY
, orHASH
reference. any other value is returned unchangedprint join " " => map deref, 1, [2, 3, 4], \5, {6 => 7}, 8, 9, 10; # prints 1 2 3 4 5 6 7 8 9 10
- curse
HASHREF PACKAGE
-
many of the functions in this package utilize closure objects to avoid the speed penalty of dereferencing fields in their object during each access.
curse
is similar tobless
for these objects and while a blessing makes a reference into a member of an existing package, a curse conjures a new package to do the reference's biddingpackage Closure::Object; sub new { my ($class, $name, $value) = @_; curse { get => sub {$value}, set => sub {$value = $_[1]}, name => sub {$name}, } => $class }
Closure::Object
is functionally equivalent to the following normal perl object, but with faster method calls since there are no hash lookups or other dereferences (around 40-50% faster for short getter/setter type methods)package Normal::Object; sub new { my ($class, $name, $value) = @_; bless { name => $name, value => $value, } => $class } sub get {$_[0]{value}} sub set {$_[0]{value} = $_[1]} sub name {$_[0]{name}}
the trade off is in creation time / memory, since any good curse requires drawing at least a few pentagrams in the blood of an innocent package.
the returned object is blessed into the conjured package, which inherits from the provided
PACKAGE
. always use$obj->isa(...)
rather thanref $obj eq ...
due to this. the conjured package name matches/${PACKAGE}::_\d+/
special keys:
-bless => $reference # returned instead of HASHREF -overload => [fallback => 1, '""' => sub {...}]
when fast just isn't fast enough, since most cursed methods don't need to be passed their object, the fastest way to call the method is:
my $obj = Closure::Object->new('tim', 3); my $set = $obj->{set}; # fetch the closure # or $obj->can('set') $set->(undef, $_) for 1 .. 1_000_000; # call without first arg
which is around 70% faster than pre-caching a method from a normal object for short getter/setter methods, and is the method used internally in this module.
SEE ALSO
see List::Gen::Cookbook for usage tips.
see List::Gen::Benchmark for performance tips.
see List::Gen::Haskell for an experimental implementation of haskell's lazy list behavior.
see List::Gen::Lazy for the tools used to create List::Gen::Haskell.
see List::Gen::Lazy::Ops for some of perl's operators implemented as lazy haskell like functions.
see List::Gen::Lazy::Builtins for most of perl's builtin functions implemented as lazy haskell like functions.
see List::Gen::Perl6 for a source filter that adds perl6's meta operators to use with generators, rather than the default overloaded operators
CAVEATS
version 0.90 added glob
to the default export list (which gives you syntactic ranges <1 .. 10>
and list comprehensions.). version 0.90 also adds many new features and bug-fixes, as usual, if anything is broken, please send in a bug report. the ending conditions of zip
and zipgen
have changed, see the documentation above. test
has been removed from the default export list. setting $List::Gen::LIST
true to enable list context generators is no longer supported and will now throw an error. list
has been added to the default export list. genzip
has been renamed zipgen
version 0.70 comes with a bunch of new features, if anything is broken, please let me know. see filter
for a minor behavior change
versions 0.50 and 0.60 break some of the syntax from previous versions, for the better.
- code generation
-
a number of the syntactic shortcuts that List::Gen provides will construct and then evaluate code behind the scenes. Normally this is transparent, but if you are trying to debug a problem, hidden code is never a good thing. You can lexically enable the printing of evaled code with:
local $List::Gen::SAY_EVAL = 1; my $fib = <0, 1, *+*...>; # eval: ' @pre = (0, 1)' at (file.pl) line ## # eval: 'List::Gen::iterate { if (@pre) {shift @pre} # else { $fetch->(undef, $_ - 2) + $fetch->(undef, $_ - 1) } # } 9**9**9' at (file.pl) line ## my $gen = <1..10>->map('$_*2 + 1')->grep('some_predicate'); # eval: 'sub ($) {$_*2 + 1}' at (file.pl) line ## # eval: 'sub ($) {some_predicate($_)}' at (file.pl) line ##
a given code string is only evaluated once and is then cached, so you will not see any additional output when using the same code strings in multiple places. in some cases (like the iterate example above) the code is closing over external variables (
@pre
and$fetch
) so you will not be able to see everything, but$SAY_EVAL
should be a helpful debugging aid.any time that code evaluation fails, an immediate fatal error is thrown. the value of
$SAY_EVAL
does not matter in that case. - captures of compile time constructed lists
-
the
cap
function and its twin operator&\
are faster than the[...]
construct because they do not copy their arguments. this is why the elements of the captures remain aliased to their arguments. this is normally fine, but it has an interesting effect with compile time constructed constant lists:my $max = 1000; my $range = & \(1 .. $max); # 57% faster than [1 .. $max] my $nums = & \(1 .. 1000); # 366% faster than [1 .. 1000], but cheating
the first example shows the expected speed increase due to not copying the values into a new empty array reference. the second example is much faster at runtime than the
[...]
syntax, but this speed is deceptive. the reason is that the list being passed in as an argument is generated by the compiler before runtime begins. so all perl has to do is place the values on the stack, and call the function.normally this is fine, but there is one catch to be aware of, and that is that a capture of a compile time constant list in a loop or subroutine (or any structure that can execute the same segment of code repeatedly) will always return a reference to an array of the same elements.
# two instances give two separate arrays my ($a, $b) = (&\(1 .. 3), &\(1 .. 3)); $_ += 10 for @$a; say "@$a : @$b"; # 11 12 13 : 1 2 3 # here the one instance returns the same elements twice my ($x, $y) = map &\(1 .. 3), 1 .. 2; $_ += 10 for @$x; say "@$x : @$y"; # 11 12 13 : 11 12 13
this only applies to compile time constructed constant lists, anything containing a variable or non constant function call will give you separate array elements, as shown below:
my ($low, $high) = (1, 3); my ($x, $y) = map &\($low .. $high), 1 .. 2; # non constant list $_ += 10 for @$x; say "@$x : @$y"; # 11 12 13 : 1 2 3
AUTHOR
Eric Strom, <asg at cpan.org>
BUGS
overloading has gotten fairly complicated and is probably in need of a rewrite. if any edge cases do not work, please send in a bug report.
both threaded methods ($gen->threads_slice(...)
) and function composition with overloaded operators (made with List::Gen::Lazy::fn {...}
) do not work properly in versions of perl before 5.10. patches welcome
report any bugs / feature requests to bug-list-gen at rt.cpan.org
, or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=List-Gen.
comments / feedback / patches are also welcome.
COPYRIGHT & LICENSE
copyright 2009-2011 Eric Strom.
this program is free software; you can redistribute it and/or modify it under the terms of either: the GNU General Public License as published by the Free Software Foundation; or the Artistic License.
see http://dev.perl.org/licenses/ for more information.