The London Perl and Raku Workshop takes place on 26th Oct 2024. If your company depends on Perl, please consider sponsoring and/or attending.

NAME

PDL::Opt::ParticleSwarm - Particle Swarm Optimization (object oriented)

SYNOPSIS

use PDL::Opt::ParticleSwarm;

my $pso = PDL::Opt::ParticleSwarm->new (
    -fitFunc    => \&calcFit,
    -dimensions => 1,
    );

my $bestFit        = $pso->optimize();
my $bestPos        = $pso->getBestPos();

print "Fit $bestFit at $bestPos\n";

sub calcFit {
    my $vec = shift;
    my $x = $vec->slice('(0)');

    # The parabola (x+3)^2 - 5 has a minima at x=-3:
    return (($x+3)**2 - 5);
}

Description

The Particle Swarm Optimization technique uses communication of the current best position found between a number of particles moving over a hyper surface as a technique for locating the best location on the surface (where 'best' is the minimum of some fitness function). For a Wikipedia discussion of PSO see http://en.wikipedia.org/wiki/Particle_swarm_optimization.

This pure Perl module is an implementation of the Particle Swarm Optimization technique for finding minima of hyper surfaces using the PDL module for accelerated vector calculations. It presents an object oriented interface that facilitates easy configuration of the optimization parameters and (in principle) allows the creation of derived classes to reimplement all aspects of the optimization engine (a future version will describe the replaceable engine components).

This implementation allows communication of a local best point between a selected number of neighbours.

This module and its documentation is based on AI::ParticleSwarmOptimization by Peter Jaquiery to support PDL objects and additional features such as search space scaling and initial guess options.

Methods

PDL::Opt::ParticleSwarm provides the following public methods. The parameter lists shown for the methods denote optional parameters by showing them in [].

new(%parameters)

Create an optimization object. The following parameters may be used:

-initialGuess: a vector of initial "best" values to start with.

This must be a PDL object. As long as it is correctly broadcasts, this can take any PDL representation.

-searchSize: a scalar multiple to control the search distance from -initialGuess

For example, if your initialGuess is "5", posMin/posMax range is 0-10, and searchSize is 0.5 then it will initialize the particles to search in the region between 2.5 and 7.5.

If undefined, then search the entire space between posMin and posMax

Default value: undef

-dimensions: positive number, semi-required

The number of dimensions of the hypersurface being searched. This can be omitted if -intitialGuess is provided.

-exitFit: number, optional

If provided, -exitFit allows early termination of optimize if the fitness value becomes equal or less than -exitFit.

-fitFunc: required

-fitFunc is a reference to the fitness function used by the search. If extra parameters need to be passed to the fitness function an array ref may be used with the code ref as the first array element and parameters to be passed into the fitness function as following elements. User provided parameters are passed as the first parameters to the fitness function when it is called:

    my $pso = PDL::Opt::ParticleSwarm->new (
		fitFunc    => [\&calcFit, $context],
		dimensions => 3,
        );

    ...

    sub calcFit {
        my ($context, $pdl_vec_to_optimize) = @_;
		... do something with $pdl_vec_to_optimize
		return $fitness;
        }

In addition to any user provided parameters the list of values representing the current particle position in the hyperspace is passed in. There is one value per hyperspace dimension.

-logFunc: log function callback, optional

This function is called after each iteration with the current bestPos and bestFit values as follows:

$self->{logFunc}->($self->getBestPos(), $self->getBestFit());
-inertia: positive or zero number, optional

Determines what proportion of the previous velocity is carried forward to the next iteration. This can be a PDL object, so it should work in any dimension so long as it works in the broadcast sense.

Defaults to 0.9

See also -meWeight and -themWeight.

-iterations: number, optional

Number of optimization iterations to perform. Defaults to 1000.

-meWeight: number, optional

Coefficient determining the influence of the current local best position on the next iterations velocity. This can be a PDL object, so it should work in any dimension so long as it works in the broadcast sense.

Defaults to 0.5.

See also -inertia and -themWeight.

-numNeighbors: positive number, optional

Number of local particles considered to be part of the neighbourhood of the current particle.

Defaults to the square root of the total number of particles.

Background: "The basic version of the algorithm uses the global topology as the swarm communication structure [when (numNeighbors=numParticles-1)]. This topology allows all particles to communicate with all the other particles, thus the whole swarm share the same best position from a single particle. However, this approach might lead the swarm to be trapped into a local minimum." [ https://en.wikipedia.org/wiki/Particle_swarm_optimization ]

Thus, as numNeighbors approaches numParticles-1, there is a greater risk of getting stuck in a local minima.

The neighbor selection algorithm in PDL::Opt::ParticleSwarm uses numNeighbors particles following the Nth particle index in the particle piddle being evaluated for its neighbors. The term "neighbor" is perhaps a misnomer in the sense that it is not a vector distance, but an index distance.

Future work could include adding other neighbor selection algorithms (ie, random, vector distance, ring, others).

-numParticles: positive number, optional

Number of particles in the swarm. Defaults to 10 times the number of dimensions.

-posMax: number, optional

Maximum coordinate value for any dimension in the hyper space. This can be a PDL object, so it should work in any dimension so long as it works in the broadcast sense. For example, different dimensions could have different posMax values by passing a vector of length dimension.

Defaults to 100.

-posMin: number, optional

Minimum coordinate value for any dimension in the hyper space. This can be a PDL object, so it should work in any dimension so long as it works in the broadcast sense. For example, different dimensions could have different posMax values by passing a vector of length dimension.

Defaults to --posMax (if -posMax is negative -posMin should be set more negative).

-randStartVelocity: boolean, optional

Set true to initialize particles with a random velocity. Otherwise particle velocity is set to 0 on initalization.

A range based on 1/100th of --posMax - -posMin is used for the initial speed in each dimension of the velocity vector if a random start velocity is used.

-stallSpeed: positive number, optional

Speed below which a particle is considered to be stalled and is repositioned to a new random location with a new initial speed. This can be a PDL object, so it should work in any dimension so long as it works in the broadcast sense.

By default -stallSpeed is undefined but particles with a speed of 1e-9 will be repositioned.

-stallSearchScale: positive number, optional

If a particle stalls, then jump to a random location that exists +/- searchSize% of the current location, but increase searchSize% for that particle by stallSearchScale after each stall:

$searchSize = $searchSize * ($stallSearchScale ** $numStalls)

Default: 1 (does not change searchSize).

A recommended value is that which will slightly increase searchSize over time. For example, if stallSearchScale = 1.1 and searchSize = 0.5 then searchSize will change for that particle as follows:

Stalls                  searchSize
------                  ----------
  0                        0.5        # initial configured value
  1                        0.55
  2                        0.605
  3                        0.6655
  5                        0.8053
  7                        0.9744

In this example, searchSize will be capped at a value of 1.0 after the particle stalls 8 times.

See also: -searchSize

-themWeight: number, optional

Coefficient determining the influence of the neighbourhod best position on the next iterations velocity. This can be a PDL object, so it should work in any dimension so long as it works in the broadcast sense.

Defaults to 0.5.

See also -inertia and -meWeight.

-exitPlateau: boolean, optional

Set true to have the optimization check for plateaus (regions where the fit hasn't improved much for a while) during the search. The optimization ends when a suitable plateau is detected following the burn in period.

Defaults to undefined (option disabled).

-exitPlateauDP: number, optional

Specify the number of decimal places to compare between the current fitness function value and the mean of the previous -exitPlateauWindow values.

Defaults to 10.

-exitPlateauWindow: number, optional

Specify the size of the window used to calculate the mean for comparison to the current output of the fitness function. Correlates to the minimum size of a plateau needed to end the optimization.

Defaults to 10% of the number of iterations (-iterations).

-exitPlateauBurnin: number, optional

Determines how many iterations to run before checking for plateaus.

Defaults to 50% of the number of iterations (-iterations).

-verbose: flags, optional

If set to a non-zero value -verbose determines the level of diagnostic print reporting that is generated during optimization.

The following constants may be bitwise ored together to set logging options:

  • kLogBetter

    prints particle details when its fit becomes bebtter than its previous best.

  • kLogStall

    prints particle details when its velocity reaches 0 or falls below the stall threshold.

  • kLogIter

    Shows the current iteration number.

setParams(%parameters)

Set or change optimization parameters. See -new above for a description of the parameters that may be supplied.

init()

Reinitialize the optimization. init() will be called during the first call to optimize() if it hasn't already been called.

optimize()

Runs the minimization optimization. Returns the fit value of the best fit found. The best possible fit is negative infinity.

optimize() may be called repeatedly to continue the fitting process. The fit processing on each subsequent call will continue from where the last call left off.

Use getBestPos() and getBestFit() after optimization to get the optimized result.

getBestPos()

Return the best position vector that has been found so far, as determined by fitFunc. Use this after calling optimize().

getBestFit()

Return the fit value that has been found so far, as returned by fitFunc Use this after calling optimize().

getIterationCount()

Return the number of iterations performed. This may be useful when the -exitFit criteria has been met or where multiple calls to optimize have been made.

getStallCount()

Returns a PDL of stall counts per particle.

Hint: to get a total stall count, call `$pso->getStallCount()->sum()` .

SEE ALSO

PDL::Opt::ParticleSwarm::Simple - Use names for Particle Swarm-optimized values
PDL::Opt::Simplex - A PDL implementation of the Simplex optimization algorithm
PDL::Opt::Simplex::Simple - Use names for Simplex-optimized values
http://en.wikipedia.org/wiki/Particle_swarm_optimization
AI::ParticleSwarmOptimization
AI::PSO

ACKNOWLEDGEMENTS

This PDL implementation is based on AI::ParticleSwarmOptimization by Peter Jaquiery (GRANDPA), which in turn was based on the AI::PSO originally created by Kyle Schlansker (KYLESCH).

AUTHORS

All rights reserved.

LICENSE

This module is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This module is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this module. If not, see <http://www.gnu.org/licenses/>.