The London Perl and Raku Workshop takes place on 26th Oct 2024. If your company depends on Perl, please consider sponsoring and/or attending.
 / 
MachineLearning-1.03
/ MachineLearning::NeuralNetwork

NAME

MachineLearning::NeuralNetwork - create, train, test, resize, store, and run a neural network

SYNOPSIS

  use MachineLearning::NeuralNetwork;
  my $nn = MachineLearning::NeuralNetwork->new({
    "Name" => "Sample Network",
    "Description" => "Sample network",
    "HiddenLayerSize" => 5,
    "InputFieldNames" => ["Input1", "Input2"],
    "OutputFieldNames" => ["Output"]});
  my $nn = MachineLearning::NeuralNetwork->open(
    "~/neural_networks/sample.nn");
  my $success = $nn->get_success();
  my $message = $nn->get_message();
  my $network_name = $nn->get_name();
  my $network_description = $nn->get_description();
  my $ran_ok = $nn->run($data_file_path);
  my $trained_ok = $nn->train({
    "TrainingDataFilePath" => $training_data_file_path,
    "CyclesPerTemperature" => $cycles_per_temperature,
    "MinimumOnesPercentage" => $minimum_ones_percentage});
  my $test_results = $nn->test($validation_data_file_path);
  my $new_hidden_layer_size = $nn->grow({
    "TrainingDataFilePath" => $training_data_file_path,
    "ValidationDataFilePath" => $validation_data_file_path,
    "CyclesPerTemperature" => $cycles_per_temperature,
    "MinimumOnesPercentage" => $minimum_ones_percentage});
  my $new_hidden_layer_size = $nn->prune({
    "TrainingDataFilePath" => $training_data_file_path,
    "ValidationDataFilePath" => $validation_data_file_path,
    "CyclesPerTemperature" => $cycles_per_temperature,
    "MinimumOnesPercentage" => $minimum_ones_percentage});
  my $saved_ok = $nn->save("~/neural_networks/perfected_network.nn");

DESCRIPTION

This module defines a package for creating a MachineLearning::NeuralNetwork object.

This module uses the MachineLearning::SimulatedAnnealing module to optimize the network's weights and thresholds during training.

A neural network as implemented by the MachineLearning::NeuralNetwork module favors quality over quantity. That is, it is optimized to find the highest quality predictions for a single result value without particular regard to how many data records (or instances) get screened out in the process. This is highly useful for many applications. For example, out of many potential financial investments, you might want to identify a small group that have unusually good prospects for success.

The result values supported by the MachineLearning::NeuralNetwork module are 1 and 0, and the accuracy of the results is important only for the 1 values. To ensure that the neural network's output layer generates a satisfactory minimum quantity of ones, the methods for training, growing, and pruning the neural network take an optional MinimumOnesPercentage argument.

PREREQUISITES

To use this module, you must have both the MachineLearning module and the MachineLearning::SimulatedAnnealing module installed.

METHODS

MachineLearning::NeuralNetwork->new($args);

This is the constructor.

In addition to the class-name argument, which is passed automatically when you use the MachineLearning::NeuralNetwork->new() syntax, the constructor takes a reference to a hash containing the following keys:

  Name
  Description
  HiddenLayerSize
  InputFieldNames
  OutputFieldNames

The Name and Description must be non-empty strings. The HiddenLayerSize must be a positive integer specifying the number of neurons in the neural network's hidden layer. The value associated with the InputFieldNames key must be a reference to an array of input field names. The value associated with the OutputFieldNames key must be a reference to an array containing exactly one output field name.

All field names (for input and output fields combined) must be unique. Field names must not contain commas or line-break characters. There must be at least two input field names and exactly one output field name.

The constructor returns a reference to a MachineLearning::NeuralNetwork object, which is implemented internally as a hash. All functionality is exposed through methods.

If the constructor receives a valid hash reference providing all required information, the get_success() instance method returns true (1) and the get_message() instance method returns an empty string; otherwise, get_success() returns false (0) and get_message() returns a string containing an error message.

$nn->get_success();

This returns true (1) if the neural network object was initialized successfully; otherwise, it returns false (0).

$nn->get_message();

When get_success() returns true (1), this returns an empty string. When get_success() returns false (0), get_message() returns an error message.

$nn->get_name();

Returns the name of the neural network, or an empty string if the neural network was never successfully initialized.

$nn->get_description();

Returns the description of the neural network, or an empty string if the neural network was never successfully initialized.

$nn->run($data_file_path);

This method runs the neural network on the specified data.

The specified data file must be in CSV format with a header row. The header row must contain the names of the input neurons in the correct order followed by the name of the output neuron. NOTE: There can be more than one output field; however, the neural network uses and preserves only the one for which the column heading matches the output-field name associated with the neural network.

Each record in the data must contain the correct number of input values as well as a blank or replaceable output value in the appropriate output field. IMPORTANT: The method removes all other output fields from the file.

The method adds the neural network generated output value to each record, overwriting the output value that is already there, if any. NOTE: Input and output values must not contain commas or line-break characters.

If everything goes OK, the method returns true (1); otherwise, the method returns false (0). If the neural network was in a valid state previously but something went wrong during execution of the run() method, the method sets the _success field (returned by the get_success() method) to false (0) and places an error message in the _message field (returned by the get_message() method).

$nn->train($args);

$args is a reference to a hash containing the following keys:

  TrainingDataFilePath
  CyclesPerTemperature
  MinimumOnesPercentage

This method trains the neural network using the specified data and the specified number of cycles per temperature. The value specifying the minimum percentage of ones required for the output node to avoid an automatic cost assessment of 100 is optional and, if missing or invalid, is assumed to be 0. Even if the minimum ones percentage is 0, however, there must always be at least one occurrence of a 1 result for the output neuron to avoid an automatic cost assessment of 100.

The training data must be in CSV format with a header row. The header row must contain the names of the input neurons in the correct order followed by the name of the output neuron. NOTE: There can be more than one output field; however, the neural network uses only the one for which the column heading matches the output field name associated with the neural network.

Each record in the data must contain the correct number of input values as well as an output value in the appropriate output field. The output value supplied with the training data is typically the expected, ideal, or real-life result for the supplied input values. NOTE: Input and output values must not contain commas or line-break characters.

CyclesPerTemperature is a positive integer (for example, 1_000) specifying the number of randomization cycles performed at each temperature level during the simulated annealing process. (For more information, see the MachineLearning::SimulatedAnnealing module).

During training, the network minimizes cost using simulated annealing to optimize the weights and thresholds. Cost is a number that represents how much error results when a particular set of weights and thresholds is applied to the training data. The cost is the percentage of the time that the 1 values in the output generated by the network do not match the corresponding values provided by the training data.

If specified, the MinimumOnesPercentage value must be a positive number less than 100 that represents a percentage. The cost calculated for the output neuron's values over a data set will be set to 100 automatically if the percentage of ones is less than the specified minimum (or if there are no ones at all).

If everything goes OK, the method returns true (1); otherwise, the method returns false (0). If the neural network was in a valid state previously but something went wrong during execution of the train() method, the method sets the _success field (returned by the get_success() method) to false (0) and places an error message in the _message field (returned by the get_message() method).

$nn->test($validation_data_file_path);

This method tests the network using the supplied validation data, which must be in the same format as the training data.

The method returns a string containing the test results in the form of a formatted report, which gives the cost for the output neuron.

Cost is a number that represents how much error results when the neural network is applied to the test data. The cost is the percentage of the time that the 1 values in the output generated by the network do not match the corresponding values provided by the test data.

During training, the network minimizes cost using simulated annealing to optimize the weights and thresholds. During testing, however, there are no adjustments to the weights and thresholds; the results are simply calculated and reported.

TIP: Testing reveals how well the network generalizes to out-of-sample data. Therefore, make sure that the validation data does not overlap with the training data. To compare the test results with the results of applying the network to the data on which it was trained, you can run a test using the training data. The cost is typically higher for the test data, so the important question is whether that cost is sufficiently low for the network to be useful.

If something goes wrong during execution of the method, the method returns an empty string and, if the _success field (returned by the get_success() method) is currently set to true (1), sets that field to false (0) and places an error message in the _message field (returned by the get_message() method).

$nn->grow($args);

$args is a reference to a hash containing the following keys:

  TrainingDataFilePath
  ValidationDataFilePath
  CyclesPerTemperature
  MinimumOnesPercentage

The MinimumOnesPercentage key is optional.

This method grows the neural network by performing training and testing with a progressively increasing number of hidden neurons. The size of the hidden layer starts at five and then progresses upward through the Fibonacci series to 144 (that is, the sizes used are 5, 8, 13, 21, 34, 55, 89, and 144). Once the neural network has been trained and tested with a hidden layer size of 144, the method chooses the size with the best result (the lowest cost) based on post-training validation, retrains the network with that number of hidden neurons, and then returns that number. NOTE: In the case of a tie, the method favors the smaller number of hidden neurons.

If something goes wrong during execution of the method, the method returns 0 and, if the _success field (returned by the get_success() method) is currently set to true (1), sets that field to false (0) and places an error message in the _message field (returned by the get_message() method).

$nn->prune($args);

$args is a reference to a hash containing the following keys:

  TrainingDataFilePath
  ValidationDataFilePath
  CyclesPerTemperature
  MinimumOnesPercentage

The MinimumOnesPercentage key is optional.

This method prunes the neural network by performing training followed by testing with a progressively decreasing number of hidden neurons. The size of the hidden layer decreases by one for each cycle of training and testing. Once all sizes have been tried from the initial size down to the closest lower number that is in the Fibonacci series, the method chooses the size with the best result (the lowest cost), retrains the network with that number of hidden neurons, and returns that number. NOTE: In the case of a tie, the method favors the smaller number of hidden neurons.

If something goes wrong during execution of the method, the method returns 0 and, if the _success field (returned by the get_success() method) is currently set to true (1), sets that field to false (0) and places an error message in the _message field (returned by the get_message() method).

$nn->save($file_path);

This method saves (serializes) the neural network object to a file. A neural network must be already trained before you can save it.

If the serialization and file-writing operations succeed, this method returns true (1); otherwise, the method sets the _success field to false (0), places an error message in the _message field, and returns false (0).

MachineLearning::NeuralNetwork->open($file_path);

This method returns a new MachineLearning::NeuralNetwork object created by restoring such an object from the specified file. If the file-reading and deserialization operations succeed, the resulting object's get_success() method returns true (1) and the get_message() method returns an empty string. Otherwise, the open() method creates and returns a new object that has a false value in the _success field and an error message in the _message field.

NEURAL NETWORK DATA

All input values must be decimal numbers in the range -1 to 1, inclusive.

Internally, the neural network uses weight values in the range -1 to 1 (inclusive) and thresholds in the range 0.00000001 to n, where n is the number of neurons in the preceding layer. Both the hidden and output layers have thresholds, and the output value is determined by whether the threshold for the output node is reached (1 if yes, 0 if no).

All output values provided by training data or validation data must be either 0 or 1.

NEURAL NETWORK ARCHITECTURE

This module uses a feed-forward neural network architecture with one hidden layer. The number of hidden nodes is variable, and the recommended approach is to try various numbers in ascending order (for example, by using the grow() method). Then, starting with the number that produced the best results based on post-training validation, prune the neural network using the prune() method.

TIP: You can grow and then prune a neural network several times using different data sets in order to gain more insight into the optimal size for the hidden layer. You can also switch the training and validation sets to get twice as many train-and-test cycles from your data. When using these approaches, consider reserving sufficient data for a final test; data that is not part of any of the data sets that you are using for training and validation during the development phase of the neural network. If the final test is not satisfactory, you might have to reconsider the types of inputs that you are using for the neural network, gather sufficient additional data for a new final test, and then develop the neural network again using a different input framework.

AUTHOR

Benjamin Fitch, <blernflerkl@yahoo.com>

COPYRIGHT AND LICENSE

Copyright 2009 by Benjamin Fitch

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.