AI::MXNet::Metric - Evaluation Metric API. =head1 DESCRIPTION This module hosts all the evaluation metrics available to evaluate the performance of a learned model. L<Python Docs|https://mxnet.apache.org/api/python/metric/metric.html>
Save configurations of metric. Can be recreated from configs with mx->metric->create(%{ $config })
AI::MXNet::Accuracy - Computes accuracy classification score.
The accuracy score is defined as accuracy(y, y^) = (1/n) * sum(i=0..n-1) { y^(i)==y(i) } Parameters: axis (Int, default=1) – The axis that represents classes. name (Str, default='accuracy') – Name of this metric instance for display. pdl> use AI::MXNet qw(mx) pdl> $predicts = [mx->nd->array([[0.3, 0.7], [0, 1.], [0.4, 0.6]])] pdl> $labels = [mx->nd->array([[0, 1, 1]])] pdl> $acc = mx->metric->Accuracy() pdl> $acc->update($labels, $predicts) pdl> use Data::Dumper pdl> print Dumper([$acc->get]) $VAR1 = [ 'accuracy', '0.666666666666667' ];
AI::MXNet::TopKAccuracy - Computes top k predictions accuracy.
TopKAccuracy differs from Accuracy in that it considers the prediction to be True as long as the ground truth label is in the top K predicated labels. If top_k = 1, then TopKAccuracy is identical to Accuracy. Parameters: top_k(Int, default 1) – Whether targets are in top k predictions. name (Str, default 'top_k_accuracy') – Name of this metric instance for display. use AI::MXNet qw(mx); $top_k = 3; $predicts = [mx->nd->array( [[0.80342804, 0.5275223 , 0.11911147, 0.63968144, 0.09092526, 0.33222568, 0.42738095, 0.55438581, 0.62812652, 0.69739294], [0.78994969, 0.13189035, 0.34277045, 0.20155961, 0.70732423, 0.03339926, 0.90925004, 0.40516066, 0.76043547, 0.47375838], [0.28671892, 0.75129249, 0.09708994, 0.41235779, 0.28163896, 0.39027778, 0.87110921, 0.08124512, 0.55793117, 0.54753428], [0.33220307, 0.97326881, 0.2862761 , 0.5082575 , 0.14795074, 0.19643398, 0.84082001, 0.0037532 , 0.78262101, 0.83347772], [0.93790734, 0.97260166, 0.83282304, 0.06581761, 0.40379256, 0.37479349, 0.50750135, 0.97787696, 0.81899021, 0.18754124], [0.69804812, 0.68261077, 0.99909815, 0.48263116, 0.73059268, 0.79518236, 0.26139168, 0.16107376, 0.69850315, 0.89950917], [0.91515562, 0.31244902, 0.95412616, 0.7242641 , 0.02091039, 0.72554552, 0.58165923, 0.9545687 , 0.74233195, 0.19750339], [0.94900651, 0.85836332, 0.44904621, 0.82365038, 0.99726878, 0.56413064, 0.5890016 , 0.42402702, 0.89548786, 0.44437266], [0.57723744, 0.66019353, 0.30244304, 0.02295771, 0.83766937, 0.31953292, 0.37552193, 0.18172362, 0.83135182, 0.18487429], [0.96968683, 0.69644561, 0.60566253, 0.49600661, 0.70888438, 0.26044186, 0.65267488, 0.62297362, 0.83609334, 0.3572364 ]] )]; $labels = [mx->nd->array([2, 6, 9, 2, 3, 4, 7, 8, 9, 6])]; $acc = mx->metric->TopKAccuracy(top_k=>$top_k); $acc->update($labels, $predicts); use Data::Dumper; print Dumper([$acc->get]); $VAR1 = [ 'top_k_accuracy_3', '0.3' ];
AI::MXNet::F1 - Calculate the F1 score of a binary classification problem.
The F1 score is equivalent to harmonic mean of the precision and recall, where the best value is 1.0 and the worst value is 0.0. The formula for F1 score is: F1 = 2 * (precision * recall) / (precision + recall) The formula for precision and recall is: precision = true_positives / (true_positives + false_positives) recall = true_positives / (true_positives + false_negatives) Note: This F1 score only supports binary classification. Parameters: name (Str, default 'f1') – Name of this metric instance for display. average (Str, default 'macro') – Strategy to be used for aggregating across mini-batches. “macro”: average the F1 scores for each batch. “micro”: compute a single F1 score across all batches. $predicts = [mx.nd.array([[0.3, 0.7], [0., 1.], [0.4, 0.6]])]; $labels = [mx.nd.array([0., 1., 1.])]; $f1 = mx->metric->F1(); $f1->update($labels, $predicts); print $f1->get; f1 0.8
AI::MXNet::MCC - Computes the Matthews Correlation Coefficient of a binary classification problem.
While slower to compute than F1 the MCC can give insight that F1 or Accuracy cannot. For instance, if the network always predicts the same result then the MCC will immeadiately show this. The MCC is also symetric with respect to positive and negative categorization, however, there needs to be both positive and negative examples in the labels or it will always return 0. MCC of 0 is uncorrelated, 1 is completely correlated, and -1 is negatively correlated. MCC = (TP * TN - FP * FN)/sqrt( (TP + FP)*( TP + FN )*( TN + FP )*( TN + FN ) ) where 0 terms in the denominator are replaced by 1. This version of MCC only supports binary classification. Parameters ---------- name : str, 'mcc' Name of this metric instance for display. average : str, default 'macro' Strategy to be used for aggregating across mini-batches. "macro": average the MCC for each batch. "micro": compute a single MCC across all batches. Examples -------- In this example the network almost always predicts positive >>> $false_positives = 1000 >>> $false_negatives = 1 >>> $true_positives = 10000 >>> $true_negatives = 1 >>> $predicts = [mx->nd->array( [ ([.3, .7])x$false_positives, ([.7, .3])x$true_negatives, ([.7, .3])x$false_negatives, ([.3, .7])xtrue_positives ] )]; >>> $labels = [mx->nd->array( [ (0)x($false_positives + $true_negatives), (1)x($false_negatives + $true_positives) ] )]; >>> $f1 = mx->metric->F1(); >>> $f1->update($labels, $predicts); >>> $mcc = mx->metric->MCC() >>> $mcc->update($labels, $predicts) >>> print $f1->get(); f1 0.95233560306652054 >>> print $mcc->get(); mcc 0.01917751877733392
AI::MXNet::Perplexity - Calculate perplexity.
Perplexity is a measurement of how well a probability distribution or model predicts a sample. A low perplexity indicates the model is good at predicting the sample. Parameters ---------- ignore_label : int or undef index of invalid label to ignore when counting. usually should be -1. Include all entries if undef. axis : int (default -1) The axis from prediction that was used to compute softmax. By default uses the last axis. $predicts = [mx->nd->array([[0.3, 0.7], [0, 1.], [0.4, 0.6]])]; $labels = [mx->nd->array([0, 1, 1])]; $perp = mx->metric->Perplexity(ignore_label=>undef); $perp->update($labels, $predicts); print $perp->get() Perplexity 1.77109762851559
AI::MXNet::MAE - Calculate Mean Absolute Error loss =head1 DESCRIPTION >>> $predicts = [mx->nd->array([3, -0.5, 2, 7])->reshape([4,1])] >>> $labels = [mx->nd->array([2.5, 0.0, 2, 8])->reshape([4,1])] >>> $mean_absolute_error = mx->metric->MAE() >>> $mean_absolute_error->update($labels, $predicts) >>> print $mean_absolute_error->get() ('mae', 0.5)
AI::MXNet::MSE - Calculate Mean Squared Error loss =head1 DESCRIPTION >>> $predicts = [mx->nd->array([3, -0.5, 2, 7])->reshape([4,1])] >>> $labels = [mx->nd->array([2.5, 0.0, 2, 8])->reshape([4,1])] >>> $mean_squared_error = mx->metric->MSE() >>> $mean_squared_error->update($labels, $predicts) >>> print $mean_squared_error->get() ('mse', 0.375)
AI::MXNet::RMSE - Calculate Root Mean Squred Error loss =head1 DESCRIPTION >>> $predicts = [mx->nd->array([3, -0.5, 2, 7])->reshape([4,1])] >>> $labels = [mx->nd->array([2.5, 0.0, 2, 8])->reshape([4,1])] >>> $root_mean_squared_error = mx->metric->RMSE() >>> $root_mean_squared_error->update($labels, $predicts) >>> print $root_mean_squared_error->get() 'rmse', 0.612372457981
AI::MXNet::CrossEntropy - Calculate Cross Entropy loss =head1 DESCRIPTION >>> $predicts = [mx->nd->array([[0.3, 0.7], [0, 1.], [0.4, 0.6]])] >>> $labels = [mx->nd->array([0, 1, 1])] >>> $ce = mx->metric->CrossEntropy() >>> $ce->update($labels, $predicts) >>> print $ce->get() ('cross-entropy', 0.57159948348999023)
AI::MXNet::NegativeLogLikelihood - Computes the negative log-likelihood loss. =head1 DESCRIPTION >>> $predicts = [mx->nd->array([[0.3, 0.7], [0, 1.], [0.4, 0.6]])] >>> $labels = [mx->nd->array([0, 1, 1])] >>> $nll_loss = mx->metric->NegativeLogLikelihood >>> $nll_loss->update($labels, $predicts) >>> print $nll_loss->get() ('cross-entropy', 0.57159948348999023)
AI::MXNet::PearsonCorrelation - Computes Pearson correlation.
Computes Pearson correlation. Parameters ---------- name : str Name of this metric instance for display. Examples -------- >>> $predicts = [mx->nd->array([[0.3, 0.7], [0, 1.], [0.4, 0.6]])] >>> $labels = [mx->nd->array([[1, 0], [0, 1], [0, 1]])] >>> $pr = mx->metric->PearsonCorrelation() >>> $pr->update($labels, $predicts) >>> print pr->get() ('pearson-correlation', '0.421637061887229')
AI::MXNet::Loss - Dummy metric for directly printing loss.
Dummy metric for directly printing loss. Parameters ---------- name : str Name of this metric instance for display.
AI::MXNet::Confidence - Accuracy by confidence buckets.
Accuracy by confidence buckets. Parameters ---------- name : str Name of this metric instance for display. num_classes: Int number of classes confidence_thresholds: ArrayRef[Num] confidence buckets For example my $composite_metric = AI::MXNet::CompositeEvalMetric->new; $composite_metric->add(mx->metric->create('acc')); $composite_metric->add( AI::MXNet::Confidence->new( num_classes => 2, confidence_thresholds => [ 0.5, 0.7, 0.8, 0.9 ], ) );
AI::MXNet::CustomMetric - Custom evaluation metric that takes a sub ref.
Custom evaluation metric that takes a sub ref. Parameters ---------- eval_function : subref Customized evaluation function. name : str, optional The name of the metric allow_extra_outputs : bool If true, the prediction outputs can have extra outputs. This is useful in RNN, where the states are also produced in outputs for forwarding.
Create an evaluation metric. Parameters ---------- metric : str or sub ref The name of the metric, or a function providing statistics given pred, label NDArray.
To install AI::MXNet, copy and paste the appropriate command in to your terminal.
cpanm
cpanm AI::MXNet
CPAN shell
perl -MCPAN -e shell install AI::MXNet
For more information on module installation, please visit the detailed CPAN module installation guide.