NAME

Test::MemoryGrowth - assert that code does not cause growth in memory usage

SYNOPSIS

use Test::More;
use Test::MemoryGrowth;

use Some::Class;

no_growth {
   my $obj = Some::Class->new;
} 'Constructing Some::Class does not grow memory';

my $obj = Some::Class->new;
no_growth {
   $obj->do_thing;
} 'Some::Class->do_thing does not grow memory';


#### This test will fail ####
my @list;
no_growth {
   push @list, "Hello world";
} 'pushing to an array does not grow memory';

done_testing;

DESCRIPTION

This module provides a function to check that a given block of code does not result in the process consuming extra memory once it has finished. Despite the name of this module it does not, in the strictest sense of the word, test for a memory leak: that term is specifically applied to cases where memory has been allocated but all record of it has been lost, so it cannot possibly be reclaimed. While the method employed by this module can detect such bugs, it can also detect cases where memory is still referenced and reachable, but the usage has grown more than would be expected or necessary.

The block of code will be run a large number of times (by default 10,000), and the difference in memory usage by the process before and after is compared. If the memory usage has now increased by more than one byte per call, then the test fails.

In order to give the code a chance to load initial resources it needs, it will be run a few times first (by default 10); giving it a chance to load files, AUTOLOADs, caches, or any other information that it requires. Any extra memory usage here will not count against it.

This simple method is not a guaranteed indicator of the absence of memory resource bugs from a piece of code; it has the possibility to fail in both a false-negative and a false-positive way.

False Negative

It is possible that a piece of code causes memory usage growth that this module does not detect. Because it only detects memory growth of at least one byte per call, it cannot detect cases of linear memory growth at lower rates than this. Most memory usage growth comes either from Perl-level or C-level bugs where memory objects are created at every call and not reclaimed again. (These are either genuine memory leaks, or needless allocations of objects that are stored somewhere and never reclaimed). It is unlikely such a bug would result in a growth rate smaller than one byte per call.

A second failure case comes from the fact that memory usage is taken from the Operating System's measure of the process's Virtual Memory size, so as to be able to detect memory usage growth in C libraries or XS-level wrapping code, as well as Perl functions. Because Perl does not aggressively return unused memory to the Operating System, it is possible that a piece of code could use un-allocated but un-reclaimed memory to grow into; resulting in an increase in its requirements despite not requesting extra memory from the Operating System.

False Positive

It is possible that the test will claim that a function grows in memory, when the behaviour is in fact perfectly normal for the code in question. For example, the code could simply be some function whose behaviour is required to store extra state; for example, adding a new item into a list. In this case it is in fact expected that the memory usage of the process will increase.

By careful use of this test module, false indications can be minimised. By splitting tests across many test scripts, each one can be started in a new process state, where most of the memory assigned from the Operating System is in use by Perl, so anything extra that the code requires will have to request more. This should reduce the false negative indications.

By keeping in mind that the module simply measures the change in allocated memory size, false positives can be minimised, by not attempting to assert that certain pieces of code do not grow in memory, when in fact it would be expected that they do.

Devel::Gladiator Integration

Since version 0.04.

If Devel::Gladiator is installed, this test module will use it as a second potential source of detecting memory growth. A walk of the Perl memory heap is taken before running the code, in order to count the number of every kind of object present. This is then compared to a second count taken afterwards. Any object types that have increased by at least one per call are reported.

For example, the output might contain the following extra lines of diagnostic output:

# Growths in arena object counts:
#   ARRAY 1735 -> 11735 (1.00 per call)
#   HASH 459 -> 10459 (1.00 per call)
#   REF 1387 -> 21387 (2.00 per call)
#   REF-ARRAY 163 -> 10163 (1.00 per call)
#   REF-HASH 66 -> 10066 (1.00 per call)
#   WithContainerSlots 10 -> 10010 (1.00 per call)

Devel::MAT Integration

If Devel::MAT is installed, this test module will use it to dump the state of the memory after a failure. It will create a .pmat file named the same as the unit test, but with the trailing .t suffix replaced with -TEST.pmat where TEST is the number of the test that failed (in case there was more than one). It will then run the code under test one more time, before writing another file whose name is suffixed with -TEST-after.pmat. This pair of files may be useful for differential analysis.

FUNCTIONS

no_growth

no_growth { CODE } %opts, $name;

Assert that the code block does not consume extra memory.

Takes the following named arguments:

calls => INT

The number of times to call the code during growth testing.

burn_in => INT

The number of times to call the code initially, before watching for memory usage.

TODO

• More OS portability

  Currently, this module uses OS-specific methods of determining process memory usage (namely, by inspecting /proc/self/status on Linux or calling procstat on FreeBSD). It would be nice to support more OSes, and potentially find a better abstraction for doing so.

  Currently I am unaware of a simple portable mechanism to query this. Patches very much welcome. :)

AUTHOR

Paul Evans <leonerd@leonerd.org.uk>