Test::Usage - Manage test usage example suites throughout module development.
Let's say we are building module Foo.pm. To exercise it, we write a test usage examples module, Foo_T.pm, which may eventually look something like this:
package Foo_T; use strict; use Test::Usage; use Foo; example('e1', sub { ... ok(...); ... ok(...); ... }); example('a1', sub { ... ok(...) or diag(...); ... }); example('a2', sub { ... ok(...); ... }); example('a3', sub { my $f = Foo->new(); my $got_foo = $f->foo(); my $exp_foo = 'FOO'; ok( $got_foo eq $exp_foo, "Expecting foo() to return '$exp_foo'.", "But got '$got_foo'." ); });
Here are a few ways to test its examples:
perl -MFoo_T -e 'test(a => "a3")' # Run example a3 only. perl -MFoo_T -e 'test(a => "a*")' # Run all a: a1, a2, a3. perl -MFoo_T -e 'test()' # Run all: e1, a1, a2, a3. perl -MFoo_T -e 'test(a => "a3", v => 2, c => 0)' # See test().
To summarize tests ran on all "*_T.pm" files found under current directory:
perl -MTest::Usage -e files # See files().
This module helps Perl developers maintain and run test usage examples.
Test files are usually paired with an ordinary Perl module, Foo.pm for example. The test file is also a Perl module, conventionally named Foo_T.pm, and also conventionally resides in the same directory as Foo.pm (it's really part of the code, so I like it to stay close to the code). If the test module is not associated to any module in particular, that's okay too. The test module has a conventional structure, like the one shown in the SYNOPSIS. Basically, it just names the module, loads Test::Usage and defines a bunch of examples.
When Test::Usage is loaded, it determines which color mechanism to apply (currently, ANSI or Win32Console) and it runs its import() method, which sets up a you-might-call-it-a-singleton instance of Test::Usage, setting its test runs counters to 0. It also exports some of its methods to the calling package and some to main, to make them easier to use.
Each example(), identified by its label, adds to the tests that the module can run, upon request.
When the developer wishes to run a test, he invokes it from the command line (perhaps with a coating of shell syntaxic sugar), as shown in the synopsis.
All methods apply to a single instance of Test::Usage, named $t, initialized by import().
The module defines the following methods and functions.
Sets $t to an empty hash ref, blessed in Test::Usage.
Resets $t's counters to 0. See the corresponding methods:
time_took() nb_succ() nb_fail() died() warned()
Sets $t's default label to '-'.
Resets $t's options to default values. Here are the as-shipped values:
For the test() method: a => '*' # Accept tests whose label matches this glob. e => '__*' # Exclude tests whose label matches this glob. s => 1 # Print a summary line if true. v => REPORT_FAILURES # Verbosity level. fail => 0 # Fail tests systematically if true. For the files() method: d => '.' # Directory in which to look for files. g => '*_T.pm' # Test files whose name matches this glob. r => 1 # Look for files recursively through dir if true. i => '' # Add to Perl %INC path. t => {} # Option values to pass to test() for each file. For both test() and files(): c => 1 # Use color if possible.
Exports these methods to the calling package:
t example ok ok_labeled diag
Exports these methods to main:
t test files labels
Both return $t, effectively giving access to all Test::Usage methods.
Add a test example labeled $label and implemented by $sub_ref to the tests that can be run by $t->test().
$label is an arbitrary string that is used to identify the example. The label will be displayed when reporting tests results. Labels can be chosen to make it easy to run selected subsets; for example, you may want to label a bunch of examples that you usually run together with a common prefix.
The $sub_ref is a reference to the subroutine implementing the test. It often calls a number of ok(), wrapped in setup/tear-down scaffolding to express the intended usage.
Here's a full example:
example('t1', sub { my $f = Foo->new(); my $exp = 1; my $got = $f->get_val(); ok( $got == $exp, "Expected get_val() to be $exp for a new Foo object.", "But got $got.", ); });
$bool is an expression that will be evaluated as true or false, and thus determine the return value of the method. Also, if $bool is true, $t will increment the number of successful tests it has seen, else the number of failed tests.
Note that $bool will be evaluated in list context; for example, if you want to use a bind operator here, make sure you wrap it with 'scalar'. For example:
ok(scalar($x =~ /abc/), "Expected \$x to match /abc/.", "But its value was '$x'." );
In that example, if 'scalar' is not used, the bind operator is evaluated in list context, and if there is no match, an empty list is returned, which results in ok() receiving only the last two arguments.
If the test() flags are such that the result of the ok() is to be printed, something like one of the following will be printed:
ok a1 # Expected $x to match /abc/. not ok a1 # Expected $x to match /abc/. # But its value was 'def'.
ok() is most useful when it describes expected result and helps debug when it fails. So the $exp_msg should tell the user what the test is expecting, and the $got_msg, what it got instead. It is useful to formulate them in terms that are useful for development, maintenance, and debugging. Compare the following examples:
ok(! defined($got = foo()), 'foo() should return undef if no arguments are given.', "But returned '$got' instead." );
Whether it succeeds of fails, the following messages are helpful:
ok a1 # foo() should return undef if no arguments are given. not ok a1 # foo() should return undef if no arguments are given. # But returned '' instead.
ok(! defined(my $got = foo()), 'Result is undefined.', 'Didn\'t work.' );
Whether it succeeds of fails, we don't really know what exactly went right, or wrong:
ok a1 # Result is undefined. not ok a1 # Result is undefined. # Didn't work.
Same as ok(), except that ".$sub_label" is appended to the label in the printed output. This is useful for examples containing many ok() whose labels we want to distinguish.
Prefixes each line of each string in @msgs with ' # ' and displays them using the 'diag' color tag. Returns true (contrary to Test::Builder, Test::More, et al.).
Returns a ref to an array holding the labels of all the examples, in the order they were defined.
Run all the known examples, subject to the constraints indicated by %options. If %options is undefined or if some of its keys are missing, default values apply. Here is the meaning and default value of the elements of %options:
The value is a glob. Tests whose label matches this glob will be run. All tests are run when the value is the default.
The value is a glob. Tests whose label matches this glob will not be run. I use this when I want to keep a test in the test module, but I don't want to run it for some reason. When using the default value, prepending the string '__' to a test label will effectively disactivate it. When you are ready to run those tests, remove the '__' prefix from the label, or pass the 'e => ""' argument.
Determines the verbosity of the testing mechanism:
0: Display no individual results. 1: Display individual results for failing tests only. 2: Display individual results for all tests.
If true, two lines like the following will wrap the test output:
# module_name ... # +3 -1 -d +w (00h:00m:02s) module_name
That means that of the ok*() calls that were made, 3 succeeded and 1 failed, that no dies but some warnings occurred, and it took about 2 seconds to run.
If true, any ok*() invoked will act as though it failed. When combined with a verbosity of 1 or 2, (to display failures), you will see all the actual messages that would get printed when failures occur.
After having found all the files that correspond to the criteria defined in %options (for example, directory to look in), for each file calls perl in a subshell to run something like this:
perl -M$file -e 'test()'
The results of each run are collected, examined and tallied, and a summary line is displayed.
All values in %options are optional. Their meaning and default value are as follows:
All options starting with the letter 'd' designate directories in which to look for files matching the glob specified by option 'g'. These directories should be in perl's current module search path, else add to the path using the 'i' option.
Only files matching this glob will be tested.
If set to true, files matching the 'g' glob will be searched for recursively in all subdirs starting from (and including) those specified by the 'd' options. FIXME: Currently, it's always true.
All options starting with the letter 'i' designate directories that you want to add to the @INC path for finding modules. They will be added in the order of the sorted 'i*' keys.
These options will be passed to the test() method, invoked for each tested file.
Resets all options to their default values. To display them, run , as defined in %_D.
Returns a ref to the hash representing current option settings. Its structure is:
Returns the number of seconds the last test() run took. This value is rough and is of limited use for benchmarking.
Returns the number of tests that failed during the last test() run.
Returns the number of tests that succeeded during the last test() run.
Returns the number of test()s that die`d.
Returns the number of test()s that had warnings.
Back in the late 90s, readings on Extreme Programming prompted me to try out what is now called Test Driven Development (TDD). The idea is that before you write code, you write a test that demonstrates what the code should do. You can find on the web a lot of literature on TDD, and all I'll say here is that it has radically changed my development experience.
So at the time, I started building a Perl tool that would help our team at work to benefit from TDD. The Test::Usage module is the current incarnation of this tool.
One feature of the module is that the user can ask to run only one or a selected number of tests at a time; during code development, we didn't always care to run the whole test suite each time a change was made.
It turned out that the test files, containing all those examples(), required the same intensity of work and refactoring as the main code itself. The embodied examples evolved as understanding of the requirements did. I think it helped us build better software.
The supplied ./share/TestUsageSample is an example Perl module that maintains its test suites with Test::Usage.
If you want to distribute your module in the standard Perl fashion, and want to make your Test::Usage tests the ones to be run, you need to make Test::Usage a prerequisite in your Makefile.PL, and you can use a test.pl file whose contents are like this:
use strict; use FindBin qw($Bin); use lib "$Bin/lib"; use Test::Usage; files( d => "$Bin/lib", i => "$Bin/lib", v => 2, c => 0, );
(Note that this will be evaluated in the 'main' package, where files() is visible.)
No checks are made for duplicated labels.
We want our test module to have as little influence as possible on what is being tested, but this can be problematic. For example, suppose the module being tested needs package Foo, but forgot to 'use' it. If our testing module uses Foo, the test will not reveal its absence from the main program.
If a module we are testing has an END block, it won't be invoked in time for testing.
Luc St-Louis, <lucs@cpan.org>
Copyright (C) 2005 by Luc St-Louis
This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.8.3 or, at your option, any later version of Perl 5 you may have available.
To install Test::Usage, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Test::Usage
CPAN shell
perl -MCPAN -e shell install Test::Usage
For more information on module installation, please visit the detailed CPAN module installation guide.