Slay::Makefile::Gress - Use Slay::Makefile for software regression testing
This module provides support for running a set of regression tests from a .t file by doing builds with a
To use this module, the .t file calling it should contain something like:
use Slay::Makefile::Gress qw(do_tests); do_tests("SlayMakefile", @ARGV);
The basic functionality of this module is to take an initialization subdirectory (by default ending ".init") and copy it over to a run directory (by default ending ".dir"), doing a
chdir into the run directory, and then parsing a
Slay::Makefile to define and run the tests. A reasonable way to accomplish this is to have the
Slay::Makefile create a file with extension ".ok" that is empty if the test passes; frequently this file can be the output of a diff between the test's output and an expect file. The
Slay::Makefile can be in a parent directory so it can be shared by more than one suite of tests (.t file). A shared
Slay::Makefile can use the include mechanism to bring in a local
Slay::Makefile files in the run directory. You can even use this methodology for developing families of suites of tests, etc.
do_tests($makefile[, @tests] [, \%options ])
Runs a series of tests using
@testsare specified, it contains the list of tests, each of which is a target that is built in order; otherwise the dependencies of the
testtarget will be built as the list of tests. The following options are recognized:
The extension for the initialization directory. Default is '.init'.
A hash reference to be passed to Slay::Makefile::new as its options list.
The name of the target to be built prior to running tests to set everything up. Default is 'pretest'.
The extension for the run directory. Default is '.run'.
The name for perl scripts to run to check whether all tests should be skipped. The name is also used as an extension for a test's base name to see if an individual tests should be skipped.
The name of the target whose dependencies gives the list of tests. Default is 'test'.
Processing proceeds by the following steps:
Search for an initialization directory with the same base name as the .t file invoking
do_testsand extension equal to the
initoption. Croaks if there is no such directory. For example, if the file invoking c<do_tests> is
cmdline.tand the default initialization extension is used, it looks for directory
Copy the initialization directory to a run directory and
cdinto that directory.
Check for a script with the name of the
skipoption. If it exists, execute it. If it returns a non-zero exit code, skip all the tests. The text this script prints becomes the reason for skipping the tests.
Slay::Makefileto parse the
$makefilefile. Note that the working directory is the run directory when this file is processed.
pretesttarget, if it exists. The name of the
pretesttarget is 'pretest' unless specified in the options.
@testsis empty, create a list of tests to execute by getting the dependencies of the
testtarget. The name of the
testtarget is 'test' unless specified in the options.
For each test
Check for a script with the same base name as
tand the extension
.and the name of the
skipoption.. For example, if the default value of the
skipoption is used, then a test
algebra.okwould use a script called
algebra.skip.pl. If the script exists, execute it and skip the test if it returns a non-zero exit code. The text this script prints becomes the reason for skipping the test.
If no file
twas generated, report a failed test as failing to build the file. If
twas generated, then it should be empty for a passing test. Any text in the file is returned as the reason for the tests's failure.