- Module descriptions
- Other Resources
Bundle::Test - A set of modules for software and hardware testing using Perl
A bundle does not have code in it. It is only a list of related modules. It exists only to make it easier to install these modules with one command:
perl -MCPAN -e 'install Bundle::Test'
This bundle, however, in addition to the plain list of modules, also contains some explanation about the modules. There are explanataion even about modules not installed by this bundle but related to testing. Read on.
This bundle installs a set of modules you can use to build all kinds of tests both for your Perl modules and for applications written in any language.
If you know what would you like to test and are looking for the tools, you came to the right place. After reading a short explanation here you'll be redirected to the documentation of the respective modules.
If you'd like to learn about the testing framework in Perl you should start by reading Test::Tutorial which is a good introduction. After that go on and read the documentation of Test::Simple and Test::More.
Fields that can be tested using Perl
- Web Applications
- Web Services
- Networking devices using Command Line Interface (CLI)
- GUI based applications on Windows
- GUI based applications on X-Windows
Test::Simple - can be used as a first attempt to write simple test for your perl module. Don't worry, the work you invest here can be transfered when you move using Test::More without loss and without hassel.
Test::More is an extension of Test::Simple. It provides a number of tools to compare various data structures of Perl.
With Test::Harness you are lucky. In most of the cases you won't need to learn it as it contains a framework to execute the various test scripts created using the other modules. At some point later in your test development you might need this though in order to execute all your tests nicely.
Test::MockObject One of the biggest problems in testing is that you have to control your environment. This module lets you do just that.
Test::Pod provides a way to test the syntax of your PODs.
Test::Builder is used by most of the modern test modules to have a common back-end. When you need to implement your own domain specific test functions using Test::Builder will ensure your functions fit in the general framework and play well with the other Test modules.
Your application will have to print warnings on certain occassions that are considerd partially as failor. It is always a good idea to make sure when the conditions are met you indeed print out the correct warning. Test::Warn provides tool for this.
Some people like to add their tests right next to their code. This module let's you do just that. Add some POD documentation to your code and embed your tests in your documentation next to your code.
Checking if all the modules and script in the distribution have
use strict; use warnings;
and if they can compile using -c
Testing several aspects of a distibution. It has an odd interface with the rest of the TAP modules as it computes its own plan.
Test the Kwalitee of a module as described in http://cpants.perl.org
When you are testing your code, you can apply various measurements on how good your tests are. One of the best measurements is to check weather your test suit executes all parts of your code. Checking this is called coverage testing. In perl the ultimate tool for this is Devel::Cover.
One warning though: The fact that all your tests pass and they fully cover your code does not mean that it is bug free. Consider this example:
Our super-duper calculator program is this:
Our test suit indeed prints 3. Test passes OK.
myapp.pl 6 2
Checking coverage: the code has full coverage.
Still it is obvious that there is a bug in there, that will cause a crash.
Easy database access, mapping each table to a perl class.
A deep-structure testing module.
Modules specific to Microsoft Windows.
This is a Windows specific module for testing the GUI part of the application. It can be used at simple tasks such as automating the installation of an application up till full testing of the application.
There are lots of web applications out there that are built specifically for Internet Explorer. It would be very hard to test them with anything else than the real thing. Win32::IE::Mechanizes provides an API very similar to the one of WWW::Mechanize that will use a real Internet Explorer as the user agent. Of course for this you will have to run the testing suit on Windows.
Modules specific to Unix/Linux or those that do not work on other platforms.
Being able to click on windows and type in texts in the X-Windows environment on Unix/Linux/etc. Unfortunately as of June 2004, this module does not provide a way to read what is the text in a Widget.
This module can only be used on Unix/Linux though I think it can be installed on Windows if using Cygwin. It is a port of the Tcl based Expect tool. It provides the user with an easy way to mimic the behaviour of a human being using a command line application. This means if you have a device that you need to configure via a telnet session, you can describe every step you would do manually and then let your Expect script execute it on your behalf.
The basic idea behind this module is that working with a command line appltication involves two steps repeated a lot of times.
- 1 User types in something and presses ENTER
- 2 User waits for some response Expecting a string. Upon receiving that string s/he goes back to step 1
Modules specific to Apache
CPAN Test Results: http://testers.cpan.org/
In addition to being a repository for Perl modules CPAN also provides a framework for testing these modules in various environments and platforms. As the sole developer of a module usually has access to only his/her own development environment it is extremly useful to have other people volunteer their own computers for testing purposes. These people are called the CPAN Tester.
Every time a new version of a module is uploaded to PAUSE and thus to CPAN an e-mail is sent out to the mailing list of the CPAN testers. Based on this e-mail they can automate their system to download the new module and initiate an installation cycle. What they are interested is usually not the installed module but the test phase of the installation process. The results of these test runs are then sent back to a database. You can view the data at http://testers.cpan.org/
The mailing list of the CPAN Testers: http://lists.cpan.org/showlist.cgi?name=cpan-testers
QA and kwalitee
QA and kwalitee for Perl: http://qa.perl.org/
This web site and the accompaining mailing list http://lists.cpan.org/showlist.cgi?name=perl-qa are the central place for all the questions regarding QA within Perl. The site contains a number of links to various resources related to Quality Assurance. In the mailing list there are quite a number of people who can answer questions regarding the use of Perl in QA.
Perl QA Wiki: http://perl-qa.yi.org/
- CPANTS http://www.pobox.com/~schwern/talks/CPANTS/
- An introduction to Testing by chromatic http://www.perl.com/pub/a/2001/12/04/testing.html
- Web Testing with HTTP::Recorder by Linda Julien http://www.perl.com/pub/a/2004/06/04/recorder.html
- A Test::MockObject Illustrated Example by chromatic http://www.perl.com/pub/a/2002/07/10/tmo.html
- Testing mod_perl 2.0 by Geoffrey Young http://www.perl.com/pub/a/2003/05/22/testing.html
- Building Testing Libraries by Casey West http://www.perl.com/pub/a/2004/05/07/testing.html
- Chris Dolan: Private Regression Tests http://www.chrisdolan.net/talk/2005/11/14/private-regression-tests/
Perl books at http://books.perl.org/ Testing related books: Perl Testing: A Developer's Notebook: http://books.perl.org/book/236
You write test in order make your software of higher quality. An important requirement for this is that your testing tools should be of high quality.
If you find a bug in any of the modules mentioned in this document please report it by submitting a bug report to RT http://rt.cpan.org/ the bug tracking system of CPAN. Reports submitted there will be automatically sent out to the maintainer of the module. This way bugs and their resolution (or the lack of it) are all logged in a central place and others evaluating a modul can easily look up the history of the reported bugs and see how many bug reports are currently open. An easy way to send a bug report to RT is to send an email to an address similar to the following address: bug-MODULE-NAME@rt.cpan.org
E.g. in case you'd like to report a bug about Bundle::Test, send an e-mail to firstname.lastname@example.org
Specifically in the case of this module it is less important to have a log of the reports as this module has no code in it. You can send your reports directly to mailto:email@example.com
Stevan Little for his suggestions
The author provides training sessions for QA engineers that covers a number of fields where Perl can be used in QA Automation. The syllabus of the course can be found here: L<http://szabgab.com/perl_in_test_automation.html> In addition he provides an irregular newsletter with Test Automatin Tips L<http://szabgab.com/test_automation_tips.html>
Gabor Szabo L<mailto:firstname.lastname@example.org> L<http://www.pti.co.il/>
Copyright 2002-2009 by Gabor Szabo L<http://www.pti.co.il>. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.