HTTP::WebTest - Test remote URLs or local web files
use HTTP::WebTest; my $webtest = new HTTP::WebTest; # run test from file $webtest->run_wtscript('script.wt'); # or (to pass test parameters as method arguments) $webtest->run_tests($tests);
THIS IS A BETA VERSION THAT IS A REWRITE OF VERSION 1.07 AND IS PROBABLY NOT AS WELL DEBUGGED AS VERSION 1.07. Version 1.07 can be downloaded at http://search.cpan.org/search?dist=HTTP-WebTest-1.07
This module runs tests on remote URLs or local web files containing Perl/JSP/HTML/JavaScript/etc. and generates a detailed test report. This module can be used "as-is" or its functionality can be extended using plugins. Plugins can define test types and provide additional report capabilities. This module comes with a set of default plugins but can be easily extended with third party plugins.
The wt script is provided for running HTTP::WebTest from the command line.
HTTP::WebTest
The test specifications can be read from a parameter file in wtscript format or input as method arguments. If you are testing a local file, Apache is started on a private/dynamic port with a configuration file in a temporary directory.
The test results can be displayed on the terminal, directed to a file, stored in a scalar variable. The test results can also be emailed. The report can be modified and extended using report plugins.
Each URL/web file is tested by fetching it from the web server using a local instance of an HTTP user agent. The basic test is simply whether or not the fetch was successful. You may also test using literal strings or regular expressions that are either required to exist or forbidden to exist in the fetched page. You may also specify tests for the minimum and maximum number of bytes in the returned page. You may also specify tests for the minimum and maximum web server response time.
Data flow for HTTP::WebTest using a remote URL:
-------------- ------------- | | | | | Input |------------->| WebTest | | parameters | | | | | ------------- -------------- | ^ | | V | ------------- ------------ | | request | | | Remote |<--------------| HTTP | | webserver |-------------->| user | | | response | agent | ------------- | | ------------
Data flow diagram for HTTP::WebTest using a local web file:
-------------- --------------------- | | | | | Input | | Web page code | | parameters | | (Perl/HTML/etc.) | | | | | -------------- --------------------- | | | --------------------------- | | V V ------------------------ ------------- | | | |---------->| Temporary Apache | | WebTest | | directories (htdocs, | | |<----------| conf, logs) | ------------- | | | ^ ------------------------ | | | ^ V | V | ------------ ---------------------- | | request | | | HTTP |------------>| Temporary local | | user | | instance of Apache | | agent |<------------| | | | response ---------------------- ------------
This module has complex functionality, but using it to run simple tests is simple. Create a file of test parameters in the wtscript format and use the wt program to process the file using the command wt filename. The only required parameters are test_name and url.
wt filename
test_name
url
This document describes:
How tests can be specified. See section TEST SPECIFICATION.
All test parameters supported by core HTTP::WebTest plugins. See section TEST PARAMETERS.
See "perldoc wt" for documentation on the wt program.
Other useful documentation is:
perldoc HTTP::WebTest::Cookbook - examples of wtscript files and examples of HTTP::WebTest API usage.
perldoc HTTP::WebTest::API - full documentaion on API of HTTP::WebTest.
perldoc HTTP::WebTest::Plugins - for developers of HTTP::WebTest plugins.
The test specifications can be read from a parameter file (in the wtscript format described below) or passed as method arguments as an array of hashes.
HTTP::WebTest can read test specification from file in format called as wtscript.
wtscript
Tests defined by wtscript file can be run either using Perl API of HTTP::WebTest
use HTTP::WebTest; my $webtest = new HTTP::WebTest; $webtest->run_wtscript('script.wt');
or by using program wt supplied with this module.
If you are running dozens of tests, you may want to divide them into several parameter files. This will organize the tests and reduce the size of the output and e-mail messages. However, cookies passed to or received from the web server(s) are not shared between tests in different parameter files.
The wtscript file is a text file containing global parameters and test blocks containing test block parameters. A test block begins with a test_name parameter and ends with an end_test directive. The order of the parameters and test blocks is arbitrary.
Test block parameters MUST occur between a test_name parameter and an end_test directive. (Test block parameters affect only an individual test.) Global parameters must NOT occur between a test_name parameter and an end_test directive. (This requirement does not apply to certain parameters that are both global and test block parameters.)
The following lines are ignored:
lines consisting of nothing but white space (blanks or tabs)
lines beginning with a number sign (#)
#
lines beginning with white space (blanks or tabs) followed by a number sign
Parameters are either scalar (single-valued) or lists (single or multi-valued).
You can specify scalar parameters using forms such as:
name=value name = value name = 'value'
You can specify list parameters using forms such as:
name = ( first value second value ) name=( first value => second value third value => fourth value ) name = ( first value => second value ) name = ( 'first value' 'second value' ) name= ( first value second value third value => 'fourth value' ) name = ( first value 'second value' ) name = ( 'first value' 'second value' )
You can specify a null (placeholder) value using '' or "". Within single or double quotes, the usual Perl string quoting rules apply. Thus, single quotes mean that all enclosed characters are interpreted literally: '\n' is backslash-n rather than a newline character. Double quotes mean that Perl metasymbols are interpreted: "\n\t" is a newline and a tab. Double quoted strings can also contain Perl variables to be expanded: "$var" is string which contains value of Perl variable $var. Perl variables can be defined by plugin modules or in code sections described below.
$var
Also it is possible to specify Perl code instead of scalar, instead of list parameter value or instead of element of list paramater. Curly brackets are used to denote Perl code inside wtscript files. This code will be evaluated during test run.
HTTP::WebTest compiles this Perl code as anonymous subroutines which are called during test run when value of corresponding test parameters are required. When these subroutines are called HTTP::WebTest object is passed to them.
Some examples of syntax:
# scalar value name = { 1 + 1 } # list value (Perl code should return array reference) name = { [ a => 'b', c => 'd' ] } # element of list value name = ( 'first value' { "first " . "value" } ) # accessing HTTP::WebTest object name = { my $webtest = shift; ..... }
The parameters below specify tests of a local file and a remote URL. The tests specified by the text_forbid parameter apply to both the "RayCosoft home page" and the "Yahoo home page" tests. Hence, if either returned page contains one of the case-insensitive strings in text_forbid, the test fails. If any test fails or the fetch of the URL fails, an e-mail will be sent to tester@unixscripts.com.
text_forbid
apache_exec = /usr/sbin/apache ignore_case = yes mail = errors mail_addresses = ( tester@unixscripts.com ) mail_server = mailhost.unixscripts.com text_forbid = ( Premature end of script headers an error occurred while processing this directive ) test_name = 'RayCosoft home page (static)' file_path = ( raycosoft_home.html => . ) text_require = ( <a href="/dept/peopledev/new_employee/"><font color="#0033cc"> <a href="https://www.raycosoft.com/"><font color= ) end_test test_name = Yahoo home page url = www.yahoo.com text_require = ( <a href=r/qt>Quotations</a>...<br> ) min_bytes = 13000 max_bytes = 99000 min_rtime = 0.010 max_rtime = 30.0 end_test
The parameters below specify a test of a local file containing Perl code using the Apache::ASP module. The includes.htm file requires five include files and two Perl modules, which are copied using the include_file_path parameter.
includes.htm
include_file_path
apache_exec = /usr/sbin/apache ignore_case = yes include_file_path = ( footer.inc => htdocs/apps/myapp/inc header.inc => htdocs/apps/myapp/inc head.inc => htdocs/apps/myapp/inc go.script => htdocs/shared/includes go.include => htdocs/shared/includes ../utils/DBconn.pm => lib/perl/utils ../utils/Window.pm => lib/perl/utils ) test_name = includes.htm file_path = ( includes.htm => apps/myapp ) min_bytes = 33000 max_bytes = 35000 text_require = ( input type=hidden name=control value= ) text_forbid = ( Premature end of script headers an error occurred while processing this directive ) end_test
If you are using Perl API of HTTP::WebTest then the test parameters can be defined in form of array of hashes.
Each hash in array defines tests for one URL or local web file. Keys in hashes are test parameter names and values in hashes are values of test parameters. Additionally, optional global test parameters can be passed in a hash passed as the second argument.
Instead of test parameter values subroutine references can be specified. Referenced subroutines are called during test run when values of corresponding test parameters are required. When called these subroutines get HTTP::WebTest object passed to them.
Tests can be run as
use HTTP::WebTest; my $webtest = new HTTP::WebTest; $webtest->run_tests( [ # test 1 { param1 => value1, param2 => value2 }, # test 2 { param1 => value1, param2 => value2 }, ], { global_param1 => value1, global_param2 => value2 } );
This Perl script tests Yahoo home page and sends full test report to tester@unixscripts.com.
tester@unixscripts.com
use HTTP::WebTest; my $tests = [ { name => 'Yahoo home page', url => 'http://www.yahoo.com', text_require => [ '<a href=r/qt>Quotations</a>...<br>' ], min_bytes => 13000, max_bytes => 99000, } ]; my $params = { mail_server => 'mailhost.unixscripts.com', mail_addresses => [ 'tester@unixscripts.com' ], mail => 'all', ignore_case => 'yes', }; $webtest->run_tests($tests, $params);
HTTP::WebTest provides a number of core plugin modules which are loaded by default:
This plugin provides support for local web file test mode.
This plugin provides size checks of HTTP response bodies.
This plugin provides means to control sending and recieve cookies.
Default test report plugin.
This plugin allows to load external plugin modules.
This plugin provides support for response time tests.
This plugin initializes test HTTP requests.
This plugin checks HTTP response statuses.
This plugin provides test parameters which allow to check body of HTTP responses.
Information about test parameters supported by core plugins is summarized below in section TEST PARAMETERS.
Following plugin modules come with HTTP::WebTest but they are not loaded by default. They should be loaded using global test parameter plugins when needed.
plugins
This plugin allows to use names of links and button on HTML pages to build test requests.
This report plugin can generate Test::Harness compatible test reports.
This plugin allows to define callback test parameters which are evaluated at specific time of HTTP::WebTest test run. These test parameters can define user-defined checks.
Information about test parameters supported by add-on plugin modules is summarized below in section TEST PARAMETERS.
perldoc HTTP::WebTest::Plugins contains information needed for HTTP::WebTest plugin developers.
Most parameters can be used both as global and as test block parameters. If you specify such parameter as global its value applies to all test blocks. Value of parameter specified as global can be overriden individually in each test block by specifying this parameter with different values in test blocks.
Parameters marked as GLOBAL PARAMETER can be used only as global and it cannot be overriden in test blocks.
Parameters marked as NON-CORE PARAMETER are defined in add-on plugin modules which must be loaded explicitly using test parameter plugins.
Option to accept cookies from the web server.
These cookies exist only while the program is executing and do not affect subsequent runs. These cookies do not affect your browser or any software other than the test program. These cookies are only accessible to other tests executed during test sequence execution.
See also the <send_cookies> parameter.
yes, no
yes
no
GLOBAL PARAMETER
Absolute or relative path name of directory containing Apache files. See the APACHE DIRECTORY AND FILES section. This parameter is ignored unless the file_path parameter is specified.
file_path
/usr/local/etc/http-webtest
Absolute or relative path name of Apache executable. This command can be in your $PATH. This parameter is ignored unless the file_path parameter is specified.
$PATH
/usr/sbin/apache
Apache logging level. If you use a level less than warn (i.e., debug, info, or notice), the program may generate irrelevant errors. This parameter is ignored unless the file_path parameter is specified. See also the ignore_error_log parameter.
warn
debug
info
notice
ignore_error_log
debug, info, notice, warn, error, crit, alert, emerg
error
crit
alert
emerg
Maximum number of seconds to wait for Apache to start. This parameter is ignored unless the file_path parameter is specified.
60
Additional Apache command line options. Many of the options cause Apache to exit immediately after starting, so the web page tests will not run. This parameter is ignored unless the file_path parameter is specified.
See Apache documentation
A list which contains two elements: userid/password pair to be used for web page access authorization.
NON-CORE PARAMETER from HTTP::WebTest::Plugin::Click
Given name of submit button (i.e. <input type="submit"> tag inside of <form> tag) on previosly requested HTML page builds test request to the submitted page.
<input type="submit">
<form>
Note that you still need to pass all form parameters yourself using params test parameter.
params
See example in HTTP::WebTest::Cookbook.
Given name of link (i.e. <a> tag) on previosly requested HTML page builds test request to the linked page.
<a>
Synonym to cookies.
cookies
Specifies a cookie(s) to send to the web server.
Each cookie is specified by following list:
( version name value path domain port path_spec secure maxage discard name1 value1 name2 value2 ... )
Any element not marked below as REQUIRED may be defaulted by specifying a null value or ''.
version (REQUIRED)
Version number of cookie spec to use, usually 0.
name (REQUIRED)
Name of cookie. Cannot begin with a $ character.
value (REQUIRED)
Value of cookie.
path (REQUIRED)
URL path name for which this cookie applies. Must begin with a / character. See also path_spec.
domain (REQUIRED)
Domain for which cookie is valid. (REQUIRED). Should begin with a period. Must either contain two periods or be equal to .local.
.local
port
List of allowed port numbers that the cookie may be returned to. If not specified, cookie can be returned to any port. Must be specified using the format N or N, N, ..., N where N is one or more digits.
N
N, N, ..., N
path_spec
Ignored if version is less than 1. Option to ignore the value of path. Default value is 0.
1
Use the value of path.
Ignore the specified value of path.
secure
Option to require secure protocols for cookie transmission. Default value is 0.
Use only secure protocols to transmit this cookie.
Secure protocols are not required for transmission.
maxage
Number of seconds until cookie expires.
discard
Option to discard cookie when the program finishes. Default is 0. (The cookie will be discarded regardless of the value of this element.)
Discard cookie when the program finishes.
Don't discard cookie.
name/value
Zero, one or several name/value pairs may be specified. The name parameters are words such as Comment or CommentURL and the value parameters are strings that may contain embedded blanks.
An example cookie would look like:
( 0 WebTest cookie #1 expires&2592000&type&consumer / .unixscripts.com '' 0 0 200 1 )
See RFC 2965 for details (ftp://ftp.isi.edu/in-notes/rfc2965.txt).
You may specify multiple cookies within each test block by specifying multiple instances of the cookies parameter.
Use arrayref of arrayrefs containing cookies to pass with the HTTP request.
Each array must have at least 5 elements; if the number of elements is over 10 it must have an even number of elements.
This parameter defines if default report plugin should be used for test report creation. Value yes means that default report plugin should be used, value no means that it should not. It can be useful if it is desired to use another non-default report for creation of test report. It can be used to disable any output at all also (i.e. if this parameter has value no and no other report plugins are loaded).
It is not really test parameter but a part of wtscript format. It marks end of test block. =head2 error_log
The pathname of a local web server error log. The module counts the number of lines in the error log before and after each request. If the number of lines increases, an error is counted and the additional lines are listed in the report. This argument should be used only when the local web server is running in single-process mode. Otherwise, requests generated by other processes/users may add lines to the error log that are not related to the requests generated by this module. See also parameter ignore_error_log.
A filehandle (or anything else that supports print) to use for test report output. This parameter is ignored if test parameter output_ref is specified also.
print
output_ref
This parameter can be used only when passing the test parameters as arguments from a calling Perl script.
If HTTP::WebTest encounters parameter file_path it switches in local web file test mode. In local web file test mode it launches an instance of Apache daemon, copies local test file(s) under DocumentRoot of this Apache and performs test checks against it.
Two-element list. First element is the file to test, either an absolute or a relative pathname. Second element is the subdirectory pathname, relative to the Apache htdocs directory, to copy the file to. The copied file will have the same basename as the first element and the relative pathname of the second element. To copy the file directly to the htdocs directory, use a pathname of . or ./..
.
./.
A list of HTTP header/value pairs. Can be used to override default HTTP headers or to add additional HTTP headers.
http_headers = ( Accept => text/plain, text/html )
Option to do case-insensitive string matching for text_forbid, text_require, regex_forbid and regex_require test parameters.
text_require
regex_forbid
regex_require
Option to ignore any errors found in the Apache error log. The default behavior is to flag an error if the fetch causes any errors to be added to the error log and echo the errors to the program output. This check is available only if error_log parameter is specified. See also the apache_loglevel parameter.
error_log
apache_loglevel
List with an even number of elements. Odd-numbered elements are files to copy to the the temporary Apache directory before running the tests. These files can be specified using either an absolute or a relative pathname. Even-numbered elements are the subdirectory pathname, relative to the Apache ServerRoot directory, to copy the corresponding file to. The copied file will have the same basename as the odd-numbered element and the relative pathname of the corresponding even-numbered element. To copy the file directly to the ServerRoot directory, use a pathname of . or ./..
For example:
include_file_path = (/home/tester/inc/header.inc => htdocs/includes)
will copy the file to htdocs/includes/header.inc.
This parameter is also useful for adding Perl modules that are needed by the web page specified by the file_path parameter. For example:
include_file_path = ( ../apps/myapp/DBconn.pm => lib/perl/apps )
will copy the Perl module DBconn.pm to a directory that is in the Perl @INC array.
Option to e-mail output to one or more addresses specified by mail_addresses test parameter.
mail_addresses
all
Send e-mail containing test results.
errors
Send e-mail only if one or more tests fails.
Do not send e-mail.
A list of e-mail addresses where report will be send (if sending report is enabled with mail test parameter).
mail
Sets From: header for test report e-mails.
Name of user under which test script runs.
Fully-qualified name of of the mail server (e.g., mailhost.mycompany.com).
localhost
Maximum number of bytes expected in returned page.
Any integer greater that zero and greater than min_bytes (if min_bytes is specified).
min_bytes
Maximum web server response time (seconds) expected.
Any number greater that zero and greater than min_rtime (if min_rtime is specified).
min_rtime
HTTP request method.
See RFC 2616 (HTTP/1.1 protocol).
GET, PUT
GET
PUT
Minimum number of bytes expected in returned page.
Any integer less than max_bytes (if max_bytes is specified).
max_bytes
Minimum web server response time (seconds) expected.
Any number less than max_rtime (if max_rtime is specified).
max_rtime
NON-CORE PARAMETER from HTTP::WebTest::Plugin::Hooks
Value of this test parameter is ignored. However it is evaluted before test request to web page is done so it is useful to do some initalization before the request.
This is a list parameter which is treated as test result. It is evaluted when response for the test request is received.
It can be useful to define custom tests without writting new plugins and/or it can be useful to run some code when response for the test request is received.
( YESNO1, COMMENT1 YESNO2, COMMENT2 .... YESNON, COMMENTN )
Here YESNO, COMMENT - is a test result. YESNO - is either yes if test is successful or no if it is not. COMMENT is a text of comment associated with this test.
YESNO
COMMENT
A reference on scalar which accumulates text of test report. If this test parameter is specified then value of test parameter fh_out is ignore.
fh_out
A list of name/value pairs to be passed as parameters to the URL. (This element is used to test pages that process input from forms.) Unless the method key is set to POST, these pairs are URI-escaped and appended to the requested URL.
POST
The names and values will be URI-escaped as defined by RFC 2396.
url = http://www.hotmail.com/cgi-bin/hmhome params = ( curmbox F001 A005 from HotMail )
generates the HTTP request:
http://www.hotmail.com/cgi-bin/hmhome?curmbox=F001%20A005&from=HotMail
A list which contains two elements: userid/password pair to be used for proxy server access authorization.
A list of module names. Loads these modules and registers them as HTTP::WebTest plugins. If name of plugin starts with :: prepends it with HTTP::WebTest::Plugin. So
::
HTTP::WebTest::Plugin
plugins = ( ::Click )
is equal to
plugins = ( HTTP::WebTest::Plugin::Click )
A list of service name/proxy URL pairs that specify proxy servers to use for requests.
proxies = ( http => http://http_proxy.mycompany.com ftp => http://ftp_proxy.mycompany.com )
List of regular expressions that are forbidden to exist in the returned page.
For more information, see perldoc perlre or see Programming Perl, 3rd edition, Chapter 5.
See also the text_forbid and ignore_case parameters.
ignore_case
List of regular expressions that are required to exist in the returned page.
See also the text_require and ignore_case parameters.
Option to send cookies to web server. This applies to cookies received from the web server or cookies specified using the cookies test parameter.
This does NOT give the web server(s) access to cookies created with a browser or any user agent software other than this program. The cookies created while this program is running are only accessible to other tests in the same test sequence.
See also the <accept_cookies> parameter.
Option to display any cookies sent or received.
Include request and response headers in the test report.
Include content of HTTP response in the test report.
Option to display shorter test report.
summary
Only a one-line summary for each URL
failed_only
Only tests that failed and the summary
Show all tests and the summary
Name associated with this url in the test report and error messages.
List of text strings that are forbidden to use exist in the returned page.
See also the regex_forbid and ignore_case parameters.
List of text strings that are required to use exist in the returned page.
See also the regex_require and ignore_case parameters.
URL to test. If schema part of URL is omitted (i.e. URL doesn't start with http://, ftp://, etc) then http:// is implied.
http://
ftp://
Set the product token that is used to identify the user agent on the network.
HTTP-WebTest/NN
where NN is version number of HTTP-WebTest.
NN
A tree of directories with templates of Apache config files is required to run local web file tests.
The apache_dir parameter must be set to the name of a directory that contains the subdirectories conf, logs and htdocs. The conf subdirectory must contain a file named httpd.conf-dist. The htdocs subdirectory must contain a subdirectory named webtest that contains a file named is_apache_responding.html. If your installation of Apache has the Perl module Apache::ASP configured, the apache_dir directory must also contain a subdirectory named asp_tmp.
apache_dir
conf
logs
htdocs
httpd.conf-dist
webtest
is_apache_responding.html
asp_tmp
The file httpd.conf-dist is used as template for the Apache config file. It contains tags which are replaced with the values needed by the Apache server that the program starts at runtime.
Please_do_not_modify_PORT
To be replaced with port number on which Apache runs during tests.
Please_do_not_modify_HOST_NAME
To be replace with Apache host name.
Please_do_not_modify_SERVER_ROOT
To be replaced with Apache server root.
Please_do_not_modify_LOG_LEVEL
To be replaced with Apache log level.
This module have been tested only on Unix (e.g., Solaris, Linux, AIX, etc.) but it should work on Win32 systems.
Local file tests don't work on Win32 systems.
The module's HTTP requests time out after 3 minutes (the default value for LWP::UserAgent). If the file_path parameter is specified, Apache must be installed.
Please email bug reports, suggestions, questions, etc. to HTTP::WebTest maillist http-webtest-general@lists.sourceforge.net. You can sign up at http://lists.sourceforge.net/lists/listinfo/http-webtest-general.
http-webtest-general@lists.sourceforge.net
Richard Anderson <richard@richard-anderson.org> have wrote HTTP::WebTest 1.xx.
HTTP::WebTest 1.xx
Ilya Martynov <ilya@martynov.org> made rewrite of HTTP::WebTest. New version of HTTP::WebTest have introduced extended API and plugin based architecture.
Please don't email authors directly. Use HTTP::WebTest maillists.
Copyright (c) 2000-2001 Richard Anderson. All rights reserved.
Copyright (c) 2001,2002 Ilya Martynov. All rights reserved.
This module is free software. It may be used, redistributed and/or modified under the terms of the Perl Artistic License.
HTTP::WebTest::Cookbook
HTTP::WebTest::API
HTTP::WebTest::Plugins
wt
4 POD Errors
The following errors were encountered while parsing the POD:
You forgot a '=back' before '=head3'
=back without =over
To install HTTP::WebTest, copy and paste the appropriate command in to your terminal.
cpanm
cpanm HTTP::WebTest
CPAN shell
perl -MCPAN -e shell install HTTP::WebTest
For more information on module installation, please visit the detailed CPAN module installation guide.