NAME

HTTP::WebTest - Test remote URLs or local web files

SYNOPSIS

    use HTTP::WebTest;

    my $webtest = new HTTP::WebTest;

    # run test from file
    $webtest->run_wtscript('script.wt');

    # or (to pass test parameters as method arguments)
    $webtest->run_tests($tests);

DESCRIPTION

Beta software warning

THIS IS A BETA VERSION THAT IS A REWRITE OF VERSION 1.07 AND IS PROBABLY NOT AS WELL DEBUGGED AS VERSION 1.07. Version 1.07 can be downloaded at http://search.cpan.org/search?dist=HTTP-WebTest-1.07

Introduction

This module runs tests on remote URLs or local web files containing Perl/JSP/HTML/JavaScript/etc. and generates a detailed test report. This module can be used "as-is" or its functionality can be extended using plugins. Plugins can define test types and provide additional report capabilities. This module comes with a set of default plugins but can be easily extended with third party plugins.

The wt script is provided for running HTTP::WebTest from the command line.

The test specifications can be read from a parameter file in wtscript format or input as method arguments. If you are testing a local file, Apache is started on a private/dynamic port with a configuration file in a temporary directory.

The test results can be displayed on the terminal, directed to a file, stored in a scalar variable. The test results can also be emailed. The report can be modified and extended using report plugins.

Each URL/web file is tested by fetching it from the web server using a local instance of an HTTP user agent. The basic test is simply whether or not the fetch was successful. You may also test using literal strings or regular expressions that are either required to exist or forbidden to exist in the fetched page. You may also specify tests for the minimum and maximum number of bytes in the returned page. You may also specify tests for the minimum and maximum web server response time.

Data flow for HTTP::WebTest using a remote URL:

          --------------              -------------
          |            |              |           |
          | Input      |------------->|  WebTest  |
          | parameters |              |           |
          |            |              -------------
          --------------                  |   ^
                                          |   |
                                          V   |
          -------------               ------------
          |           |    request    |          |
          | Remote    |<--------------|   HTTP   |
          | webserver |-------------->|   user   |
          |           |    response   |   agent  |
          -------------               |          |
                                      ------------

Data flow diagram for HTTP::WebTest using a local web file:

          --------------           ---------------------
          |            |           |                   |
          | Input      |           |  Web page code    |
          | parameters |           |  (Perl/HTML/etc.) |
          |            |           |                   |
          --------------           ---------------------
                |                            |
                |  ---------------------------
                |  |
                V  V              ------------------------
          -------------           |                      |
          |           |---------->| Temporary Apache     |
          |  WebTest  |           | directories (htdocs, |
          |           |<----------| conf, logs)          |
          -------------           |                      |
              |  ^                ------------------------
              |  |                        |    ^
              V  |                        V    |
          ------------             ----------------------
          |          |   request   |                    |
          |   HTTP   |------------>| Temporary local    |
          |   user   |             | instance of Apache |
          |   agent  |<------------|                    |
          |          |   response  ----------------------
          ------------

Getting Started

This module has complex functionality, but using it to run simple tests is simple. Create a file of test parameters in the wtscript format and use the wt program to process the file using the command wt filename. The only required parameters are test_name and url.

This document describes:

See "perldoc wt" for documentation on the wt program.

Other useful documentation is:

TEST SPECIFICATION

The test specifications can be read from a parameter file (in the wtscript format described below) or passed as method arguments as an array of hashes.

Running HTTP::WebTest using a parameter file

HTTP::WebTest can read test specification from file in format called as wtscript.

Tests defined by wtscript file can be run either using Perl API of HTTP::WebTest

    use HTTP::WebTest;

    my $webtest = new HTTP::WebTest;
    $webtest->run_wtscript('script.wt');

or by using program wt supplied with this module.

If you are running dozens of tests, you may want to divide them into several parameter files. This will organize the tests and reduce the size of the output and e-mail messages. However, cookies passed to or received from the web server(s) are not shared between tests in different parameter files.

File format

The following is ignored in wtscript file:

  • lines consisting of nothing but white space (blanks or tabs)

  • lines beginning with a number sign (#)

  • lines beginning with white space (blanks or tabs) followed by a number sign

The order of the parameters in the parameter file is arbitrary, with the following exceptions:

  • Test block parameters MUST occur between a test_name parameter and an end_test directive. (Test block parameters affect only an individual test.)

  • Global parameters must NOT occur between a test_name parameter and an end_test directive. (This requirement does not apply to parameters that are both global and test block parameters.)

Parameters are either scalar (single-valued) or lists (single or multi-valued).

You can specify scalar parameters using forms such as:

    name = value
    name =
           value
    name = 'value'

You can specify list parameters using forms such as:

    name = ( first value
             second value )
    name = ( first value => second value
             third value => fourth value
           )
    name = ( first value => second value )
    name = (
             'first value'
             'second value' )
    name = (
             first value
             second value
             third value => 'fourth value'
           )
    name =
           ( first value
             'second value' )
    name = (
             'first value'
             'second value'
           )

(The equals sign must be followed by a space, tab or newline; all other spaces are optional.)

PARAMETER VALUES BEGINNING AND ENDING WITH A SINGLE QUOTE WILL HAVE THE SINGLE QUOTES REMOVED. For example, 'foobar' is parsed as a value of foobar and ''foobar'' is parsed as a value of 'foobar'. To specify a null (placeholder) value, use ''.

You MUST enclose the parameter value in single quotes if you want to specify:

  • a value beginning with a left parenthesis

  • a value ending with a right parenthesis

  • a value beginning with leading white space (blanks or tabs)

  • a value ending with trailing white space (blanks or tabs)

  • a value beginning and ending with single quotes

Examples of wtscript files

The parameters below specify tests of a local file and a remote URL. The tests specified by the text_forbid parameter apply to both the "RayCosoft home page" and the "Yahoo home page" tests. Hence, if either returned page contains one of the case-insensitive strings in text_forbid, the test fails. If any test fails or the fetch of the URL fails, an e-mail will be sent to tester@unixscripts.com.

    apache_exec = /usr/sbin/apache
    ignore_case = yes
    mail = errors
    mail_addresses = ( tester@unixscripts.com )
    mail_server = mailhost.unixscripts.com
    text_forbid = ( Premature end of script headers
                    an error occurred while processing this directive
                  )

    test_name = 'RayCosoft home page (static)'
        file_path = ( raycosoft_home.html => . )
        text_require = (
            <a href="/dept/peopledev/new_employee/"><font color="#0033cc">
            <a href="https://www.raycosoft.com/"><font color=
                       )
    end_test

    test_name = Yahoo home page
        url = www.yahoo.com
        text_require = ( <a href=r/qt>Quotations</a>...<br> )
        min_bytes = 13000
        max_bytes = 99000
        min_rtime = 0.010
        max_rtime = 30.0
    end_test

The parameters below specify a test of a local file containing Perl code using the Apache::ASP module. The includes.htm file requires five include files and two Perl modules, which are copied using the include_file_path parameter.

    apache_exec = /usr/sbin/apache
    ignore_case = yes
    include_file_path = ( footer.inc => htdocs/apps/myapp/inc
                          header.inc => htdocs/apps/myapp/inc
                          head.inc   => htdocs/apps/myapp/inc
                          go.script  => htdocs/shared/includes
                          go.include => htdocs/shared/includes
                          ../utils/DBconn.pm  => lib/perl/utils
                          ../utils/Window.pm  => lib/perl/utils
                        )

    test_name = includes.htm
        file_path = ( includes.htm => apps/myapp )
        min_bytes = 33000
        max_bytes = 35000
        text_require = ( input type=hidden name=control value= )
        text_forbid  = ( Premature end of script headers
                         an error occurred while processing this directive
                       )
    end_test

Calling HTTP::WebTest from a Perl program

If you are using Perl API of HTTP::WebTest then the test parameters can be defined in form of array of hashes.

Each hash in array defines tests for one URL or local web file. Keys in hashes are test parameter names and values in hashes are values of test parameters. Additionally, optional global test parameters can be passed in a hash passed as the second argument.

Tests can be run as

    use HTTP::WebTest;

    my $webtest = new HTTP::WebTest;
    $webtest->run_tests(
                        [ # test 1
                          { param1 => value1,
                            param2 => value2 },
                          # test 2
                          { param1 => value1,
                            param2 => value2 },
                        ],
                        { global_param1 => value1,
                          global_param2 => value2 }
                       );

Example

This Perl script tests Yahoo home page and sends full test report to tester@unixscripts.com.

    use HTTP::WebTest;

    my $tests = [
                 { name         => 'Yahoo home page',
                   url          => 'http://www.yahoo.com',
                   text_require => [ '<a href=r/qt>Quotations</a>...<br>' ],
                   min_bytes    => 13000,
                   max_bytes    => 99000,
                 }
                ];

    my $params = { mail_server    => 'mailhost.unixscripts.com',
                   mail_addresses => [ 'tester@unixscripts.com' ],
                   mail           => 'all',
                   ignore_case    => 'yes',
                 };

    $webtest->run_tests($tests, $params);

TEST PARAMETERS

Most parameters can be used both as global and as test block parameters. If you specify such parameter as global its value applies to all test blocks. Value of parameter specified as global can be overriden individually in each test block by specifying this parameter with different values in test blocks.

Parameters marked as GLOBAL PARAMETER can be used only as global and it cannot be overriden in test blocks.

accept_cookies

Option to accept cookies from the web server.

These cookies exist only while the program is executing and do not affect subsequent runs. These cookies do not affect your browser or any software other than the test program. These cookies are only accessible to other tests executed during test sequence execution.

See also the <send_cookies> parameter.

Allowed values

yes, no

Default value

yes

apache_dir

GLOBAL PARAMETER

Absolute or relative path name of directory containing Apache files. See the APACHE DIRECTORY AND FILES section. This parameter is ignored unless the file_path parameter is specified.

Default value

/usr/local/etc/http-webtest

apache_exec

GLOBAL PARAMETER

Absolute or relative path name of Apache executable. This command can be in your $PATH. This parameter is ignored unless the file_path parameter is specified.

Default value

/usr/sbin/apache

apache_loglevel

GLOBAL PARAMETER

Apache logging level. If you use a level less than warn (i.e., debug, info, or notice), the program may generate irrelevant errors. This parameter is ignored unless the file_path parameter is specified. See also the ignore_error_log parameter.

Allowed values

debug, info, notice, warn, error, crit, alert, emerg

Default value

warn

apache_max_wait

GLOBAL PARAMETER

Maximum number of seconds to wait for Apache to start. This parameter is ignored unless the file_path parameter is specified.

Default value

60

apache_options

GLOBAL PARAMETER

Additional Apache command line options. Many of the options cause Apache to exit immediately after starting, so the web page tests will not run. This parameter is ignored unless the file_path parameter is specified.

Allowed values

See Apache documentation

auth

A list which contains two elements: userid/password pair to be used for web page access authorization.

Synonym to cookies.

cookies

Specifies a cookie(s) to send to the web server.

Each cookie is specified by following list:

    ( version
      name
      value
      path
      domain
      port
      path_spec
      secure
      maxage
      discard
      name1
      value1
      name2
      value2
      ...
    )

Any element not marked below as REQUIRED may be defaulted by specifying a null value or ''.

  • version (REQUIRED)

    Version number of cookie spec to use, usually 0.

  • name (REQUIRED)

    Name of cookie. Cannot begin with a $ character.

  • value (REQUIRED)

    Value of cookie.

  • path (REQUIRED)

    URL path name for which this cookie applies. Must begin with a / character. See also path_spec.

  • domain (REQUIRED)

    Domain for which cookie is valid. (REQUIRED). Should begin with a period. Must either contain two periods or be equal to .local.

  • port

    List of allowed port numbers that the cookie may be returned to. If not specified, cookie can be returned to any port. Must be specified using the format N or N, N, ..., N where N is one or more digits.

  • path_spec

    Ignored if version is less than 1. Option to ignore the value of path. Default value is 0.

    • 1

      Use the value of path.

    • Ignore the specified value of path.

  • secure

    Option to require secure protocols for cookie transmission. Default value is 0.

    • 1

      Use only secure protocols to transmit this cookie.

    • Secure protocols are not required for transmission.

  • maxage

    Number of seconds until cookie expires.

  • discard

    Option to discard cookie when the program finishes. Default 0. (The cookie will be discarded regardless of the value of this element.)

    • 1

      Discard cookie when the program finishes.

    • Don't discard cookie.

  • name/value

    Zero, one or several name/value pairs may be specified. The name parameters are words such as Comment or CommentURL and the value parameters are strings that may contain embedded blanks.

An example cookie would look like:

    ( 0
      WebTest cookie #1
      expires&2592000&type&consumer
      /
      .unixscripts.com
      ''
      0
      0
      200
      1
    )

See RFC 2965 for details (ftp://ftp.isi.edu/in-notes/rfc2965.txt).

Usage in wtscript files

You may specify multiple cookies within each test block by specifying multiple instances of the cookies parameter.

Usage in Perl scripts

Use arrayref of arrayrefs containing cookies to pass with the HTTP request.

Each array must have at least 5 elements; if the number of elements is over 10 it must have an even number of elements.

default_report

GLOBAL PARAMETER

This parameter defines if default report plugin should be used for test report creation. Value yes means that default report plugin should be used, value no means that it should not. It can be useful if it is desired to use another non-default report for creation of test report. It can be used to disable any output at all also (i.e. if this parameter has value no and no other report plugins are loaded).

Allowed values

yes, no

Default value

yes

end_test

It is not really test parameter but a part of wtscript format. It marks end of test block. =head2 error_log

GLOBAL PARAMETER

The pathname of a local web server error log. The module counts the number of lines in the error log before and after each request. If the number of lines increases, an error is counted and the additional lines are listed in the report. This argument should be used only when the local web server is running in single-process mode. Otherwise, requests generated by other processes/users may add lines to the error log that are not related to the requests generated by this module. See also parameter ignore_error_log.

fh_out

GLOBAL PARAMETER

A filehandle (or anything else that supports print) to use for test report output. This parameter is ignored if test parameter output_ref is specified also.

This parameter can be used only when passing the test parameters as arguments from a calling Perl script.

file_path

If HTTP::WebTest encounters parameter file_path it switches in local web file test mode. In local web file test mode it launches an instance of Apache daemon, copies local test file(s) under DocumentRoot of this Apache and performs test checks against it.

Allowed values

Two-element list. First element is the file to test, either an absolute or a relative pathname. Second element is the subdirectory pathname, relative to the Apache htdocs directory, to copy the file to. The copied file will have the same basename as the first element and the relative pathname of the second element. To copy the file directly to the htdocs directory, use a pathname of . or ./..

ignore_case

Option to do case-insensitive string matching for text_forbid, text_require, regex_forbid and regex_require test parameters.

Allowed values

yes, no

Default value

no

ignore_error_log

Option to ignore any errors found in the Apache error log. The default behavior is to flag an error if the fetch causes any errors to be added to the error log and echo the errors to the program output. This check is available only if error_log parameter is specified. See also the apache_loglevel parameter.

Allowed values

yes, no

Default value

no

include_file_path

List with an even number of elements. Odd-numbered elements are files to copy to the the temporary Apache directory before running the tests. These files can be specified using either an absolute or a relative pathname. Even-numbered elements are the subdirectory pathname, relative to the Apache ServerRoot directory, to copy the corresponding file to. The copied file will have the same basename as the odd-numbered element and the relative pathname of the corresponding even-numbered element. To copy the file directly to the ServerRoot directory, use a pathname of . or ./..

For example:

    include_file_path = (/home/tester/inc/header.inc => htdocs/includes)

will copy the file to htdocs/includes/header.inc.

This parameter is also useful for adding Perl modules that are needed by the web page specified by the file_path parameter. For example:

    include_file_path = ( ../apps/myapp/DBconn.pm => lib/perl/apps )

will copy the Perl module DBconn.pm to a directory that is in the Perl @INC array.

mail

GLOBAL PARAMETER

Option to e-mail output to one or more addresses specified by mail_addresses test parameter.

  • all

    Send e-mail containing test results.

  • errors

    Send e-mail only if one or more tests fails.

  • no

    Do not send e-mail.

Default value

no

mail_addresses

GLOBAL PARAMETER

A list of e-mail addresses where report will be send (if sending report is enabled with mail test parameter).

mail_from

GLOBAL PARAMETER

Sets From: header for test report e-mails.

Default Value

Name of user under which test script runs.

mail_server

GLOBAL PARAMETER

Fully-qualified name of of the mail server (e.g., mailhost.mycompany.com).

Default value

localhost

max_bytes

Maximum number of bytes expected in returned page.

Allowed values

Any integer greater that zero and greater than min_bytes (if min_bytes is specified).

max_rtime

Maximum web server response time (seconds) expected.

Allowed values

Any number greater that zero and greater than min_rtime (if min_rtime is specified).

method

HTTP request method.

See RFC 2616 (HTTP/1.1 protocol).

Allowed values

GET, PUT

Default value

GET

min_bytes

Minimum number of bytes expected in returned page.

Allowed values

Any integer less than max_bytes (if max_bytes is specified).

min_rtime

Minimum web server response time (seconds) expected.

Allowed values

Any number less than max_rtime (if max_rtime is specified).

output_ref

GLOBAL PARAMETER

A reference on scalar which accumulates text of test report. If this test parameter is specified then value of test parameter fh_out is ignore.

This parameter can be used only when passing the test parameters as arguments from a calling Perl script.

params

A list of name/value pairs to be passed as parameters to the URL. (This element is used to test pages that process input from forms.) Unless the method key is set to POST, these pairs are URI-escaped and appended to the requested URL.

For example,

    url = http://www.hotmail.com/cgi-bin/hmhome
    params = ( curmbox
               F001 A005
               from
               HotMail )

generates the HTTP request:

    http://www.hotmail.com/cgi-bin/hmhome?curmbox=F001%20A005&from=HotMail

The names and values will be URI-escaped as defined by RFC 2396.

pauth

A list which contains two elements: userid/password pair to be used for proxy server access authorization.

plugins

GLOBAL PARAMETER

A list of module names. Loads these modules and registers them as HTTP::WebTest plugins. If name of plugin starts with :: prepends it with HTTP::WebTest::Plugin. So

    plugins = ( ::ValidateHTML )

is equal to

    plugins = ( HTTP::WebTest::Plugin::ValidateHTML )

proxies

A list of service name/proxy URL pairs that specify proxy servers to use for requests.

For example (wtscript usage):

    proxies = ( http => http://http_proxy.mycompany.com
                ftp  => http://ftp_proxy.mycompany.com )

regex_forbid

List of regular expressions that are forbidden to exist in the returned page.

For more information, see perldoc perlre or see Programming Perl, 3rd edition, Chapter 5.

See also the text_forbid and ignore_case parameters.

regex_require

List of regular expressions that are required to exist in the returned page.

For more information, see perldoc perlre or see Programming Perl, 3rd edition, Chapter 5.

See also the text_require and ignore_case parameters.

send_cookies

Option to send cookies to web server. This applies to cookies received from the web server or cookies specified using the cookies test parameter.

This does NOT give the web server(s) access to cookies created with a browser or any user agent software other than this program. The cookies created while this program is running are only accessible to other tests in the same test sequence.

See also the <accept_cookies> parameter.

Allowed values

yes, no

Default value

yes

show_cookies

Option to display any cookies sent or received.

Allowed values

yes, no

Default value

no

show_html

Include content of HTTP response in the test report.

Allowed values

yes, no

Default value

no

terse

Option to display shorter test report.

  • summary

    Only a one-line summary for each URL

  • failed_only

    Only tests that failed and the summary

  • no

    Show all tests and the summary

Default value

no

test_name

Name associated with this url in the test report and error messages.

text_forbid

List of text strings that are forbidden to use exist in the returned page.

See also the regex_forbid and ignore_case parameters.

text_require

List of text strings that are required to use exist in the returned page.

See also the regex_require and ignore_case parameters.

url

URL to test. If schema part of URL is omitted (i.e. URL doesn't start with http://, ftp://, etc) then http:// is implied.

APACHE DIRECTORY AND FILES

A tree of directories with templates of Apache config files is required to run local web file tests.

The apache_dir parameter must be set to the name of a directory that contains the subdirectories conf, logs and htdocs. The conf subdirectory must contain a file named httpd.conf-dist. The htdocs subdirectory must contain a subdirectory named webtest that contains a file named is_apache_responding.html. If your installation of Apache has the Perl module Apache::ASP configured, the apache_dir directory must also contain a subdirectory named asp_tmp.

The file httpd.conf-dist is used as template for the Apache config file. It contains tags which are replaced with the values needed by the Apache server that the program starts at runtime.

  • Please_do_not_modify_PORT

    To be replaced with port number on which Apache runs during tests.

  • Please_do_not_modify_HOST_NAME

    To be replace with Apache host name.

  • Please_do_not_modify_SERVER_ROOT

    To be replaced with Apache server root.

  • Please_do_not_modify_LOG_LEVEL

    To be replaced with Apache log level.

RESTRICTIONS / BUGS

This module have been tested only on Unix (e.g., Solaris, Linux, AIX, etc.) but it should work on Win32 systems.

Local file tests don't work on Win32 systems.

The module's HTTP requests time out after 3 minutes (the default value for LWP::UserAgent). If the file_path parameter is specified, Apache must be installed.

SUPPORT

Please email bug reports, suggestions, questions, etc. to HTTP::WebTest maillist http-webtest-general@lists.sourceforge.net. You can sign up at http://lists.sourceforge.net/lists/listinfo/http-webtest-general.

AUTHORS

Richard Anderson <Richard.Anderson@unixscripts.com> have wrote HTTP::WebTest 1.xx.

Ilya Martynov <ilya@martynov.org> made rewrite of HTTP::WebTest. New version of HTTP::WebTest have introduced extended API and plugin based architecture.

Please don't email authors directly. Use HTTP::WebTest maillists.

COPYRIGHT

Copyright (c) 2000-2001 Richard Anderson. All rights reserved.

Copyright (c) 2001,2002 Ilya Martynov. All rights reserved.

This module is free software. It may be used, redistributed and/or modified under the terms of the Perl Artistic License.

SEE ALSO

HTTP::WebTest::Cookbook

HTTP::WebTest::API

HTTP::WebTest::Plugins

wt

4 POD Errors

The following errors were encountered while parsing the POD:

Around line 827:

You forgot a '=back' before '=head3'

Around line 831:

=back without =over

Around line 1058:

You forgot a '=back' before '=head3'

Around line 1062:

=back without =over