++ed by:
ILUX

1 PAUSE user
2 non-PAUSE users.

Author image Abe Timmerman

NAME

FAQ - Test::Smoke frequently asked questions

DESCRIPTION

What is Test::Smoke?

Test::Smoke is the symbolic name for set of scripts and modules that try to run the perl core tests on as many configurations as possible and combine the results into an easy to read report.

The basic cycle looks like:

    fetch_sourcetree
    foreach $config ( @configurations ) {
        make distclean
        ./Configure $config
        make
        make test (and log failures)
    }
    create_report()
    mail_report()

Why is Test::Smoke?

The reports from Test::Smoke give a good picture of the current state of Perl development. We try to get as many of the supported platforms as possible.

Where is Test::Smoke?

On CPAN: http://search.cpan.org/authour/ABELTJE/Test-Smoke

What are these configuration files about?

Test::Smoke uses two types of configuration files:

smoke-configuration-file

This is a file that uses a _config suffix and contains settings for the smokeperl.pl program and is created by configsmoke.pl.

This is basically a Data::Dumper::Dump() of a hash.

build-configurations-file

This file contains the information needed to assemble the list of perl configurations that will be used to build and test perl.

The file is devided into sections separated by a line that starts with an equals-sign (=). Each line in a section is combined with all lines from all the other sections. The default build-configurations file perlcurrent.cfg constist of four (4) sections:

section 1 one (1) (blank) line
section 2 two (2) lines
section 3 seven (7) lines
section 4 two (2) lines (and a policy indicator line)

This file wil yield 1 * 2 * 7 * 2 = 28 configurations. Test::Smoke will build perl in all of these 28 configurations and run the core test suite one (1), two (2) or three (3) times for each of those perls:

  • with $ENV{PERLIO}=stdio (allways)

  • with $ENV{PERLIO}=perlio (unless -Uuseperlio)

  • with $ENV{PERL_UNICODE}="", $ENV{LC_ALL}=xx_YY.utf8 (if specified and !-Uuseperlio)

How do I create different smoke-configurations?

By specifying a diffent prefix to configsmoke.pl:

    $ perl configsmoke.pl -p mysmoke

This will create the smokeconfiguration mysmoke_config and a small shellscript mysmoke.sh (or mysmoke.cmd if you are on Windows).

There are two (2) special prefixes: smoke562 and smoke58x. These will set the default answers for configsmoke.pl to be appropriate for smoking the maintenance branches for these versions of perl.

What are all the scripts in the smoke suite for?

smokeperl.pl

This is a wrapper script that goes through all the steps in the smoke process.

synctree.pl

Small wrapper around Test::Smoke::Syncer that can use the settings from the smoke-configuration file. It will bring the source-tree in sync.

patchtree.pl

Small wrapper around Test::Smoke::Patcher that can use the settings from the smoke-configuration file. It will apply the patches that are specified in the patches-to-be-applied file.

mktest.pl

This is the main script that runs Configure, make and make test for all configurations and log the results in <builddir>/mktest.out.

As of 1.17_55 (1.18-to-be) the use of mktest.pl is deprecated in favour of the new runsmoke.pl frontend to Test::Smoke::Smoker. This script will not be maintained!

runsmoke.pl

Small wrapper around Test::Smoke::run_smoke() that can use the settings from the smoke-configuration file. It runs the smoke cycle for all configurations and is a replacement for the now unmaintained mktest.pl.

mkovz.pl

This script generates the report as <builddir>/mktest.rpt from <builddir>/mktest.out. It can also send the report, but mailrpt.pl is the preferred way.

mailrpt.pl

Small wrapper around Test::Smoke::Mailer that can use the settings from the smoke-configuration file. It will check if the report is already generated and do so if not (by calling mkovz.pl).

archiverpt.pl

This script copies the report and the logfile to a configurable place.

smokestatus.pl

This script prints a little status report for a smoke confifuration:

    $ ./smokestatus.pl -c smoke58x
    Busy smoking patchlevel 20562.
    13 out of 18 configurations finished in 7 hours 19 minutes.
    10 configuration(s) showed failures (F).
    0 failure(s) in the running configuration.
    5 configuration(s) to finish, in est. 2 hours 49 minutes.
W32Configure.bat

Small wrapper around Test::Smoke::Util::Configure_win32() that can use some of the settings from the smoke-configuration file.

This helps users on Windows to create the <builddir>/win32/smoke.mk makefile for a specific configuration in a unixy way.

Can I interrupt a smoke run?

Yes, you can kill all the smoke related processes. Each stage is directly logged in <builddir>/mktest.out.

As of version 1.17.02 you can specify a timeout (if you have alarm())

Can I still generate a report after an interrupted smoke?

Yes, but some information might be missing (the compiler info).

    $ perl mailrpt.pl -c [configname]

Where do the reports go?

To the address configured in configsmoke.pl. The default is <smokers-reports@perl.org>.

There are several ways to monitor the reports send in:

Subscribe to the mailinglist

You can subscribe to that list by sending an empty message to <smokers-reports-subscribe@perl.org>. You do not have to be subscribed to send reports!

The WWW archive

The mailinglist has an archive on the web (although this archive seems to be behind at the moment): http://archive.develooper.com/daily-build-reports@perl.org/.

The usenet mirror

The mailinglist also has a mirror on usenet: <news:nntp.perl.org/perl.daily-build.reports>

Posts to the newsgroup will also be mirrored on the mailinglist.

The usenet WWW archive

The usenet mirror also has an archive on the web: http://nntp.x.perl.org/group/perl.daily-build.reports

How do I investigate failures?

When investigating a testfailure you need to make sure you have Configured and build perl with the same options as the one with the failures.

    $ cd <builddir>
    $ make distclean
    $ sh Configure -des -Dusedevel [other build options]
    $ make
    $ make test-prep
    $ cd t

Now you can start with a manual run of the failing test:

    $ ./perl harness -v <pathto/failingtest>
Note for Win32

As Windows lacks a Configure to create the Makefile, Test::Smoke supplies the helper script W32Configure.bat that will create a new Makefile with the right options: <builddir>\win32\smoke.mk.

After that you can:

    S:\perl-current\win32>nmake -f smoke.mk
    S:\perl-current\win32>nmake -f smoke.mk test-prep
    S:\perl-current\win32>cd ..\t

For -Duseperlio builds (per default) failures can occur in different environments, and you will also need to set these up:

PERLIO=stdio
    $ PERLIO=stdio ./perl harness -v <pathto/failingtest>

Note that Windows builds (MSVC, Borland or MinGW) also need the "crlf" layer:

    C:>(set PERLIO=stdio :crlf) && .\perl harness -v <pathto\failingtest>

Be careful with the environment variables on Windows, they are sticky.

PERLIO=perlio
    $ PERLIO=perlio ./perl harness -v <pathto/failingtest>

Note that Windows builds (MSVC, Borland or MinGW) also need the "crlf" layer:

    C:>(set PERLIO=perlio :crlf) && .\perl harness -v <pathto\failingtest>

Be careful with the environment variables on Windows, they are sticky.

UTF-8 locale

If the failure occurs in the UTF-8 locale test-run the environment setup is a bit different (assuming "en_US.utf8"):

    $ PERL_UNICODE= LC_ALL=en_US.utf8 ./perl harness -v <pathto/failingtest>

What's with the "daily-build" and "smokers" names?

There has been enough discussion on the subject, so lets agree to disagree (it was all before my time).

The case is now that the official name for the mailinglist is daily-build-reports and there is an alias to smokers-reports.

REVISION

$Id: FAQ 401 2003-08-22 00:50:55Z abeltje $

COPYRIGHT

(c) 2002-2003, All rights reserved.

  * Abe Timmerman <abeltje@cpan.org>

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

See:

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.