The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.

NAME

Test::Stream::Architecture - overview of how the Test-More dist works.

DESCRIPTION

This document explains the Test::More architecture from top to bottom.

KEY COMPONENTS

This is the list of primary components and their brief description, the most critical ones will have more details in later sections.

Test::More
Test::Simple

These contain the public subroutines for anyone who wishes to write tests.

Test::More::Tools

All of the tools that Test::More provided have been relocated into Test::More::Tools and refactored to make them generic and reusable.

This means you can use them without inadvertently firing off events. In many cases this is what tool builders actually want but instead they settle for bumping $Level and calling is(), like(), or ok() and producing extra events.

Test::Builder

This was the old under-the-hood module for anyone who wished to write a Test::More-compatible test library. It still works and should be fully functional and backwards compatible. It is, however, discouraged as it is mostly a compatibility wrapper.

Test::Stream

This is the new heart and soul of the Test::* architecture. It is not the primary interface that a unit-test author will use. This module is responsible for collecting all events from all threads and processes and then forwarding them to TAP and any other added listeners.

Test::Stream::IOSets

This manages the IO handles to which all TAP is sent.

In the old days, Test::Builder cloned STDERR and STDOUT and applied various magic to them.

This module provides that legacy support while also adding support for utf8 and other encodings. By default, all TAP goes to the 'legacy' outputs, which mimick what Test::Builder has always done. The 'legacy' outputs are also what get altered if someone uses the Test::Builder->output interface.

Test::Stream::Toolset

This is the primary interface a test module author should use. It ties together some key functions you should use. It provides 3 critical functions:

    is_tester($package)

    init_tester($package)

    my $ctx = context();
Test::Stream::Context

A context is used to generate events in test functions.

Once a context object is created (the normal way) it is remembered and anything that requests a context object will obtain the same instance.

After the context instance is destroyed (at end of your test function) it is forgotten. The next test function to run must obtain a new context instance.

Test::Stream::Event
Test::Stream::Event::Ok
Test::Stream::Event::Diag
Test::Stream::Event::Note
Test::Stream::Event::*

All events generated by Test::More and other test tools now boil down to a proper object. All event subclasses must use Test::Stream::Event as a base.

Test::Stream::HashBase

This is the Moose of Test::Stream. It is responsible for generating accessors and similar work.

Test::Stream::Tester

This module can validate testing tools and their events.

THE HUB OBJECT

Test::Stream

HISTORY

Test::Builder was (and still is) a singleton. The singleton model was chosen to solve the problem of synchronizing everything to a central location. Ultimately, all test results needed to make their way to a central place that could assign each test a number and create output in the correct order.

The singleton model proved to be a major headache.

Intercepting events typically meant replacing the singleton permanently (Test::Tester) or for a limited scope. Another option people took (Test::Builder::Tester) was to simply replace the IO handles Test::Builder was tracking.

Test::Builder did not provide any real mechanisms for altering events before processing them, or for intercepting them before they were turned into TAP. As a result many modules have monkeypatched Test::Builder, particularily the ok() method.

CURRENT DESIGN

Test::Stream::Hub unfortunately must still act as a singleton (mostly). This time, the design put as little logic as possible into the singleton.

RESPONSIBILITIES OF TEST::STREAM::HUB

Test::Stream::Hub has 4 main jobs:

Collect events from all threads and processes into 1 place
    $hub->send($event);

The send() method will ensure that the event gets to the right place, no matter which thread or process your code is in. (Forking support must be turned on. It is off by default).

Note: This method is key to performance. send() and everything it calls must remain as lean and tight as possible.

Provide a pre-output hook to alter events
    $hub->munge(sub { my ($hub, $event) = @_; ... })

munge() lets you modify events before they are turned into output. It cannot remove the event, nor can it add events. Mungers are additive and proceessed in the order they are added.

There is not currently any way to remove a munger.

Note: each munger is called in a loop in the send() method, so keep them as fast and small as possible.

Forward all events to all listeners (including TAP output)
    $hub->listen(sub { my ($hub, $event) = @_; .... })

listen() adds a listener. All events that come from the hub object will be sent to all listeners.

There is not currently any way to remove a listener.

Note: each listener is called in a loop in the send() method, so keep them as fast and small as possible.

Maintaining the legacy exit behavior from Test::Builder

This is sets $? to the number of tests that failed (up to 255). It also provides some other output such as when a test file is missing a plan.

SEMI-SINGLETON MODEL

Test::Stream has a semi-singleton model. Instead of 1 singleton, it has a singleton stack. Anything that wants to send an event to the current acting hub should send it to the hub returned by Test::Stream->shared. Nothing should ever cache this result as the current hub may change.

This mechanism is primarily used for intercepting and hiding all events for a limited scope. Test::Stream::Tester uses this to push a hub onto the stack so that events can be generated that do not go to the listeners or TAP. Once the stack is popped, the previous hub is restored, which allows real events to be generated.

You can also create new Test::Stream::Hub objects at-will that are not present in the stack. This lets you create alternate hubs for any purpose you want.

THE CONTEXT OBJECT

Test::Stream::Context

This module is responsible for 2 things: knowing where to report errors and making it easy to issue events.

ERROR REPORTING

Use the context() function to get the current context.

    sub ok {
        my $context = context();
        ...
    }

    ok() # Errors are reported here.

If there is a context already in play, that instance will be returned. Otherwise, a new context will be returned.

The context assumes that the stack level just above your call is where errors should be reported.

You can optionally provide an integer as the only argument, in which case that number will be added to the caller() call to find the correct frame for reporting.

Note: The integer argument will be completely ignored if there is already an active context.

    sub ok {
        my $context = context();
        ...
    }

    sub my_ok {
        my $context = context();
        ok(...);
    }

    my_ok();

In the example above, c<my_ok()> generates a new context and then it calls ok(). In this case, both functions will have the same context object (the one generated by my_ok()). The result is that ok will report errors to the correct place.

IMPLEMENTATION

There is a lexical variable $CURRENT in Test::Stream::Context that can not be directly touched. When the context() function is called, it first checks if $CURRENT is set, and if so, returns that. If there is no current context, it generates a new one.

When a new context is generated, it is assigned to $CURRENT, but then the reference is weakened. This means that once the returned copy falls out of scope, or is otherwise removed, $CURRENT will vanish on its own. This means that so long as you hold on to your context object, anything you call will find it.

The caveat here is that if you decide to hold on to your context beyond your scope, you could sabatoge any future test functions. If you need to hold on to a context you need to call $context->snapshot, and store the cloned object it returns. In general you should not need to do this. Event objects all store the context but do so using a snapshot.

Note I am open to changing this to remove the weak-reference magic and instead require someone to call $context->release or similar when they are done with a context but that seems more likely to result in rogue contexts. This method would also require its own form of reference counting. This decision will need to be made before we go stable.

GENERATING EVENTS

All event subclasses should use Test::Stream::Event to set them up as proper event objects. IT is best to use the send_event() method provided by the context object:

    my $ctx = context;
    $ctx->send_event('Ok', pass => 1, name => "pass");
    $ctx->send_event(
        'Ok',
        pass => 0,
        name      => "fail",
        diag      => ["This test failed, here is some diag ..."],
    );
    $ctx->send_event('Note', message => "I am a teapot");

EVENT OBJECTS

Here are the primary public events. There are other events, but they are used internally.

Test::Stream::Event

This is just a base class. Do not use it directly.

Test::Stream::Event::Diag
Test::Stream::Event::Note
Test::Stream::Event::Plan
Test::Stream::Event::Bail

These are fairly simple and obvious event types.

Test::Stream::Event::Ok
Test::Stream::Event::Subtest

Note: Subtest is a subclass of Ok.

Ok can contain diag objects related to that specific ok. Subtest contains all the events that went into the final subtest result.

All events have the context in which they were created, which includes the file and line number where errors should be reported. They also have details on where and how they were generated. All other details are event-specific.

The subclass event should never be generated on its own. In fact, just use the subtest helpers provided by Test::More, or Test::Stream::Context. Under the hood, a Child event is started which adds a subtest to a stack in Test::Stream, and then all events get intercepted by that subtest. When the subtest is done, issue another Child event to close it out. Once closed, a Subtest event will be generated for you and sent to the hub.

SOURCE

The source code repository for Test::More can be found at http://github.com/Test-More/test-more/.

MAINTAINER

Chad Granum <exodist@cpan.org>

AUTHORS

The following people have all contributed to the Test-More dist (sorted using VIM's sort function).

Chad Granum <exodist@cpan.org>
Fergal Daly <fergal@esatclear.ie>>
Mark Fowler <mark@twoshortplanks.com>
Michael G Schwern <schwern@pobox.com>
唐鳳

COPYRIGHT

There has been a lot of code migration between modules, here are all the original copyrights together:

Test::Stream
Test::Stream::Tester

Copyright 2014 Chad Granum <exodist7@gmail.com>.

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

See http://www.perl.com/perl/misc/Artistic.html

Test::Simple
Test::More
Test::Builder

Originally authored by Michael G Schwern <schwern@pobox.com> with much inspiration from Joshua Pritikin's Test module and lots of help from Barrie Slaymaker, Tony Bowden, blackstar.co.uk, chromatic, Fergal Daly and the perl-qa gang.

Idea by Tony Bowden and Paul Johnson, code by Michael G Schwern <schwern@pobox.com>, wardrobe by Calvin Klein.

Copyright 2001-2008 by Michael G Schwern <schwern@pobox.com>.

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

See http://www.perl.com/perl/misc/Artistic.html

Test::use::ok

To the extent possible under law, 唐鳳 has waived all copyright and related or neighboring rights to Test-use-ok.

This work is published from Taiwan.

http://creativecommons.org/publicdomain/zero/1.0

Test::Tester

This module is copyright 2005 Fergal Daly <fergal@esatclear.ie>, some parts are based on other people's work.

Under the same license as Perl itself

See http://www.perl.com/perl/misc/Artistic.html

Test::Builder::Tester

Copyright Mark Fowler <mark@twoshortplanks.com> 2002, 2004.

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.