The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.

NAME

Test::Stream::Architecture - Overview of how the Test-More dist works.

DESCRIPTION

This is the document that explains the architecture of Test::More and all the stuff driving it under the hood.

KEY COMPONTENTS

This is the list of primary components and their brief description, The most critical ones will have more details in later sections.

Test::More
Test::Simple

These are the primary public interfaces for anyone who wishes to write tests.

Test::More::Tools

All of the tools Test::More provides have been relocated and refactored into Test::More::Tools in such a way as to make them generic and reusable. This means you can use them without firing off events, you can then fire off your own events compiled from multiple tools. In many cases this is what tool builders actually want, but instead they settle for bumping $Level and calling is/like/ok and producing extra events.

Test::Builder

This used to be the main under the hood module for anyone who wished to write a Test::More compatible test library. It still works, and should be fully functional and backwards compatible. It is however discouraged as it is mostly a compatability wrapper.

Test::Stream

This is the new heart and soul of the Test::* architecture. However it is not the primary interface. This module is responsible for collecting all events from all threads and processes, then forwarding them to TAP and any added listeners.

Test::Stream::IOSets

This module is used to manage the IO handles to which all TAP is sent. Test::Builder cloned STDERR and STDOUT, then applied various magic to them. This module provides that legacy support while also adding support for utf8 and other encodings. By default all TAP goes to the 'legacy' outputs, which mimick what Test::Builder has always done. The 'legacy' outputs are also what get altered if someone uses the Test::Builder->output interface.

Test::Stream::Toolset

This is the primary interface a test module author should use. It ties together some key functions you should use. It proved 3 critical functions:

    is_tester($package)

    init_tester($package)

    my $ctx = context();
Test::Stream::Context

This is the primary interface as far as generating events goes. Every test function should grab a context, and use it to generate events.

Once a context object is created (the normal way) it is remembered, and anything that requests a context object will obtain the same instance. However once the instance is destroyed (end of your test function) it is forgotten, the next test function to run will then obtain a new context instance.

Test::Stream::Event
Test::Stream::Event::Ok
Test::Stream::Event::Diag
Test::Stream::Event::Note
Test::Stream::Event::*

All events generated by Test::More and other test tools now boil down to a proper object. All events must use Test::Stream::Event as a base.

Test::Stream::ArrayBase

This is the Moose of Test::Stream. It is responsible for generating accessors and similar work. Unlike moose and others it uses an arrayref as the underlying object. This design descision was made to improve performance. Performance was a real problem in some early alphas, the gains from the descision are huge.

Test::Stream::Tester

This is actually what spawned the ideas for the new Test::Stream work. This is a module that lets you validate your testing tools.

THE STREAM OBJECT

Test::Stream

HISTORY

Test::Builder is/was a singleton. The singleton model was chosen to solve the problem of synchronizing everything to a central location. Ultimately all results need to make their way to a central place that can assign them a number, and shove them through the correct output.

The singleton model proved to be a major headache.

Intercepting events typically meant replacing the singleton perminantly (Test::Tester) or for a limited scope. Another option people took (Test::Builder::Tester) was to simply replace the IO handles Test::Builder was tracking.

Test::Builder did not provide any real mechanisms for altering events before processing them, or for intercepting them before they were turned into TAP. As a result many modules have monkeypatched Test::Builder, particularily the ok() method.

CURRENT DESIGN

Test::Stream unfortunately must still act as a singleton (mostly). But this time the design was to put as little as possible into the singleton.

RESPONSIBILITIES OF TEST::STREAM

Test::Stream has 4 main jobs:

Collect events from all threads and processes into 1 place
    $stream->send($event);

The send() method will ensure that the event gets to the right place, no matter what thread or process you are in. (Forking support must be turned on, it is off by default).

Note: This method is key to performance. This method and everything it calls must remain as lean and tight as possible.

Provide a pre-output hook for altering events
    $stream->munge(sub { my ($stream, $event) = @_; ... })

This lets you modify events before they are turned into output. You cannot remove the event, nor can you add events. Mungers are additive, and proceessed in the order they are added.

There is not currently any way to remove a munger.

Note: each munger is called in a loop in the send() method, so keep it is fast and small as possible.

Forward all events to listeners (including TAP output)
    $stream->listen(sub { my ($stream, $event) = @_; .... })

This lets you add a listener. All events that come to the stream object will be sent to all listeners.

There is not currently any way to remove a listener.

Note: each listener is called in a loop in the send() method, so keep it is fast and small as possible.

Maintaining the legacy exit behavior from Test::Builder

This is primarily setting $? to the number of tests that failed, up to 255, as well as providing other output such as missing a plan.

SEMI-SINGLETON MODEL

Test::Stream has a semi-singleton model. Instead of 1 singleton, it is a singleton stack. Anything that wants to send an event to the current acting stream should send it to the stream returned by Test::Stream->shared. Nothing should ever cache this result as the current stream may change.

This mechanism is primarily used for intercepting, and hiding, all events for a limited scope. Test::Stream::Tester uses this to push a stream onto the stack so that you can generate events that do not go to the listeners or TAP. Once the stack is popped the previous stream is restored allowing you to generate real events.

You can also create new Test::Stream objects at-will that are not present in the stack, this lets you create alternate streams for any purpose you want.

THE CONTEXT OBJECT

Test::Stream::Context

This module is responsbile for 2 things, knowing where to report errors, and making it easy to issue events.

ERROR REPORTING

To get the context you use the context() function.

    sub ok {
        my $context = context();
        ...
    }

    ok() # Errors are reported here.

If there is a context already in play, that instance will be returned. Otherwise a new context will be returned. The context assumes that the stack level just above your call is where errors should be reported.

You can optionally provide an integer as the only argument, in whcih case that number will be added to the caller() call to find the correct frame for reporting. This will be completely ignored if there is already an active context.

    sub ok {
        my $context = context();
        ...
    }

    sub my_ok {
        my $context = context();
        ok(...);
    }

    my_ok();

In the example above c<my_ok()> generates a new context, then it calls ok(), in this case both function will have the same context object, the one generated by my_ok. The result is that ok will report errors to the correct place.

IMPLEMENTEATION

There is a variable $CURRENT in Test::Stream::Context, it is a lexical, so you can not touch it directly. When the context() function is called, it first checks if $CURRENT is set, if so it returns that. If there is no current context it generates a new one.

When a new context is generated, it is assigned to $CURRENT, but then the reference is weakened. This means that once the returned copy falls out of scope, or is otherwise removed, $CURRENT will vanish on its own. This means that so long as you hold on to your context object, anything you call will find it.

The caveat here is that if you descide to hold on to your context beyond your scope, you could sabatoge any future test functions. If you need to hold on to a context you need to call $context->snapshot, and store the cloned object it returns. In general you should not need to do this, event objects all store the context, but do so using a snapshot.

Note I am open to changing this to remove the weak-reference magic and instead require someone to call $context->release or similar when they are done with a context, but that seams more likely to result in rougue contexts... This method would also require its own form of reference counting.. This descision will need to be made before we go stable.

GENERATING EVENTS

All event objects should use Test::Stream::Event which will set them up as a proper event object, as well as add a method to Test::Stream::Context which is a shortcut for generating that event type. As such you can fire off an event directly from your context object using the lowercase name of the event class.

    my $ctx = context;
    $ctx->ok(1, "pass");
    $ctx->ok(0, "fail, ["This test failed, here is some diag ..."]);
    $ctx->note("I am a teapot");

All events take a context, and 2 other arguments as the first 3 arguments of their constructor, these shortcut methods handle those first 3 arguments for you, making life much easier.

The other arguments are:

created

Should be an arrayref with caller information for where the event was generated.

in_subtest

True if the event belongs in a subtest, false otherwise.

EVENT OBJECTS

Here are the primary/public events. There are other events, but they are used internally.

Test::Stream::Event

This is just a base-class, you do not use it directly.

Test::Stream::Event::Diag
Test::Stream::Event::Note
Test::Stream::Event::Plan
Test::Stream::Event::Bail

These are faily simple and obvious event types.

Test::Stream::Event::Ok
Test::Stream::Event::Subtest

Note: Subtest is a subclass of Ok.

Ok can contain diag objects related to that specific ok. Subtest contains all the events that went into the final subtest result.

All events have a context in which they were created, which includes the file and line number where errors should be reported. They also have details on where/how they were generated. All other details are event specific.

The subclass event should never be generated on its own. In fact, just use the subtest helpers provided by Test::More, or Test::Stream::Context. Under the hood a Child event is started which adds a subtest to a stack in Test::Stream, all events then get intercepted by that subtest. When the subtest is done you issue another Child event to close it out. Once closed a Subtest event will be generated for you and sent to the stream.

SOURCE

The source code repository for Test::More can be found at http://github.com/Test-More/test-more/.

MAINTAINER

Chad Granum <exodist@cpan.org>

AUTHORS

The following people have all contributed to the Test-More dist (sorted using VIM's sort function).

Chad Granum <exodist@cpan.org>
Fergal Daly <fergal@esatclear.ie>>
Mark Fowler <mark@twoshortplanks.com>
Michael G Schwern <schwern@pobox.com>
唐鳳

COPYRIGHT

There has been a lot of code migration between modules, here are all the original copyrights together:

Test::Stream
Test::Stream::Tester

Copyright 2014 Chad Granum <exodist7@gmail.com>.

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

See http://www.perl.com/perl/misc/Artistic.html

Test::Simple
Test::More
Test::Builder

Originally authored by Michael G Schwern <schwern@pobox.com> with much inspiration from Joshua Pritikin's Test module and lots of help from Barrie Slaymaker, Tony Bowden, blackstar.co.uk, chromatic, Fergal Daly and the perl-qa gang.

Idea by Tony Bowden and Paul Johnson, code by Michael G Schwern <schwern@pobox.com>, wardrobe by Calvin Klein.

Copyright 2001-2008 by Michael G Schwern <schwern@pobox.com>.

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

See http://www.perl.com/perl/misc/Artistic.html

Test::use::ok

To the extent possible under law, 唐鳳 has waived all copyright and related or neighboring rights to Test-use-ok.

This work is published from Taiwan.

http://creativecommons.org/publicdomain/zero/1.0

Test::Tester

This module is copyright 2005 Fergal Daly <fergal@esatclear.ie>, some parts are based on other people's work.

Under the same license as Perl itself

See http://www.perl.com/perl/misc/Artistic.html

Test::Builder::Tester

Copyright Mark Fowler <mark@twoshortplanks.com> 2002, 2004.

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.