The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.
 / 
Assert-Refute-0.14
River stage one • 5 direct dependents • 5 total dependents
1 ++
/ Assert::Refute

NAME

Assert::Refute - Unified testing and assertion tool

DESCRIPTION

This module allows injecting Test::More-like code snippets into production code, without turning the whole application into a giant testing script.

This can be though of as a lightweight design-by-contract form.

SYNOPSIS

The following code will die unless the conditions listed there are fullfilled:

    use Assert::Refute ":all", { on_fail => 'croak' };

    # Lots of code here
    try_refute {
        cmp_ok $price + $fee, "==", $total, "Money added up correctly";
        like $description, qr/\w{3}/, "A readable description is present";
        isa_ok $my_obj, "My::Class";
    };

A code snippet like this can guard important invariants, ensure data correctness, or serve as a safety net while reworking a monolithic application into separate testable modules.

Note that the inside of the block can be copied into a unit-test as is, giving one a fine-grained speed <----> accuracy control.

The same can be done without polluting the main package namespace:

    use Assert::Refute { on_fail => 'croak' };

    try_refute {
        my $report = shift;
        $report->cmp_ok( $price + $fee, "==", $total, "Money added up correctly" );
        $report->like( $description, qr/\w{3}/, "A readable description is present" );
        $report->isa_ok( $my_obj, "My::Class" );
    };

Relying on a global (in fact, per-package) callback is not required:

    use Assert::Refute {}, ":all";

    my $report = try_refute {
        # ... assertions here
    };
    if (!$report->is_passing) {
        $my_logger->error( "Something is not right: ".$report->get_tap );
        # do whatever error handling is needed
    };

See Assert::Refute::Report for more information about the underlying object-oriented interface.

ASSERTIONS, CONTRACTS, AND SUBCONTRACTS

  • We use the term assertion here to refer to a binary statement that can be proven wrong using a well-defined, finite calculation.

    We say that assertion fails if such proof is provided, and passes otherwise.

    "X equals Y" and "a string contains such and such words" are assertions by this definition. "This code terminates" isn't because it requires solving the halting problem. "All swans are white" isn't either unless there's code that produces a black swan.

  • We use the term contract here to refer to a code block containing zero or more assertions. A contract is said to fail if any of its assertions fails, and is assumed to pass otherwise.

    This is not to be confused with full-fledged design-by-contract which is much more specific about what contracts are.

  • Note that a contract itself is an assertion by this definition. We use the term subcontract to refer to an assertion that another contract passes given certain arguments.

    These building blocks allow to create and verify arbitrarily complex specifications. See "PERFORMANCE" below for limitations, though.

EXPORT

Any number of hash references may be added to the use statement, resulting in an implicit Assert::Refute->configure call. A literal {} will also trigger configure.

Everything else will be passed on to Exporter.

    use Assert::Refute;

as well as

    use Assert::Refute qw(:core);

would only export try_refute, contract, refute, contract_is, subcontract, and current_contract functions.

Also for convenience some basic assertions mirroring the Test::More suite are exportable via :all and :basic export tag.

    use Assert::Refute qw(:all);

would also export the following assertions:

is, isnt, ok, use_ok, require_ok, cmp_ok, like, unlike, can_ok, isa_ok, new_ok, is_deeply, note, diag.

See Assert::Refute::T::Basic for more.

This distribution also bundles some extra assertions:

These need to be used explicitly.

try_refute { ... }

Check whether given contract BLOCK containing zero or more assertions passes.

Contract will fail if any of the assertions fails, a plan is declared and not fullfilled, or an exception is thrown. Otherwise it is assumed to pass.

The BLOCK must accept one argument, the contract execution report, likely a Assert::Refute::Report instance.

More arguments MAY be added in the future. Return value is ignored.

A read-only report instance is returned by try_refute instead.

If on_pass/on_fail callbacks were specified during use or using configure, they will also be executed if appropriate.

If NDEBUG or PERL_NDEBUG environment variable is set at compile time, this block is replaced with a stub which returns an unconditionally passing report.

This is basically what one expects from a module in Assert::* namespace.

refute_these

[DEPRECATED] Same as above.

It will stay available (with a warning) until as least 0.15.

contract { ... }

Save a contract BLOCK for future use:

    my $contract = contract {
        my ($foo, $bar) = @_;
        # conditions here
    };

    # much later
    my $report = $contract->apply( $real_foo, $real_bar );
    # Returns an Assert::Refute::Report with conditions applied

This is similar to how prepare / execute works in DBI.

[DEPRECATED] This function will disappear in v.0.20.

Prior to advent of try_refute, this call used to be the main entry point to this module. This is no more the case, and a simple subroutine containing assertions would fit in most places where contract is appropriate.

Use "contract" in Assert::Refute::Contract instead.

plan tests => $n

Plan to run exactly n assertions within a contract block. Plan is optional, contract blocks can run fine without a plan.

A contract will fail unconditionally if plan is present and is not fullfilled.

plan may only be called before executing any assertions. plan dies if called outside a contract block.

Not exported by default to avoid namespace pollution.

plan skip_all => $reason

[EXPERIMENTAL]. Like above, but plan is assumed to be zero and a reason for that is specified.

Note that the contract block is not interrupted, it's up to the user to call return. This MAY change in the future.

refute( $reason, $message )

Verify (or, rather, try hard to disprove) an assertion in scope of the current contract.

The test passes if the $reason is false, i.e. an empty string, 0, or undef. Otherwise the $reason is assumed to be a description of what went wrong.

You can think of it as ok and diag from Test::More combined:

    ok !$reason, $message
        or diag $reason;

As a special case, a literal 1 is considered to be a boolean value and the assertions just fails, without further explanation.

As another special case, an \@arrayref reason will be unfolded into multiple diag lines, for instance

    refute [ $answer, "isn't", 42 ], "life, universe, and everything";

will output 3 diag lines.

Returns true for a passing assertion and false for a failing one. Dies if no contract is being executed at the time.

subcontract( "Message" => $contract, @arguments )

"The specified contract passes, given the arguments" assertion. This is similar to subtest in Test::More.

[NOTE] that the message comes first, unlike in refute or other assertion types, and is required.

A contract may be an Assert::Refute::Contract object, a plain subroutine with some assertions inside, or an Assert::Refute::Report instance from a previous contract run.

A subroutine MUST accept an empty Assert::Refute::Report object.

For instance, one could apply a previously defined validation to a structure member:

    my $valid_email = contract {
        my $email = shift;
        # ... define your checks here
    };

    my $valid_user = contract {
        my $user = shift;
        is ref $user, 'HASH'
            or die "Bail out - not a hash";
        like $user->{id}, qr/^\d+$/, "id is a number";
        subcontract "Check e-mail" => $valid_email, $user->{email};
    };

    # much later
    $valid_user->apply( $form_input );

Or pass a definition as argument to be applied to specific structure parts (think higher-order functions, like map or grep).

    my $array_of_foo = contract {
        my ($is_foo, $ref) = @_;

        foreach (@$ref) {
            subcontract "Element check", $is_foo, $_;
        };
    };

    $array_of_foo->apply( $valid_user, \@user_list );

contract_is

    contract_is $report, $signature, "Message";

Assert that a contract is fullfilled exactly to the specified extent. See "get_sign" in Assert::Refute::Report for signature format.

This may be useful for verifying assertions and contracts themselves.

This is actually a clone of "contract_is" in Assert::Refute::T::Basic.

current_contract

Returns the Assert::Refute::Report object being worked on.

If Test::Builder has been detected and no contract block is executed explicitly, returns a Assert::Refute::Driver::More instance. This allows to define assertions and run them uniformly under both Assert::Refute and Test::More control.

Dies if no contract could be detected.

It is actually a clone of "current_contract" in Assert::Refute::Build.

STATIC METHODS

Use these methods to configure Assert::Refute globally.

configure

    use Assert::Refute \%options;
    Assert::Refute->configure( \%options );
    Assert::Refute->configure( \%options, "My::Package");

Set per-caller configuration values for given package. configure is called implicitly by use Assert::Refute { ... } if hash parameter(s) are present.

%options may include:

  • on_pass - callback to execute if tests pass (default: skip)

  • on_fail - callback to execute if tests fail (default: carp, but not just Carp::carp - see below).

  • driver - use that class instead of Assert::Refute::Report as contract report.

  • skip_all - reason for skipping ALL try_refute blocks in the affected package. This defaults to PERL_NDEBUG or NDEBUG environment variable.

    [EXPERIMENTAL]. Name and meaning MAY change in the future.

The callbacks MUST be either a CODEREF accepting Assert::Refute::Report object, or one of predefined strings:

  • skip - do nothing;

  • carp - warn the stringified report;

  • croak - die with stringified report as error message;

Returns the resulting config (with default values added,etc).

As of current, this method only affects try_refute.

get_config

Returns configuration from above, initializing with defaults if needed.

EXTENDING THE SUITE

Although building wrappers around refute call is easy enough, specialized tool exists for doing that.

Use Assert::Refute::Build to define new checks as both prototyped exportable functions and their counterpart methods in Assert::Refute::Report. These functions will perform absolutely the same under control of try_refute, contract, and Test::More:

    package My::Prime;

    use Assert::Refute::Build;
    use parent qw(Exporter);

    build_refute is_prime => sub {
        my $n = shift;
        return "Not a natural number: $n" unless $n =~ /^\d+$/;
        return "$n is not prime" if $n <= 1;
        for (my $i = 2; $i*$i <= $n; $i++) {
            return "$i divides $n" unless $n % $i;
        };
        return '';
    }, args => 1, export => 1;

Much later:

    use My::Prime;

    is_prime 101, "101 is prime";
    is_prime 42, "Life is simple"; # not true

Note that the implementation sub {...} only cares about its arguments, and doesn't do anything except returning a value. Suddenly it's a pure function!

Yet the exact reason for $n not being a prime will be reflected in test output.

One can also subclass Assert::Refute::Report to create new drivers, for instance, to register failed/passed tests in a unit-testing framework of choice or generate warnings/exceptions when conditions are not met.

That's how Test::More integration is done - see Assert::Refute::Driver::More.

PERFORMANCE

Set NDEBUG or PERL_NDEBUG (takes precedence) environment variable to true to replace all try_refute blocks with a stub. Carp::Assert was used as reference.

If that's not enough, use Keyword::DEVELOPMENT or just define a DEBUG constant and append an if DEBUG; statement to try_refute{ ... } blocks.

That said, refute is reasonably fast. Special care is taken to minimize the CPU usage by passing contracts.

The example/00-benchmark.pl file in this distribution is capable of verifying around 4000 contracts of 100 statements each in just under a second on my 4500 BOGOMIPS laptop. Your mileage may vary!

WHY REFUTE

Communicating a passing test normally requires 1 bit of information: everything went as planned. For failing test, however, as much information as possible is desired.

Thus refute($condition, $message) stands for an inverted assertion. If $condition is false, it is regarded as a success. If it is true, however, it is considered to be the reason for a failing test.

This is similar to how Unix programs set their exit code, or to Perl's own $@ variable, or to the falsifiability concept in science.

A subcontract is a result of multiple checks, combined into a single refutation. It will succeed silently, yet spell out details if it doesn't pass.

These primitives can serve as building blocks for arbitrarily complex assertions, tests, and validations.

SEE ALSO

Test::More, Carp::Assert, Keyword::DEVELOPMENT

BUGS

This module is still under heavy development. See TODO file in this distribution for an approximate roadmap.

New features are marked as [EXPERIMENTAL]. Features that are to be removed will stay [DEPRECATED] (with a corresponding warning) for at least 5 releases, unless such deprecation is extremely cumbersome.

Test coverage is maintained at >90%, but who knows what lurks in the other 10%.

See https://github.com/dallaylaen/assert-refute-perl/issues to browse old bugs or report new ones.

SUPPORT

You can find documentation for this module with the perldoc command.

    perldoc Assert::Refute

You can also look for information at:

ACKNOWLEDGEMENTS

  • Thanks to Alexander Kuklev for try_refute function name as well as a lot of feedback.

  • This rant by Daniel Dragan inspired me to actually start working on the first incarnation of this project.

LICENSE AND COPYRIGHT

Copyright 2017-2018 Konstantin S. Uvarin. <khedin at cpan.org>

This program is free software; you can redistribute it and/or modify it under the terms of the the Artistic License (2.0). You may obtain a copy of the full license at:

http://www.perlfoundation.org/artistic_license_2_0

Any use, modification, and distribution of the Standard or Modified Versions is governed by this Artistic License. By using, modifying or distributing the Package, you accept this license. Do not use, modify, or distribute the Package, if you do not accept this license.

If your Modified Version has been derived from a Modified Version made by someone other than you, you are nevertheless required to ensure that your Modified Version complies with the requirements of this license.

This license does not grant you the right to use any trademark, service mark, tradename, or logo of the Copyright Holder.

This license includes the non-exclusive, worldwide, free-of-charge patent license to make, have made, use, offer to sell, sell, import and otherwise transfer the Package with respect to any patent claims licensable by the Copyright Holder that are necessarily infringed by the Package. If you institute patent litigation (including a cross-claim or counterclaim) against any party alleging that the Package constitutes direct or contributory patent infringement, then this Artistic License to you shall terminate on the date that such litigation is filed.

Disclaimer of Warranty: THE PACKAGE IS PROVIDED BY THE COPYRIGHT HOLDER AND CONTRIBUTORS "AS IS' AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES. THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT ARE DISCLAIMED TO THE EXTENT PERMITTED BY YOUR LOCAL LAW. UNLESS REQUIRED BY LAW, NO COPYRIGHT HOLDER OR CONTRIBUTOR WILL BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING IN ANY WAY OUT OF THE USE OF THE PACKAGE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.