The London Perl and Raku Workshop takes place on 26th Oct 2024. If your company depends on Perl, please consider sponsoring and/or attending.
 / 
Test-ManyParams-0.04
River stage one • 1 direct dependent • 2 total dependents
1 ++
/ Test::ManyParams

NAME

Test::ManyParams - module to test many params as one test

SYNOPSIS

  use Test::ManyParams;

  all_ok {foo(@_)}  
         [ [$arg1a, $arg2a], [$arg2b, $arg2b, $arg3b, $arg4b] ],
         "Testing that foo returns true for every combination of the arguments";
         
  all_ok {bar(shift())}
         [qw/arg1 arg2 arg3 arg4 arg5 arg6/],
         "Testing every argument with bar";
         
  all_are       CODE  VALUE,   PARAMETERS, [ TEST_NAME ]
  all_arent     CODE  VALUE,   PARAMETERS, [ TEST_NAME ]
  
  any_ok {drunken_person() eq shift()}
         ["Jim Beam", "Jonny Walker", "Jack Daniels"];
  
  [NOT YET IMPLEMENTED]
  
  all_are_deeply  CODE  SCALAR,  PARAMETERS, [ TEST_NAME ]
  all_like        CODE  REGEXP,  PARAMETERS, [ TEST_NAME ]
  all_unlike      CODE  REGEXP,  PARAMETERS, [ TEST_NAME ]
  all_can         CODE  METHODS, PARAMETERS, [ TEST_NAME ]
  all_dies_ok     CODE           PARAMETERS, [TEST_NAME]
  all_lives_ok    CODE           PARAMETERS, [TEST_NAME]
  all_throws_ok   CODE  REGEXP,  PARAMETERS, [TEST_NAME]

DESCRIPTION

GENERAL PRINCIPLES

This module helps to tests many parameters at once. In general, it calls the given subroutine with every combination of the given parameter values. The combinations are created with building a cross product.

Especially it avoids writing ugly, boring code like:

  my $ok = 1;
  foreach my $x ($arg1a, $arg2a) {
      foreach my $y ($arg2a, $arg2b, $arg3b, $arg4b) {
          $ok &&= foo($x,$y);
      }
  }
  ok $ok, $testname;

Instead you simpler write

  all_ok {foo(@_)}  
         [ [$arg1a, $arg2a], [$arg2b, $arg2b, $arg3b, $arg4b] ]
         $testname;

Additionally the output contains also some useful information about the parameters that should be tested and the first parameters the test failed. E.g.

  all_ok {$_[0] != 13 and $_[1] != 13} 
         [ [1 .. 100], [1 .. 100] ], 
         "No double bad luck";

would print:

  not ok 1 - No double bad luck
  #     Failed test (x.pl at line 5)
  # Tests with the parameters: $VAR1=[[10,11,12,13,14,15],[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]];
  # Failed first using these parameters: $VAR1=[10,13];

The parameters passed to all_ok can be passed in two ways. If you need to test a crossproduct of more than one parameterlist, you have to write it as

  all_ok CODE [ \@arglist1, \@arglist2, ..., \@arglistn ], TEST_NAME;

The CODE-routine will be called with every combination of the arglists, passed as simple arguments. E.g.

  all_ok {foo(@_)} [ ["red", "green", "blue"], ["big", "medium", "little"] ];

would call

  foo("red","big");
  foo("red","medium");
  foo("red","little");
  foo("green","big");
  foo("green","medium");
  foo("green","little");
  foo("blue","big");
  foo("blue","medium");
  foo("blue","little");

Note, that the order of calling shouldn't play any role, as it could be changed in future versions without any notice about.

Please always remember, that a crossproduct of the lists can be very, very big. So don't write something like all_ok {&foo} [ [1 .. 32000], [1 .. 32000], [1 .. 32000] ], as it would test 32_768_000_000_000 parameter combinations.

If you only want to test one parameter with different values, you can write in general

  all_ok CODE \@values, TEST_NAME

So all_ok {&foo} [1,2,3] would call foo(1); foo(2); foo(3).

Please take care, that the first element of the values list isn't an array ref, as Test::ManyParams would assume that you want to test combinations of the above. If it is important to pass values that are array refs, you have to write it this way:

  my @values = ( [1 .. 10],
                 [100 .. 110],
                 [990 .. 1000] );
  all_ok {&foo} [ [@values] ];
  
  # calls foo(1), ... foo(10), foo(100), ..., foo(110), foo(990), ..., foo(1000)

what is very different to

  all_ok {&foo} [ @values ];
  
  # what would call foo(1,100,900), foo(1,100,901), ...

Of course, the test name is always optional, but recommended.

FUNCTIONS

all_ok CODE PARAMETERS, [ TEST_NAME ]

See the general comments.

all_are CODE VALUE, PARAMETERS, [ TEST_NAME ]

The equivalent to Test::More's is method. The given subroutine has to return always the given value. They are compared with 'eq'.

all_arent CODE VALUE, PARAMETERS, [ TEST_NAME ]

The equivalent to Test::More's isnt method. The given subroutine has to return always values different from the given one. They are compared with 'eq'.

any_ok CODE PARAMETERS, [ TEST_NAME ]

Returns whether the subroutine returns true for one of the given parameters.

EXPORT

all_ok all_are all_arent

BUGS

The representation of the parameters uses Data::Dumper. As this module neither set $Data::Dumper::Indent, nor reads it out, setting $Data::Dumper::Indent to some strange values can destroy a useful parameter outprint. I don't plan to fix this behaviour in the next time, as I there a more important things to do. (Who changes global variables harvest what he/she/it has seed.)

There are perhaps many mistakes in this documentation.

Please tell me everything you can find.

TODO

There are a lot of methods I'd like to implement still. The most of them are simple Here's a list of them:

all_are_deeply CODE SCALAR, PARAMETERS, [ TEST_NAME ]
all_like CODE REGEXP, PARAMETERS, [ TEST_NAME ]
all_unlike CODE REGEXP, PARAMETERS, [ TEST_NAME ]
all_can CODE REGEXP, PARAMETERS, [ TEST_NAME ]
all_dies_ok CODE PARAMETERS, [TEST_NAME]
all_lives_ok CODE PARAMETERS, [TEST_NAME]
all_throws_ok CODE REGEXP, PARAMETERS, [TEST_NAME]

Similar methods are planned with the prefix any_.

Then I'd like to implement a method that uses only some random choosen parameters instead of all parameters.

That is important when the parameter set is too big to have a full test of on them.

This method will look like

  most_ok CODE  PARAMETERS  =>  PART,  [TEST_NAME]

where the PART is something like a normal number, defining the number of parameters that have to be tested, or a percentage rate or a time rate. Also an addition to say that all bounds have to be tested is planned. Typical examples for PART could be '1000', '1%', '5s', '100 + bounds'.

The pro and contra of had been discussed a bit on perl.qa. One of the results is that random parameter tests are very sensful, but the reproducibility is very important. So the module has to seed (or recognise the seed) of the random generator and to give the possibility to set them (e.g. with use Test::ManyParams seed = 42>). Recognising a failed test, this seed has to be printed. It always seems to be sensful to set an own random numbering for each package using this module.

That's only a short synopsis of this discussion, it will be better explained when these features are built in.

Of course, there will be also some methods like most_are,most_arent,... .

SEE ALSO

This module had been and will be discussed on perl.qa.

Test::More Test::Exception

THANKS

Thanks to Nicholas Clark and to Tels (http://www.bloodgate.com) for giving a lot of constructive ideas.

AUTHOR

Janek Schleicher, <bigj@kamelfreund.de>

COPYRIGHT AND LICENSE

Copyright 2002 by Janek Schleicher

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.