Test::DocClaims - Help assure documentation claims are tested
To automatically scan for source files containing POD, find the corresponding tests and verify that those tests match the POD, create the file t/doc_claims.t with the following lines:
use Test::More; eval "use Test::DocClaims"; plan skip_all => "Test::DocClaims not found" if $@; all_doc_claims();
Or, for more control over the POD files and which tests correspond to them:
use Test::More; eval "use Test::DocClaims"; plan skip_all => "Test::DocClaims not found" if $@; plan tests => 2; doc_claims( "lib/Foo/Bar.pm", "t/doc-Foo-Bar.t", "doc claims in Foo/Bar.pm" ); doc_claims( "lib/Foo/Bar/Baz.pm", "t/doc-Foo-Bar-Baz.t", "doc claims in Foo/Bar/Baz.pm" );
If a source file (lib/Foo/Bar.pm) contains:
=head2 add I<arg1> I<arg2> This adds two numbers. =cut sub add { return $_[0] + $_[1]; }
then the corresponding test (t/doc-Foo-Bar.t) might have:
=head2 add I<arg1> I<arg2> This adds two numbers. =cut is( add(1,2), 3, "can add one and two" ); is( add(2,3), 5, "can add two and three" );
A module should have documentation that defines its interface. All claims in that documentation should have corresponding tests to verify that they are true. Test::DocClaims is designed to help assure that those tests are written and maintained.
It would be great if software could read the documentation, enumerate all of the claims made and then generate the tests to assure that those claims are properly tested. However, that level of artificial intelligence does not yet exist. So, humans must be trusted to enumerate the claims and write the tests.
How can Test::DocClaims help? As the code and its documentation evolve, the test suite can fall out of sync, no longer testing the new or modified claims. This is where Test::DocClaims can assist. First, a copy of the POD documentation must be placed in the test suite. Then, after each claim, a test of that claim should be inserted. Test::DocClaims compares the documentation in the code with the documentation in the test suite and reports discrepancies. This will act as a trigger to remind the human to update the test suite. It is up to the human to actually edit the tests, not just sync up the documentation.
The comparison is done line by line. Trailing white space is ignored. Any white space sequence matches any other white space sequence. Blank lines as well as "=cut" and "=pod" lines are ignored. This allows tests to be inserted even in the middle of a paragraph by placing a "=cut" line before and a "=pod" line after the test.
Additionally, a special marker, of the form "=for DC_TODO", can be placed in the test suite in lieu of writing a test. This serves as a reminder to write the test later, but allows the documentation to be in sync so the Test::DocClaims test will pass with a todo warning. Any text on the line after DC_TODO is ignored and can be used as a comment.
Especially in the SYNOPSIS section, it is common practice to include example code in the documentation. In the test suite, if this code is surrounded by "=begin DC_CODE" and "=end DC_CODE", it will be compared as if it were part of the POD, but can run as part of the test. For example, if this is in the documentation
Here is an example: $obj->process("this is some text");
this could be in the test
Here is an example: =begin DC_CODE =cut $obj->process("this is some text"); =end DC_CODE
Example code that uses print or say and has a comment at the end will also match a call to is() in the test. For example, this in the documentation POD
The add function will add two numbers: say add(1,2); # 3 say add(50,100); # 150
will match this in the test.
The add function will add two numbers: =begin DC_CODE =cut is(add(1,2), 3); is(add(50,100), 150); =end DC_CODE
When comparing code inside DC_CODE markers, all leading white space is ignored.
When the documentation file type does not support POD (such as mark down files, *.md) then the entire file is assumed to be documentation and must match the POD in the test file. For these files, leading white space is ignored. This allows a leading space to be added in the POD if necessary.
Verify that the lines of documentation in TEST_SPEC match the ones in DOC_SPEC. The TEST_SPEC and DOC_SPEC arguments specify a list of one or more files. Each of the arguments can be one of:
- a string which is the path to a file or a wildcard which is expanded by the glob built-in function. - a ref to a hash with these keys: - path: path or wildcard (required) - has_pod: true if the file can have POD (optional) - a ref to an array, where each element is a path, wildcard or hash as above
If a list of files is given, those files are read in order and the documentation in each is concatenated. This is useful when a module file requires many tests that are best split into multiple files in the test suite. For example:
doc_claims( "lib/Foo/Bar.pm", "t/Bar-*.t", "doc claims" );
If a wildcard is used, be sure that the generated list of files is in the correct order. It may be useful to number them (such as Foo-01-SYNOPSIS.t, Foo-02-DESCRIPTION.t, etc).
This is the easiest way to test the documentation claims. It automatically searches for documentation and then locates the corresponding test file or files. By default, it searches the lib, bin and scripts directories and their subdirectories for documentation. For each of these files it looks in (by default) the t directory for one or more matching files. It does this with the following patterns, where PATH is the path of the documentation file with the suffix removed (e.g., .pm or .pl) and slashes (/) converted to dashes (-). The patterns are tried in this order until one matches.
doc-PATH-[0-9]*.t doc-PATH.t PATH-[0-9]*.t PATH.t
If none of the patterns match, the left most directory of the PATH is removed and the patterns are tried again. This is repeated until a match is found or the PATH is exhausted. If the pattern patches multiple files, these files are processed in alphabetical order and their documentation is concatenated to match against the documentation file.
If DOC_DIRS is missing or undef, its default value of [qw< lib bin scripts >] is used. If TEST_DIRS is missing or undef, its default value of [qw< t >] is used.
When searching for documentation files, any file with one of these suffixes is used:
*.pl *.pm *.pod *.md
Also, any file who's first line matches /^#!.*perl/i is used.
The number of tests run is determined by the number of documentation files found. Do not set the number of tests before calling all_doc_claims because it will do that automatically.
Devel::Coverage, POD::Tested, Test::Inline. Test::Pod, Test::Pod::Coverage, Test::Pod::Snippets, Test::Synopsis, Test::Synopsis::Expectation.
Scott E. Lee, <ScottLee@cpan.org>
Copyright (C) 2009-2016 by Scott E. Lee
This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.18.2 or, at your option, any later version of Perl 5 you may have available.
To install Test::DocClaims, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Test::DocClaims
CPAN shell
perl -MCPAN -e shell install Test::DocClaims
For more information on module installation, please visit the detailed CPAN module installation guide.