Test::BrewBuild::Tutorial - Basic overview of using the client/server aspects of the Test::BrewBuild build system
This document gives a basic overview of setting up remote test servers, and how to dispatch build requests to them.
At this time, there is no security around the network portions of this suite. To mitigate, you can use IP filters, SSH tunnels etc. We do however do a reasonably good job of ensuring the commands executed on the testers have been verified.
Read through the brewbuild documentation so you know how the actual tester works. Documentation for the scripts used in this tutorial can be found in "SEE ALSO".
You'll need Git on both the dispatcher and all testers, your module accessible via a Git repository, and the dispatch and all tester servers configured, see "SERVER CONFIGURATION".
Recommended initial platform configuration guide: Test Platform Configuration.
This applies to all Test::BrewBuild systems, whether they'll be stand-alone, a dispatcher or a tester.
Test::BrewBuild
You've got a Linux box at 10.1.1.1 with all IP addresses and ports bound on the server available for listening on.
10.1.1.1
You've also got a Windows server on 172.16.1.1, but our default port TCP 7800 is not available, which can only listen on that specific IP address.
172.16.1.1
TCP 7800
You will be dispatching from a FreeBSD server, so you also want to run a tester on the localhost.
localhost
On the dispatcher, we'll be located within a Git repository directory, so we'll let the system take care of the repo management so we don't have to continuously specify it.
See "SEE ALSO" for more elaborate uses, command-line options of each script, and details of the optional configuration file.
Since we can listen on all IPs and the default port is available for use, this simple command will start the tester, and put it into the background:
bbtester start
On this server, we're not permitted to listen on all IP addresses, and since port 7800 is unavailable, we have to specify an alternate.
7800
bbtester start -i 172.16.1.1 -p 9999
Since this tester is the dispatching server, we don't want it listening on public-facing IP addresses, so we'll specify just the local loopback:
bbtester start -i 127.0.0.1
brewbuild itself can send in test run to remotes in an extremely basic way, but we're going to focus on bbdispatch here which allows for far more complex cases.
brewbuild
bbdispatch
By default, we dispatch the most basic build, by sending brewbuild as the test command. The following will dispatch a build to all three testers we configured above, and then wait for them to return the results.
bbdispatch -t 127.0.0.1 -t 172.16.1.1:9999 -t 10.1.1.1
To specify a different command string:
bbdispatch -c "brewbuild -r -R -d 7" -t 127.0.0.1 -t 172.16.1.1:9999 -t 10.1.1.1
To specify a different repository:
bbdispatch -r https://github.com/user/repo -t 127.0.0.1 -t 172.16.1.1:9999 -t 10.1.1.1
Note that the repository flag (-r) will prepend https://github.com/ by default if you only specify user/repo.
-r
https://github.com/
user/repo
Example output of a basic run where I'm in a git repository directory:
bbdispatch -t localhost -t 192.168.252.90 -t 192.168.252.96 -t 192.168.252.95 192.168.252.95 - x86_64-linux 5.22.1 :: PASS 192.168.252.90 - MSWin32-x64-multi-thread 5.18.4 :: PASS 5.22.1 :: PASS localhost - MSWin32-x64-multi-thread 5.22.1 :: FAIL 192.168.252.96 - amd64-freebsd 5.22.1 :: PASS 5.23.7 :: PASS 5.8.9 :: FAIL 5.10.1 :: FAIL 5.18.4 :: FAIL
All FAIL log files are stored locally when dispatching to identify the issues:
192.168.252.96_5.10.1-FAIL.bblog 192.168.252.96_5.18.4-FAIL.bblog 192.168.252.96_5.8.9-FAIL.bblog localhost_5.22.1-FAIL.bblog
In your configuration file, you can set up all of your testers, along with the preferred command string (and repository if need be):
[dispatch] cmd = brewbuild -R -d 7 testers = 127.0.0.1, 172.16.1.1:9999, 10.1.1.1
Then your dispatch run can be initiated simply by:
There are a handful of plugins available for Test::BrewBuild. The default is Test::BrewBuild::Plugin::DefaultExec, which performs the most basic of runs.
Test::BrewBuild::Plugin::Author will perform the same tests as the default plugin, but will additionally attempt to set RELEASE_TESTING=1 while logging on failure, and will also proceed to attempt an install of the basic distributions that perform the author tests (eg: Pod::Coverage, Test::Pod::Coverage and Test::Manifest).
RELEASE_TESTING=1
Search the CPAN for other Test::BrewBuild::Plugin plugins.
Test::BrewBuild::Plugin
To use a plugin, use the --plugin or -p argument to brewbuild.
--plugin
-p
brewbuild --plugin Test::BrewBuild::Plugin::Author
Some plugins allow you to send in arguments to it. The way this works is that the args are sent in within a list, and on each iteration of a test run, the next arg in the list is passed in. Each run gets exactly one argument from the list:
brewbuild -p Test::BrewBuild::Plugin::TestAgainst -a Module:A -a Module::B
If your dispatcher doesn't seem to be doing the right thing, you can enable debug logging, which will print directly to STDOUT:
STDOUT
bbdispatch [...] -d 7
If your testers don't seem to be behaving properly, first, log into the remote server and stop the currently running tester:
bbtester stop
Restarting the tester in daemon mode with a debug level will include its logging in the return to the dispatcher:
bbtester start -d 7
You can also run the tester in the foreground, and get its logging displayed to STDOUT live-time:
bbtester --fg -d 7 --stdout
If you want to get the debugging information from the actual brewbuild process included in the tester debug results, call brewbuild from the dispatcher with debug flags enabled. These debug results will be either included in the tester's return, or printed to its STDOUT, depending on how you're debugging the tester:
bbdispatch [...] -c "brewbuild -d 7"
Using the various APIs, you can enclose all debug output and return values into one scalar variable, and either dump it to a file or examine it all in one location:
use Capture::Tiny qw(capture_stdout); use Test::BrewBuild::Dispatch; use Test::BrewBuild::Tester; my $return; my $stdout = capture_stdout { my $d = Test::BrewBuild::Dispatch->new(debug => 7); my $t = Test::BrewBuild::Tester->new(debug => 7); $t->start; $return = $d->dispatch( cmd => 'brewbuild -i 5.10.1_32 -d 7', repo => 'https://stevieb9@github.com/stevieb9/mock-sub', testers => [ qw(10.1.1.1) ], ); $t->stop; }; $return .= $stdout; print $return;
Quick start basic example, with one remote tester.
Tester 10.1.1.1:
brewbuild:
brewbuild -D -t 10.1.1.1
Dispatcher:
bbdispatch -t 10.1.1.1
Auto run a specific number of dispatch runs to the testers. You *must* specify the repository URL on the command line when using auto mode:
bbtester start -a bbdispatch -t 10.1.1.1 -a -r ... # run forever bbdispatch -t 10.1.1.1 -a 10 -r ... # run 10 cycles
Here's an example of a full-blown automated run. It will automatically check the commit checksums on the local copy of a repository, and the remote. If they differ, we'll run the test suite.
You'll want to run the commands likely from somewhere like the @reboot crontab entry of the user you want running the tests.
@reboot
crontab
bbtester start -a bbdispatch -t localhost -r https://github.com/user/repo -a
You can run the test suite on every iteration even if the repository checksums are the same (ie. no commits have been pushed to remote).
bbtester start -a -c bbdispatch -t localhost -r https://github.com/user/repo -a
We provide the ability to run tests on a Raspberry Pi, and optionally display the result information to an LCD screen including date/time, pass/fail status, the last commit tested, and the number of runs that have completed. First, you need to set up an environment variable set to the pins the LCD is connected to (RS, E, D0, D1, D2, D3):
BB_RPI_LCD=5,6,4,17,27,22
The above is for 2 row, 16 column LCD units. To use the larger 4 row by 20 column LCD, append the number of rows and columns to the end of the environment variable:
BB_RPI_LCD=5,6,4,17,27,22,4,20
Then use the special --rpi flag in the dispatch command:
--rpi
bbtester start -a bbdispatch -t localhost -r https://github.com/user/repo -a --rpi
To run a tester at startup, add similar lines (correct to suit your environment) to your crontab (hint: `crontab -e`):
PATH=/home/pi/perl5/perlbrew/perls/perl-5.30.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin @reboot perl /home/pi/perl5/perlbrew/perls/perl-5.30.0/bin/bbtester start > /tmp/cron_bbtester.log 2>&1
The first is to get your current Perl into the PATH environment variable, the second actually starts the tester after a restart of the system.
PATH
The redirection at the end of the second line creates a debug output file, which you can review if you find the tester not starting up properly.
Details on the configuration file.
Dispatch script usage information: bbdispatch.
Tester script usage information: bbtester.
BrewBuild script usage information: brewbuild.
To install Test::BrewBuild, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Test::BrewBuild
CPAN shell
perl -MCPAN -e shell install Test::BrewBuild
For more information on module installation, please visit the detailed CPAN module installation guide.