NAME

bisect.pl - use git bisect to pinpoint changes

SYNOPSIS

    # When did this become an error?
    .../Porting/bisect.pl -e 'my $a := 2;'
    # When did this stop being an error?
    .../Porting/bisect.pl --expect-fail -e '1 // 2'
    # When were all lines matching this pattern removed from all files?
    .../Porting/bisect.pl --match '\b(?:PL_)hash_seed_set\b'
    # When was some line matching this pattern added to some file?
    .../Porting/bisect.pl --expect-fail --match '\buseithreads\b'
    # When did this test program stop exiting 0?
    .../Porting/bisect.pl -- ./perl -Ilib ../test_prog.pl
    # When did this test start failing?
    .../Porting/bisect.pl -- ./perl -Ilib t/TEST op/sort.t
    # When did this first become valid syntax?
    .../Porting/bisect.pl --target=miniperl --end=v5.10.0 \
         --expect-fail -e 'my $a := 2;'
    # What was the last revision to build with these options?
    .../Porting/bisect.pl --test-build -Dd_dosuid

DESCRIPTION

Together bisect.pl and bisect-runner.pl attempt to automate the use of git bisect as much as possible. With one command (and no other files) it's easy to find out

• Which commit caused this example code to break?

• Which commit caused this example code to start working?

• Which commit added the first file to match this regex?

• Which commit removed the last file to match this regex?

usually without needing to know which versions of perl to use as start and end revisions.

By default bisect.pl will process all options, then use the rest of the command line as arguments to list system to run a test case. By default, the test case should pass (exit with 0) on earlier perls, and fail (exit non-zero) on blead (note that running most of perl's test files directly won't do this, you'll need to run them through a harness to get the proper error code). bisect.pl will use bisect-runner.pl to find the earliest stable perl version on which the test case passes, check that it fails on blead, and then use bisect-runner.pl with git bisect run to find the commit which caused the failure.

Because the test case is the complete argument to system, it is easy to run something other than the perl built, if necessary. If you need to run the perl built, you'll probably need to invoke it as ./perl -Ilib ...

You need a clean checkout to run a bisect, and you can't use the checkout which contains Porting/bisect.pl (because git bisect) will check out a revision before Porting/bisect-runner.pl was added, which git bisect run needs). If your working checkout is called perl, the simplest solution is to make a local clone, and run from that. i.e.:

    cd ..
    git clone perl perl2
    cd perl2
    ../perl/Porting/bisect.pl ...

By default, bisect-runner.pl will automatically disable the build of DB_File for commits earlier than ccb44e3bf3be2c30, as it's not practical to patch DB_File 1.70 and earlier to build with current Berkeley DB headers. (ccb44e3bf3be2c30 was in September 1999, between 5.005_62 and 5.005_63.) If your db.h is old enough you can override this with -Unoextensions.

OPTIONS

• --start commit-ish

  Earliest revision to test, as a commit-ish (a tag, commit or anything else git understands as a revision). If not specified, bisect.pl will search stable perl releases until it finds one where the test case passes. The default is to search from 5.002 to 5.14.0. If bisect.pl detects that the checkout is on a case insensitive file system, it will search from 5.005 to 5.14.0

• --end commit-ish

  Most recent revision to test, as a commit-ish. If not specified, defaults to blead.

• --target target

  Makefile target (or equivalent) needed, to run the test case. If specified, this should be one of

  • config.sh

    Just run ./Configure

  • config.h

    Run the various *.SH files to generate Makefile, config.h, etc.

  • miniperl

    Build miniperl.

  • lib/Config.pm

    Use miniperl to build lib/Config.pm

  • Fcntl

    Build lib/auto/Fcntl/Fnctl.so (strictly, .$Config{so}). As Fcntl is simple XS module present since 5.000, this provides a fast test of whether XS modules can be built. Note, XS modules are built by miniperl, hence this target will not build perl.

  • perl

    Build perl. This also builds pure-Perl modules in cpan, dist and ext. XS modules (such as Fcntl) are not built.

  • test_prep

    Build everything needed to run the tests. This is the default if we're running test code, but is time consuming, as it means building all XS modules. For older Makefiles, the previous name of test-prep is automatically substituted. For very old Makefiles, make test is run, as there is no target provided to just get things ready, and for 5.004 and earlier the tests run very quickly.

• --one-liner 'code to run'

• -e 'code to run'

  Example code to run, just like you'd use with perl -e.

  This prepends ./perl -Ilib -e 'code to run' to the test case given, or ./miniperl if target is miniperl.

  (Usually you'll use -e instead of providing a test case in the non-option arguments to bisect.pl)

  -E intentionally isn't supported, as it's an error in 5.8.0 and earlier, which interferes with detecting errors in the example code itself.

• -c

  Add -c to the command line, to cause perl to exit after syntax checking.

• -l

  Add -l to the command line with -e

  This will automatically append a newline to every output line of your testcase. Note that you can't specify an argument to perl's -l with this, as it's not feasible to emulate perl's somewhat quirky switch parsing with Getopt::Long. If you need the full flexibility of -l, you need to write a full test case, instead of using bisect.pl's -e shortcut.

• -w

  Add -w to the command line with -e

  It's not valid to pass -c, -l or -w to bisect.pl unless you are also using -e

• --expect-fail

  The test case should fail for the start revision, and pass for the end revision. The bisect run will find the first commit where it passes.

• -D config_arg=value

• -U config_arg

• -A config_arg=value

  Arguments (-A, -D, -U) to pass to Configure. For example,

      -Dnoextensions=Encode
      -Uusedevel
      -Accflags=-DNO_MATHOMS

  Repeated -A arguments are passed through as is. -D and -U are processed in order, and override previous settings for the same parameter. bisect-runner.pl emulates -Dnoextensions when Configure itself does not provide it, as it's often very useful to be able to disable some XS extensions.

• --make make-prog

  The make command to use. If this not set, make is used. If this is set, it also adds a -Dmake=... else some recursive make invocations in extensions may fail. Typically one would use this as --make gmake to use gmake in place of the system make.

• --jobs jobs

• -j jobs

  Number of make jobs to run in parallel. If /proc/cpuinfo exists and can be parsed, or /sbin/sysctl exists and reports hw.ncpu, or /usr/bin/getconf exists and reports _NPROCESSORS_ONLN defaults to 1 + number of CPUs. Otherwise defaults to 2.

• --match pattern

• --no-match pattern

  Instead of running a test program to determine pass or fail, --match will pass if the given regex matches, and hence search for the commit that removes the last matching file. --no-match inverts the test, to search for the first commit that adds files that match.

  The remaining command line arguments are treated as glob patterns for files to match against. If none are specified, then they default as follows:

  • If no target is specified, the match is against all files in the repository (which is fast).

  • If a target is specified, that target is built, and the match is against only the built files.

  Treating the command line arguments as glob patterns should not cause problems, as the perl distribution has never shipped or built files with names that contain characters which are globbing metacharacters.

  Anything which is not a readable file is ignored, instead of generating an error. (If you want an error, run grep or ack as a test case). This permits one to easily search in a file that changed its name. For example:

      .../Porting/bisect.pl --match 'Pod.*Functions' 'pod/buildtoc*'

  --no-match ... is implemented as --expect-fail --match ...

• --test-build

  Test that the build completes, without running any test case.

  By default, if the build for the desired target fails to complete, bisect-runner.pl reports a skip back to git bisect, the assumption being that one wants to find a commit which changed state "builds && passes" to "builds && fails". If instead one is interested in which commit broke the build (possibly for particular Configure options), use --test-build to treat a build failure as a failure, not a "skip".

  Often this option isn't as useful as it first seems, because any build failure will be reported to git bisect as a failure, not just the failure that you're interested in. Generally, to debug a particular problem, it's more useful to use a target that builds properly at the point of interest, and then a test case that runs make. For example:

      .../Porting/bisect.pl --start=perl-5.000 --end=perl-5.002 \
          --expect-fail --force-manifest --target=miniperl make perl

  will find the first revision capable of building DynaLoader and then perl, without becoming confused by revisions where miniperl won't even link.

• --force-manifest

  By default, a build will "skip" if any files listed in MANIFEST are not present. Usually this is useful, as it avoids false-failures. However, there are some long ranges of commits where listed files are missing, which can cause a bisect to abort because all that remain are skipped revisions.

  In these cases, particularly if the test case uses miniperl and no modules, it may be more useful to force the build to continue, even if files MANIFEST are missing.

• --force-regen

  Run make regen_headers before building miniperl. This may fix a build that otherwise would skip because the generated headers at that revision are stale. It's not the default because it conceals this error in the true state of such revisions.

• --expect-pass [0|1]

  --expect-pass=0 is equivalent to --expect-fail. 1 is the default.

• --no-clean

  Tell bisect-runner.pl not to clean up after the build. This allows one to use bisect-runner.pl to build the current particular perl revision for interactive testing, or for debugging bisect-runner.pl.

  Passing this to bisect.pl will likely cause the bisect to fail badly.

• --validate

  Test that all stable revisions can be built. By default, attempts to build blead, v5.14.0 .. perl-5.002 (or perl5.005 on a case insensitive file system). Stops at the first failure, without cleaning the checkout. Use --start to specify the earliest revision to test, --end to specify the most recent. Useful for validating a new OS/CPU/compiler combination. For example

      ../perl/Porting/bisect.pl --validate -le 'print "Hello from $]"'

  If no testcase is specified, the default is to use t/TEST to run t/base/*.t

• --check-args

  Validate the options and arguments, and exit silently if they are valid.

• --check-shebang

  Validate that the test case isn't an executable file with a #!/usr/bin/perl line (or similar). As bisect-runner.pl does not prepend ./perl to the test case, a #! line specifying an external perl binary will cause the test case to always run with that perl, not the perl built by the bisect runner. Likely this is not what you wanted. If your test case is actually a wrapper script to run other commands, you should run it with an explicit interpreter, to be clear. For example, instead of ../perl/Porting/bisect.pl ~/test/testcase.pl you'd run ../perl/Porting/bisect.pl /usr/bin/perl ~/test/testcase.pl

• --usage

• --help

• -?

  Display the usage information and exit.