NAME

docs/tests.pod - Testing Parrot

A basic guide to writing tests for Parrot

This is quick and dirty pointer to how tests for Parrot should be written. The testing system is liable to change in the future, but tests written following the guidelines below should be easy to port into a new test suite.

How to write a test

Assembler tests

First, find an appropriate file in t/op/*.t (for basic ops), t/pmc/*.t (for anything to do with PMCs), and t/src/*.t for C source code tests. If there isn't an appropriate file, create one.

Near the top of each file, you'll see a line like:

  use Parrot::Test tests => 8;

This sets up the test harness used to assemble and run the tests, and lets it know how many tests you plan to run. New tests should be added by:

1. Incrementing the number of planned tests.

2. Putting some code in like this:

    pasm_output_is(<<'CODE', <<'OUTPUT', "name for test");
        *** a big chunk of assembler, eg:
        print   1
        print   "\n" # you can even comment it if it's obscure
        end          # don't forget this...!
    CODE
    *** what you expect the output of the chunk to be, eg.
    1
    OUTPUT

Tests in t/op/ or t/pmc/ are considered to be PASM tests. Tests in imcc/t/ are assumed to be PIR tests. You can put PIR tests into t/op/ or t/pmc/ by using functions like 'pir_output_is'.

C source tests

    c_output_is(<<'CODE', <<'OUTPUT', "name for test");
    #include <stdio.h>
    #include "parrot/parrot.h"
    #include "parrot/embed.h"

    static opcode_t *the_test(Parrot_Interp, opcode_t *, opcode_t *);

    int main(int argc, char* argv[]) {
        Parrot_Interp interpreter;
        interpreter = Parrot_new(NULL);

        if (!interpreter)
        return 1;

        Parrot_init(interpreter);
        Parrot_run_native(interpreter, the_test);
        printf("done\n");
    fflush(stdout);
        return 0;
    }

    static opcode_t*
    the_test(Parrot_Interp interpreter,
        opcode_t *cur_op, opcode_t *start)
    {
        /* Your test goes here. */

        return NULL;  /* always return NULL */
    }
    CODE
    # Anything that might be output prior to "done".
    done
    OUTPUT

Note, it's always a good idea to output "done" to confirm that the compiled code executed completely. When mixing printf and PIO_printf always append a fflush(stdout); after the former.

Ideal tests:

o

Probe the boundaries (including edge cases, errors thrown etc.) of whatever code they're testing. These should include potentially out of band input unless we decide that compilers should check for this themselves.

o

Are small and self contained, so that if the tested feature breaks we can identify where and why quickly.

o

Are valid. Essentially, they should conform to the additional documentation that accompanies the feature (if any). [If there isn't any documentation, then feel free to add some and/or complain to the mailing list].

o

Are a chunk of assembler and a chunk of expected output.