Philip Crow
and 1 contributors


Bigtop::Parser - the Parse::RecDescent grammar driven parser for bigtop files


Make a file like this:

    config {
        base_dir `/home/username`;
        Type1 Backend {}
        Type2 Backend {}
        Type3 Backend {}
    app App::Name {
        table name { }
        controller SomeController {}

Then run this command:

    bigtop my.bigtop all


This module is really only designed to be used by the bigtop and tentmaker scripts. It provides access to the grammar which understands bigtop files and turns them into syntax trees. It provides various utility functions for bigtop, tentmaker, backends, and similar tools you might write.

If you just want to use bigtop, you should look in Bigtop::Docs::TOC where all the docs are outlined.

Reading further is an indication that you are interested in working on Bigtop and not just in using it to serve your needs.


In this section, the methods are grouped, so that similar ones appear together.

METHODS which drive generation for scripts


The bigtop script calls this method.

Returns: the app name and the name of the build directory.

You can call this as a class method passing it the name of the bigtop file to read and a list of the things to build.

The method is actually quite simple. It merely reads the file, then calls gen_from_string.


The bigtop script calls this method when --new is used.

Returns: the app name and the name of the build directory.

This method orchestrates the build. It is called internally by gen_from_file. Call it as a class method. Pass it a hash with these keys:

    bigtop_string => the bigtop source code
    bigtop_file   => the name of the bigtop file, if you know it
    create        => are you in create mode? if so make this true
    build_list    => [ what to build ]
    flags         => command line args given to your script

bigtop_file is used by Bigtop::Init::Std to copy the bigtop file from its original location into the docs subdirectory of the build directory. If the file name is not defined, it skips that step.

If you set create to any true value, you will be in create mode and bigtop will make the build directory as a subdirectory of the current directory. Otherwise, it will make sure you are in a directory which looks like a build directory before building.

The list of things to build can include any backend type listed in the config block and/or the word 'all'. 'all' will be replaced with a list of all the backend types in the config section (in the order they appear there), as if they had been passed in.

It is legal to mention the same backend more than once. For instance, you could call gen_from_string directly

            bigtop_string => $bigtop_string,
            bigtop_file   => 'file.bigtop',
            create        => $create,
            build_list    => [ 'Init', 'Control', 'Init' ]

or equivalently, and more typically, you could call gen_from_file:

        'file.bigtop', $create, 'Init', 'Control', 'Init'

Either of these might be useful, if the first Init sets up directories that the Control backend needs, but the generated output from Control should influence the contents of file which Init finally builds. Check your backends for details.

The flags are given to Init Std as text, so they may be preserved for posterity in the Changes file.

gen_from_string internals

gen_from_string works like this. First, it attempts to parse the config section of the bigtop string. If that works, it iterates through each backend mentioned there building a list of modules to require. This includes looking in backend blocks for template statements. Their values must be template files relative to the directory from which bigtop was invoked.

Once the list is built, it calls its own import method to require them. This allows each backend to register its keywords. If any keyword used in the app section is not registered, a fatal parse error results.

Once the backends are all required, gen_from_string parses the whole bigtop string into an abstract syntax tree (AST). Then it iterates through the build list calling gen_Type on each element's backend. So this:

    config {
        Init Std      {}
        SQL  Postgres { template ``; }
    app ...

            $bigtop_string, 'file.bigtop', 'Init', 'SQL'

Results first in the loading of Bigtop::Init::Std and Bigtop::SQL::Postgres, then in calling gen_Init on Init::Std and gen_SQL on SQL::Postgres. During the loading, setup_template is called with on SQL::Postgres.

gen_* methods are called as class methods. They receive the build directory, the AST, and the name of the bigtop_file (which could be undef). Backends can do whatever they like from there. Typically, they put files onto the disk. Those files might be web server conf files, sql to build the database, control modules, templates for viewing, models, etc.

METHODS which invoke the grammar


Called as a class method (usually by gen_from_string), this method receives the bigtop input as a string. It attempts to parse only the config section which it returns as an AST. Syntax errors in the config section are fatal. Errors in the app section are not noticed.


Call this as a class method, passing it the file name to read. It reads the file into memory, then calls parse_string, returning whatever it returns.


Call this as a class method, passing it the bigtop string to parse. It calls the grammar to turn the input into an AST, which it returns.


The grammar of a bigtop file is structured, but the legal keywords in its simple statements are defined by the backends (excepts that the config keywords are defined by this module, see Config Keywords below for those).

Acutally, all the keywords that any module will use should be defined in Bigtop::Keywords so tentmaker can display them. Then the backend (or its type) should pull the keyword definitions it wants from Bigtop::Keywords.

If you are writing a backend, you should use the base module for your backend type. This will register the standard keywords for that type. For example, suppose you are writing Bigtop::Backend::SQL::neWdB. It should be enough to say:

    use Bigtop::SQL;

in your module.

If you need to add additional keywords that are specific to your backend, put them in a begin block like this:

    BEGIN {
                qw( your keywords here),

Here $type is the name of the surrounding block in which this keyword will make a valid statement. For example, if $type above is 'app' then this would be legal:

    app App::Name {
        your value;

The type must be one of these levels:


These correspond to the block types in the grammar. Note, that there are also sequence blocks, but they are deprecated and never allowed statements. Further, the various literals are blocks in the grammar (they have block idents and can have defined keywords), but they don't have brace delimiters. Instead, they have a single backquoted string.


Call this as a class method, passing it a type of keyword and a word that might be a valid keyword of that type.

Returns true if the keyword is valid, false otherwise.


Call this as a class method passing it the type of keywords you want.

Returns a list of all registered keywords, of the requested type, in string sorted order.

The two preceding methogs are really for internal use in the grammar.

METHODS which work on the AST

There are quite a few other methods not documented here (shame on me). Most of those support tentmaker manipulations of the tree, but there are also some convenience accessors.


Walks the AST for you, calling you back when it's time to build something.

The most common skeleton for gen_Backend is:

    use Bigtop;
    use Bigtop::Backend;

    sub gen_Backend {
        my $class     = shift;
        my $build_dir = shift;
        my $tree      = shift;

        # walk the tree
        my $something     = $tree->walk_postoder( 'output_something' );
        my $something_str = join '', @{ $something };

        # write the file
        Bigtop::write_file( $build_dir, $something_string );

This walks the tree from the root. The walking is postorder meaning that all children are visited before the current node. Each walk_postorder returns an array reference (which is why we have to join the result in the above skeleton). After the children have been visited, the callback (output_something in the example) is called with their output array reference. You can also pass an additional scalar (which is usually a hash reference) to walk_postorder. It will be passed along to all the child walk_postorders and to the callbacks.

With this module walking the tree, all you must do is provide the appropriate callbacks. Put one at each level of the tree that interests you.

For example, if you are generating SQL, you need to put callbacks in at least the following packages:


This does require some knowledge of the tree. Please consult bigtop.grammar, in the lib/Bigtop subdirectory of Bigtop's build directory, for the possible packages (or grep for package on this file). There are also several chapters of the Gantry book devoted to explaining how to use the AST to build backends.

The callbacks are called as methods on the current tree node. They receive the output array reference from their children and the data scalar that was passed to walk_postorder (if one was passed in the top level call). So, a typical callback method might look like this:

    sub output_something {
        my $self         = shift;
        my $child_output = shift;
        my $data         = shift;
        return [ $output ];

Remember that they must return an array reference. If you need something fancy, you might do this:

    return [ [ type1_output => $toutput, type2_output => $other_out ] ];

Then the parent package's callback will receive that and must tease apart the the two types. Note that I have nested arrays here. This prevents two children from overwriting each other's output if you are ever tempted to try saving the return list directly to a hash (think recursion).

(walk_postorder also passes the current node to each child after the data scalar. This is the child's parent, which is really only useful during parent building inside the grammar. The parent comes after the data scalar in both walk_postorder and in the callback. Most backends will just peek in $self->{__PARENT__} which is gauranteed to have the parent once the grammar finishes with the AST.)


This method is the callback used by the grammar to make sure that all nodes know who their daddy is. You shouldn't call it, but looking at it shows what the simplest callback might look like. Note that there is only one of these and it lives in the application_ancestor package, which is not one of the packages defined in the grammar. But, this module makes sure that all the grammar defined packages inherit from it.


This method builds the lookup hash you can use to find data about other parts of the tree, without walking to it.

The AST actually has three keys: configuration, application, and lookup. The first two are built in the normal way from the input file. They are genuine ASTs in their own right. The lookup key is not. It does not preserve order. But it does make it easier to retrieve things.

For example, suppose that you are in the method_body package attempting to verify that requested fields for this method are defined in the table for this controller. You could walk the tree, but the lookup hash makes it easier:

    unless (
        defined $tree->{lookup}{tables}{$table_name}{fields}{$field_name}
    ) {
        die "No such column $field_name\n";

The easiest way to know what is available is to dump the lookup hash. But the pattern is basically this. At the top level there are fixed keywords for the app level block types: tables, sequences, controllers. The next level is the name of a block. Under that, there is a fixed keyword for each subblock type, etc.

METHODS for use in walk_postorder callbacks


Use this method instead of directly calling Data::Dumper::Dump.

While you could dump $self, that's rather messy. The problem is the parent nodes. Their presence means a simple dump will always show the whole app AST. This method carefully removes the parent, dumps the node, and restores the parent, reducing clutter and leaving everything in tact. The closer to a leaf you get, the better it works.


Call this on the full AST. It returns the name of the application.


Call this on the full AST. It returns the config subtree.


Call this, from the method_body package, on the AST node ($self in the callback). Returns the name of the controller for this method. This is useful for error reporting.


Call this, from the method_body package, on the AST node ($self in the callback). Returns the name of this method. Useful for error reporting.


While this should work everywhere, it doesn't. Some packages have it. If yours does, call it. Otherwise peek in $self->{__NAME__}. But, remember that not everything has a name.


Call this, from the method_body package, on the AST node ($self in the callback). Returns the name of the table this controller controls. Useful for error reporting.

METHODS used internally


You probably don't need to call this. But, if you do, pass it a list of backends to import like this:

    use Bigtop::Parser qw( );

This will load Bigtop::Type::Backend and tell it to use You can accomplish the same thing by directly calling import as a class method:

    Bigtop::Parser->import( '' );

This method is used by the grammar to report fatal parse error in the input. It actually gives 50 characters of trailing context, not two lines, but the name stuck.


This method is used by the grammer to report on unregistered (often misspelled) keywords. It identifies the offending keyword and the line where it appeared in the input, gives the remainder of the line on which it was seen (which is sometimes only whitespace), and lists the legal choices (often wrapping them in an ugly fashion).


For simplicity, all config keywords are requested from Bigtop::Keywords in this module. This is not necessarily ideal and is subject to change.


Used only if you supply the --create flag to bigtop (or set create to true when calling gen_from_file or gen_from_string as class methods of this module).

When in create mode, the build directory will be made as a subdirectory of the base_dir. For instance, I could use my home directory:

    base_dir `/home/username`;

Note that you need the backquotes to hide the slashes. Also note, that you should use a path which looks good on your development system. In particular, this would work on the appropriate platform:

    base_dir `C:\path\to\build`;

The default base_dir is the current directory from which bigtop is run.


Used only if you supply the --create flag to bigtop (or set create to true when calling gen_from_file or gen_from_string as class methods of this module).

When in create mode, the actual generated files will be placed into base_dir/app_dir (where the slash is correctly replaced with your OS path separator). If you are in create mode, but don't supply an app_dir, a default is formed from the app name in the manner h2xs would use. Consider:

    config {
        base_dir `/home/username`;
    app App::Name {

In this case the app_dir is App-Name. So the build directory is


By specifying your own app_dir statement, you have complete control of where the app is initially built. For example:

    config {
        base_dir `/home/username`;
        app_dir  `myappdir`;
    app App::Name { }

Will build in /home/username/myappdir.

When not using create mode, all files will be built under the current directory. If that directory doesn't look like an app build directory, a fatal error will result. Either move to the proper directory, or use create mode to avoid the error.


This is passed directly to the use Framework; statement of the top level controller.


    engine MP13;

becomes something like this:

    use Framework qw/ engine=MP13 /;

in the base level controller. Both Catalyst and Gantry expect this syntax.

The available engines depend on what the framework supports. The one in the example is mod_perl 1.3 in the syntax of Catalyst and Gantry.


Similar to engine, this specifies the template engine. Choices almost always include TT, but might also include Mason or other templaters depending on what your framework supports..



This keyword applies to many backends at the app level and at some other levels. This keyword is special, because it expects a type keyword immediately before its values. For example:

    literal SQL `CREATE...`;

It always instructs someone (the backend of type SQL in the example) to directly insert the backquoted string into its output, without so much as adjusting whitespace.

Backend types that should obey this statement are:

    SQL      - for backends of type SQL
    Location - for backends constructing apache confs or the like

The literal Location statement may also be used at the controller level.


Applies to backend blocks in the config block, app blocks, controller blocks, and method blocks.

gen_from_string enforces the app level no_gen. If it has a true value only a warning is printed, nothing is generated. None of the backends are called.

gen_from_string also enforces no_gen on entire backends, if their config block has a true no_gen value.

The Control backend of your choice is responsible for enforcing no_gen at the controller and method levels.


Applies to tables and fields (although the latter only worked for Models at the time of this writing).

Each backend is responsible for enforcing not_for. It should mean that the field or table is ignored by the named backend type. Thus

    table skip_model {
        not_for Model;

should generate as normal in SQL backends, but should be completely ignored for Models. The same should hold for fields marked not_for. But my SQL backends didn't do that when I wrote this, only the Models worked.



Called by TentMaker, so it can display the backend comments to the user through their browser.

Returns: a hash reference of keyword docs understood by tentmaker's templates.


Used internally.

Get accessor for whether we are really generating, or just serving tentmaker. If we are not generating, there is no need to set up the templates for all the backends.


Used internally.

Set accessor for whether we are really generating.


Returns: the next available ident (as ident_n).


Used internally.

Accessor to ensure that only one parser is ever instantiated.


Used internally.

Responsible for loading all needed backends.


Used internally.

Strips comment lines.

Returns: a hash keyed by line number, storing the comment on that line before it was stripped..


Phil Crow <>


Copyright (C) 2005-7 by Phil Crow

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.8.6 or, at your option, any later version of Perl 5 you may have available.