/ 
App-hopen-0.000014-TRIAL
River stage zero No dependents
/ App::hopen::Conventions

NAME

App::hopen::Conventions - conventions for using hopen(1) as a build system

SYNOPSIS

Data::Hopen is a flexible dataflow processor and task runner. However, its main use case is as a build system (e.g., a Makefile generator). To keep the core of hopen as flexible as possible, App::hopen (build system) requires its components to follow the conventions described in this document.

These conventions are generally implemented/enforced in App::hopen and App::hopen::AppUtil.

Note: everything in this document is subject to change since App::hopen is currently under active development.

COMPONENTS USED BY THE BUILD SYSTEM

Project directory

Where the main hopen file file of the project is (see "HOPEN FILES").

Destination directory

Where the output will go. hopen(1) does not support in-source builds. The default directory is <project dir>/built.

Generator

A subclass of App::hopen::Gen that creates the build-system files. The generator's job is to arrange for coordinating work that needs to be done.

Toolset

A collection of commands (App::hopen::G::Cmd subclasses) that know how to process specific types of files. Toolsets are responsible for defining the work that the generator will coordinate.

Toolsets are technically independent of which generator is in use. However, a command-line toolset probably won't work with an XML-based generator! If no toolset is specified, the generator picks the default.

An architecture (optional)

An arbitrary string understood by the toolset or generator. Used, e.g., to select x86 vs. x64.

"Blueprint" files

These are the outputs of hopen. They are the inputs to make(1), ninja(1), or some other software-build tool.

HOPEN FILES

hopen configures a project by running one or more files matching .hopen.pl or *.hopen.pl. As the extension suggests, these are Perl source files.

The filename MY.hopen.pl is reserved. It is created by hopen in each destination directory to record the state of the build in that directory.

Hopen itself does not (knowingly) use any source filters, text replacement, pluggable keywords, or other fancy features. Everything in a hopen file is straight Perl.

Which hopen files are used

On any hopen(1) run, up to three hopen files are automatically located and executed. (Any of those can run any number of additional hopen files.) None of the three files has to exist. The three files are, in order of execution:

  • Build-state file

    The MY.hopen.pl in the destination directory. This sets the current phase (see "PHASES") and loads the data output by the last hopen(1) run.

    After MY.hopen.pl runs, the generator and toolset are loaded.

  • Project file

    The last-sorting .hopen.pl or *.hopen.pl file in the project directory. You can name your project file whatever you want --- only the extension has to match. That way you can call your build file .hopen.pl if you want it hidden, or z.hopen.pl if you want it to sort below all your other files. Sort order is Perl's default, which is by byte value.

  • Context file

    Sometimes you need to tweak the build of someone else's project to make it fit your environment. I run into this all the time on Cygwin. Therefore, hopen will look for a hopen file in the project's parent directory. That file is referred to as a "context file". Since the context file runs after the project file, the context file can change the way the project will be built.

    The filename of the context file is directory_name.hopen.pl for a project in directory_name. It doesn't matter what the project calls itself; the context file is found solely based on the directory name.

Note: if the project file or the context file has a newer modification time than the build-state file, the build-state file will not be loaded.

After those files are executed, any code provided by a -e command-line option is executed as if it were its own hopen file on disk, and any hopen file referenced by a -f option is executed. -e and -f options are processed in the order given on the command line.

The hopen files are run in order to build up a hashref that will be the input to the build graph. The hopen files, while they run, also populate the build graph.

Execution environment of a hopen file

Each hopen file is the body of a subroutine that receives a hashref from the last-run hopen file and may return a hashref ({...}) or undef. Each hopen file's output hashref, if any, is appended to the input hashref (using Hash::Merge with retainment precedent). Therefore, to leave the input unchanged, return {}, not $_[0].

Any return value that is not a hashref will be ignored.

Elements of the hashref starting with __R are reserved. Please don't read or write those elements if you want hopen to work! :) Similarly, all variables starting with __R are reserved.

The hopen file is executed in scalar context.

Each hopen file runs in an environment with everything from the following packages loaded (via App::hopen::HopenFileKit):

Variables usable in hopen files

These are defined in App::hopen::BuildSystemGlobals.

$Generator

The current App::hopen::Gen instance.

$Toolset

The name of the current App::hopen::T::<stem> package root.

$Build

The Data::Hopen::G::DAG instance representing the current build. Goals in $Build will become, e.g., top-level targets of a generated Makefile.

$Phase

The current phase (read-only). Only MY.hopen.pl is allowed to change the phase (and not even that if --phase is specified on the hopen command line).

PHASES

hopen is a multi-phase build system. Unlike cmake(1), hopen runs deterministically in multiple steps. That way you don't have to run the build generator over and over again until it converges. (Yes, I have had this problem with cmake.) Each time you run hopen(1), it will run the next phase (saved in MY.hopen.pl). Currently, the phases are as follows, in order.

Check

During this phase, the generator, toolset, or hopen file can collect information about the environment.

Gen

During this phase, blueprint files and any supporting files (e.g., a config.h) are written to the destination directory.

After all the phases are run

Once all of hopen's phases have been run, you can run make(1), ninja(1), or whatever build system corresponds with the generator you have selected. You can do this by running hopen --build, if you wish.

INTERNALS

Overall data flow

  1. The hopen files run, and generate a command graph. The command graph includes nodes that implement App::hopen::G::Cmd. The idea is that the hopen files specify the commands you would run to build your software by hand.

    Each command node, when run, outputs a made arrayref of App::hopen::Asset instances.

  2. The command graph is run, with a generator made available to it. Command nodes provide asset nodes to the generator and to later command nodes. The asset nodes implement App::hopen::G::AssetNode. The specific subclass they implement is determined by the generator.

  3. The generator runs the asset graph to output the blueprint files.

SCRAPS

Ignore these --- I may come back to them in the future.

Tool

App::hopen::Tool - Base class for packages that know how to process files

A tool knows how to generate a command or other text that will cause a build system to perform a particular action on a file belonging to a particular language.

A tool is an App::hopen::G::Cmd, so may interact with the current generator ("$Generator" in App::hopen::BuildSystemGlobals). Moreover, the generator will get a chance to visit the op after it is processed.

Maybe TODO:

- Each Generator must specify a list of Content-Types (media types)
    it can consume.  Each Tool must specify a specific content-type
    it produces.  Mismatches are an error unless overriden on the
    hopen command line.

Toolchain

TODO?

- permit loading any number of tools/toolchains
- The Generator specifies a default list of tools rather than a
    single default toolchain.
- add G::ToolOp to invoke tools.  A ToolOp will take a language
    and an opcode and invoke a corresponding Tool.  E.g., a GNU C
    Tool will generate command-line options for gcc-style command lines.
- Only one Tool may be loaded for each (language, opcode) pair.
    Otherwise the build would be ambiguous.
- In Conventions, define the formats for languages and opcodes.

Maybe TODO:

- Each Generator must specify a list of Content-Types (media types)
    it can consume.  Each Tool must specify a specific content-type
    it produces.  Mismatches are an error unless overriden on the
    hopen command line.

Phases

App::hopen::Phase - Namespace for hopen phases.

The specific phases in hopen live under App::hopen::Phase. The current phases are:

App::hopen::Phase::Probe

Read a foundations file and one or more context files; output a capability file and an options file. The user may edit the options file.

App::hopen::Phase::Gen

Read the capability, option, and context files, plus a recipes file identifying the build graph. Output one or more blueprint files.

Build

Not part of hopen --- run the blueprint files through make(1), ninja(1), or another build tool.

If the dependencies change as the result of a build, the Gen phase is re-run. However, the Probe phase does not have to be re-run.

PHILOSOPHY

Hopen is a single-pass build system. You do not have to re-run to converge on a final set of blueprint files.

Hopen is context-aware. You can change the build of a repo without having to touch files in that repo.