Marpa::R2::Scanless::DSL - The DSL for the Scanless interface
use Marpa::R2; my $grammar = Marpa::R2::Scanless::G->new( { bless_package => 'My_Nodes', source => \(<<'END_OF_SOURCE'), :default ::= action => [values] bless => ::lhs lexeme default = action => [range,value] bless => ::name :start ::= Script Script ::= Expression+ separator => comma comma ~ [,] Expression ::= Number bless => primary | '(' Expression ')' bless => paren assoc => group || Expression '**' Expression bless => exponentiate assoc => right || Expression '*' Expression bless => multiply | Expression '/' Expression bless => divide || Expression '+' Expression bless => add | Expression '-' Expression bless => subtract Number ~ [\d]+ :discard ~ whitespace whitespace ~ [\s]+ # allow comments :discard ~ <hash comment> <hash comment> ~ <terminated hash comment> | <unterminated final hash comment> <terminated hash comment> ~ '#' <hash comment body> <vertical space char> <unterminated final hash comment> ~ '#' <hash comment body> <hash comment body> ~ <hash comment char>* <vertical space char> ~ [\x{A}\x{B}\x{C}\x{D}\x{2028}\x{2029}] <hash comment char> ~ [^\x{A}\x{B}\x{C}\x{D}\x{2028}\x{2029}] END_OF_SOURCE } );
This page is the reference for domain-specific language at the center of Marpa's Scanless interface (SLIF). The format of Scanless domain-specific language (DSL) is that of BNF source strings, with changes and extensions as described in this document.
In reading this document, it is important to keep in mind the distinction, on one hand, between G0 and G1 rules and, on the other hand, between rules and lexemes. G1 rules have a semantics, which can be specified as described in this document. G0 rules simply recognize symbols in the input. G0 rules do not have a semantics.
Top-level G0 rules correspond to a string in the input, but when it is a value, this string is actually seen as the value of a lexeme. The G0 grammar can be thought of as similar in behavior to a set of regular expressions with the lexemes being seen as similar to named captures.
Lexemes are the symbols which form the interface between G1 and G0. They also have a semantics, which can be specified as described below. The semantics of lexemes is completely separate from that of rules.
The BNF operator ("::=") has the same function as in the Stuifzand (BNF) interface, but in the SLIF it plays an additional role; It indicates that the rule is a G1 rule. Start rules must be G1 rules.
::=
The match operator ("~") can be used between the LHS and RHS of a rule, instead of the BNF operator. Rules which use the match operator are G0 rules. Discard rules must be G0 rules. A lexical rule cannot have an action adverb or a bless adverb.
~
action
bless
:default ::= action => [values] bless => ::lhs
The default pseudo-rule changes the defaults for rule adverbs. Default pseudo-rules do not affect the defaults for G0 rules or for lexemes. There may be more than one default pseudo-rule. The scope of default pseudo-rules is lexical, applying only to rules that appear afterwards in the DSL source.
Currently only the defaults for the action and bless adverbs can be changed with a default pseudo-rule. Each default pseudo-rule creates a completely new set of defaults -- if an adverb is not specified, it is reset to its original default, the one it had before any default pseudo-rules were encountered.
lexeme default = action => [range,value] bless => ::name
The lexeme default statement changes the defaults for lexeme adverbs. It only changes the defaults for lexemes, and does not affect rules.
The lexeme default statement is a "statement", instead of a "rule". Rules are signified by the BNF operator ("::=") or the match operator ("~"), and their effect depends on their lexical position. Statements are signified by the assignment operator ("="), and their effect is global. The effect of a statement does not depend on where in the DSL source file it occurs.
=
At this writing the lexeme default statement is only way to specify the semantics of lexemes. Only one lexeme default statement is allowed in a file.
The action adverb specifies the semantics for a G1 rule or a lexeme, as described below. The default value of a G1 rule is a Perl undef. The action adverb is not allowed for G0 rules, which have no default value.
undef
The default value of a lexeme is its literal value in the input stream, as a string. This is called the token value of the lexeme.
If the action value is a Perl word (a string of alphanumerics or underscores), it will be interpreted as the action name, as described for the action named argument of rule descriptors. Word actions are not allowed for lexemes.
Ordinarily, the arguments to the semantic closure are the per-parse-tree variable, followed by values of the rule's child nodes, in the order in which they occurred in the input stream. However, if a rule with a word action is blessed, in order to allow the blessing to take effect, the arguments to a semantics closure are formed differently. If the rule is blessed, its semantic closure will always have exactly two arguments. The first will be the per-parse-tree variable. The second will be a blessed array that contains the child values in input-stream order.
If the action value begins with a double colon ("::"), it is a reserved action. The following are recognized:
::
::array
The value of the rule or lexeme is an array. For a rule, the array will contain the values of each of its children, in left-to-right order. If the rule has no children, the array will be empty. For a lexeme, the array will be of length one, and will contain the token value.
::first
The value of the rule is that of the rule's first child. If there is no such child, the value is a Perl undef.
It is a fatal error if a blessing is applied to a rule with a ::first action. This is the case even when either or both adverbs are the result of a default. It is also a fatal error to use ::first action with a lexeme.
::undef
The value of the rule or lexeme is a Perl undef. It is a fatal error if a blessing is applied to a rule with a ::undef action. This is the case even when either or both adverbs are the result of a default.
An array descriptor is a comma separated list of zero or more array descriptor items, inside a pair of matching square brackets. The value of the lexeme or rule will be an array. The contents of the array will be a series of lists, as specified by the array descriptor items. These lists will occur in the array in the order specified. If no array descriptor items are specified, the value of the lexeme or rule will be an empty array.
Currently there are only two array descriptor items. The range array descriptor item will put a list into the array that indicates the location in the input stream of the rule or lexeme. The list will contain two elements. The first element will be the start position, and the second will be the end position.
range
For a rule, the values array descriptor item indicates a list containing the values of the rule's children, in left-to-right order. For a lexeme, the value array descriptor item indicates a list containing a single element, the token value of the lexeme.
values
value
::array is equivalent to [values]. The value and values array descriptor items are synonyms, and may be used interchangeably for both rules and lexemes. This means that, for both lexemes and rules, the actions [values], [value] and ::array will do exactly the same thing.
[values]
[value]
The bless adverb causes the result of the semantics to be blessed into the class indicated by the value of the adverb. The value of a bless adverb is called a blessing. If the blessing is a Perl word (a string of alphanumerics or underscores), the name of the class will be formed by prepending the value of the bless_package named argument, followed by a double colon ("::").
bless_package
If the blessing begins with a double colon ("::"), it is a reserved blessing. A blessing of ::undef means that the rule or lexeme will not be blessed. By default, both rules and lexemes are not blessed. If any rule or lexeme of a SLIF grammar has a blessing other than ::undef, a bless_package is required, and failure to specify one results in a fatal error.
The bless adverb for rules allows one additional reserved blessing: ::lhs. A blessing of ::lhs causes the result to be blessed into a class whose name is based on the LHS of the rule. The class will be the name of the LHS with whitespace changed to an underscore. (As a reminder, the whitespace in symbol names will have been normalized, with leading and trailing whitespace removed, and all other whitespace sequences changed to a single ASCII space.) When a ::lhs blessing value applies to a rule, it is a fatal error if the LHS contains anything other than alphanumerics and whitespace. In particular, the LHS cannot already contain an underscore ("_"). The ::lhs blessing is most useful in a default rule.
::lhs
_
The bless adverb for lexemes allows only one reserved value: ::name. A reserved value of ::name causes the value of the lexeme to be blessed into a class whose name is based on the name of the lexeme. The class is derived from the symbol name in the same way, and subject to the same restrictions, as described above for deriving a class name from the LHS of a rule. The ::name reserved blessing is most useful in the lexeme default rule.
::name
:discard ~ whitespace
A discard rule is a rule whose LHS is the :discard pseudo-symbol, and whose RHS is a single symbol name, called the discarded symbol. These rules indicate that the discarded symbol is a top-level G0 symbol, but one which is not a lexeme. When a discarded symbol is recognized, it is not passed as a lexeme to the G1 parser, but is (as the name suggests) discarded.
:discard
Expression ::= Number bless => primary | '(' Expression ')' bless => paren assoc => group || Expression '**' Expression bless => exponentiate assoc => right || Expression '*' Expression bless => multiply | Expression '/' Expression bless => divide || Expression '+' Expression bless => add | Expression '-' Expression bless => subtract
Single quotes can be used in prioritized rules to indicate character strings. The characters inside the single quote will be matched in the input, literally and one-for-one. Single quoted strings can contain any characters with the exception of single quotes and vertical whitespace.
Single quoted strings do not allow "escaped" characters. A backslash ("\") represents itself and has no effect on the interpretation of the next character. If a rule needs to match one of the forbidden characters (single quote or vertical whitespace), it must use a character class.
\
Single quoted strings are always interpreted at the G0 level, but they may be used in either structural or lexical rules. When a single quoted string is used in a structural rule, Marpa creates a virtual G0 rule on behalf of the application. This is handy, but it does have a real disadvantage -- the name of the virtual rule's LHS will be one assigned automatically by Marpa. When tracing and debugging parses and grammars, these virtual LHS's can be harder for a programmer to interpret.
<vertical space char> ~ [\x{A}\x{B}\x{C}\x{D}\x{2028}\x{2029}]
A character class in square brackets ("[]") can be used in a RHS alternative of a prioritized rule, or on the RHS of a quantified rule or a discard rule. Marpa character classes may contain anything acceptable to Perl, and follow the same escaping conventions as Perl's character classes.
[]
Character classes are always interpreted at the G0 level, but they may be used in either structural or lexical rules. When a character class is used in a structural rule, Marpa creates a virtual G0 rule on behalf of the application. This is handy, but it does have a real disadvantage -- the name of the virtual rule's LHS will be one assigned automatically by Marpa. When tracing and debugging parses and grammars, these virtual LHS's can be harder for a programmer to interpret.
An implementation note: character classes are interpreted by Perl, but this involves minimal overhead when the parse is of any length. Each character class is passed to Perl to interpret exactly once and the result is memoized in a C language structure for future use.
Copyright 2013 Jeffrey Kegler This file is part of Marpa::R2. Marpa::R2 is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. Marpa::R2 is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details. You should have received a copy of the GNU Lesser General Public License along with Marpa::R2. If not, see http://www.gnu.org/licenses/.
To install Marpa::R2, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Marpa::R2
CPAN shell
perl -MCPAN -e shell install Marpa::R2
For more information on module installation, please visit the detailed CPAN module installation guide.