++ed by:

2 PAUSE users

Jeffrey Kegler
and 1 contributors


Marpa::R3::Changes - Differences between Marpa::R2 and Marpa::R3

About this document

This document describes the incompatible differences between Marpa::R2 and Marpa::R3. (Differences that do not give rise to incompatibility are outside of its scope.) It is intended for readers already familiar with Marpa::R2, who are writing new applications for Marpa::R3, and for readers migrating Marpa::XS applications and tools to Marpa::R3.


Multiple slr->read() calls allowed

The slr->read() method may now called multiple times during a parse. each time with a new string. These strings will be called input blocks. The <input block> read by the n'th call of the slr->read() method during a parse has block index n.

This change makes the previous L0 positions no longer adequately identify a location, which necessitates many changes.

New parameters: The $slr-input_length() >> and $slr-literal() >> calls allow a new, optional parameter: the block index.

Deleted calls: The $slr-g1_location_to_span() >> and $slr-last_completed_span() >> methods has been removed.

Marpa strings must be UTF-8

Marpa expects all strings passed to it to be valid UTF-8. (Note that all ASCII-7 string are valid UTF-8).

"Eager" lexemes added

See "eager" in Marpa::R3::Scanless::DSL.

Problems with kernel Earley items in progress reports fixed

Progress reports had been misreporting some Earley items. The misreported items were certain kernel Earley items instances containing one or more proper nullables. By kernel Earley item, I mean an Earley item where the dot is in the middle -- not at the beginning and and not at the end. In other words, kernel Earley items are items which are not predictions, and which are not completions.

This bug is actually quite obscure -- almost all interest in progress reports is in completed Earley items, which were not affected. So obscure, in fact, was this bug that it went unnoticed in use. It surfaced only when I reread the code and realized that some corner cases were not being dealt with correctly. This bugs is now fixed.

Semantics moved from recognizer to grammar

In Marpa::R2, the semantics was not finally settled until the $recce->value() call. In Marpa::R3, semantics will be fully settled in the grammar.

Options moved from recognizer to grammar

The exhaustion, ranking_method, rejection, semantics_package, and trace_actions named arguments are grammar options in Marpa::R3. They were recognizer name arguments in Marpa::R2,

Actions that are Perl names must resolve to subroutines

It was a little noticed feature of Marpa::R2, that actions specified as Perl names (like "My_Action::doit") could resolve to scalars. In Marpa::R3 they must resolve to Perl subroutines.

The Stuifzand interface (PSIF) has been removed

The Stuifzand interface (PSIF), and its documentation, have been removed. Important in the development of Marpa, it now has little or now usage.

The Thin interface (THIF) has been removed.

The THIF was a "thin" Perl interface. It has been removed.

The NAIF has been removed

The NAIF was an older interface using hashes of named variables, instead of a DSL. It has been removed.

LATM is now the default

[name, values] is now the default action

Unicode now works in the SLIF DSL

Context::location is now Context::g1_range

New context variable, Context::g1_span

Marpa::R2::Scanless::G named arguments removed in Marpa::R3

The actions, action_object, default_action and default_empty_action named arguments of Marpa::R2::Scanless::G named arguments have been removed in Marpa::R3.

New Marpa::R3::Scanless::G methods

Several new methods have been added as accessors for L0 grammars:

  • Marpa::R2::Scanless::G::l0_symbol_dsl_form()

  • Marpa::R2::Scanless::G::l0_rule_expand()

  • Marpa::R2::Scanless::G::l0_rule_ids()

  • Marpa::R2::Scanless::G::l0_rule_show()

  • Marpa::R2::Scanless::G::l0_symbol_display_form()

  • Marpa::R2::Scanless::G::l0_symbol_ids()

  • Marpa::R2::Scanless::G::l0_symbol_name()

  • Marpa::R2::Scanless::G::l0_show_rules()

  • Marpa::R2::Scanless::G::l0_show_symbols()

Marpa::R2::Scanless::G methods removed in Marpa::R3

The Marpa::R2::Scanless::G::g0_rule() method, the Marpa::R2::Scanless::G::g0_rule_ids() method, the Marpa::R2::Scanless::G::g1_rule_ids() method, and the Marpa::R2::Scanless::G::rule() method, discouraged in Marpa::R2, have been eliminated in Marpa::R3.

The Marpa::R2::Scanless::G::symbol_description() method has been removed.

Marpa::R2::Scanless::G methods changed in Marpa::R3

The following methods have been changed:

  • Marpa::R2::Scanless::G::symbol_dsl_form()

  • Marpa::R2::Scanless::G::rule_expand()

  • Marpa::R2::Scanless::G::rule_ids()

  • Marpa::R2::Scanless::G::rule_show()

  • Marpa::R2::Scanless::G::symbol_display_form()

  • Marpa::R2::Scanless::G::symbol_ids()

  • Marpa::R2::Scanless::G::symbol_name()

  • Marpa::R2::Scanless::G::show_rules()

  • Marpa::R2::Scanless::G::show_symbols()

They no longer take an argument indicating the subgrammar -- instead they always return result for the G1 grammar. New methods have been added to return results for the L0 grammar. For these, see "New Marpa::R3::Scanless::G methods".

Marpa::R2::Scanless::R methods renamed in Marpa::R3

For historical reasons, the methods dealing with input and G1 parse location in Marpa::R2 often had unhelpful or misleading names. In Marpa::R3 an attempt is being made to name methods dealing with G1 and input parse location consistently, and to ensure that the G1 variants have g1 somewhere in their name. Accordingly, $slr->substring() has been renamed $slr->g1_literal(); and $slr->current_g1_location() is now $slr->g1_pos().

Marpa::R2::Scanless::R methods removed in Marpa::R3

The Marpa::R2::Scanless::R::event() method, Marpa::R2::Scanless::R::last_completed_range() method, Marpa::R2::Scanless::R::pause_lexeme() method, and the Marpa::R2::Scanless::R::range_to_string() method, discouraged in Marpa::R2, have been eliminated in Marpa::R3.

The per-parse constructor has been elminated

In Marpa::R2, if the semantics package has a new() method, that method was used as the constructor of the per-parse object. In Marpa::R3, there is no per-parse constructor.

The per-parse argument never affects the semantics package

As of Marpa::R3, the per-parse argument has no effect on the semantics package. In Marpa::R2, if the c<semantics_package> named argument was not used and the per-parse argument was blessed, then the package into which the per-parse argument was blessed became the semantics package.

The semantic closure now always receives exactly 2 arguments

Under Marpa::R2, the semantic closure received a varying number of arguments, depending on circumstances. Under Marpa::R3, the semantic closure always receives exactly 2 arguments. The first argument is the per-parse object. The second argument is a reference to an array containing the values of the child nodes, in lexical order. If there were no child nodes visible to the semantics, then the second argument is an empty array.

Under Marpa::R2, the events() method returned events in a specified order, by type of event. Under Marpa::R3, the order of events returned by the events() method is completely arbitrary.


  Marpa::R3 is Copyright (C) 2017, Jeffrey Kegler.

  This module is free software; you can redistribute it and/or modify it
  under the same terms as Perl 5.10.1. For more details, see the full text
  of the licenses in the directory LICENSES.

  This program is distributed in the hope that it will be
  useful, but without any warranty; without even the implied
  warranty of merchantability or fitness for a particular purpose.