- LICENSE AND COPYRIGHT
Marpa::Tracing - Tracing Your Grammar
This document describes techniques for tracing and debugging Marpa parses and grammars.
If parsing failed in the recognizer, look at the input location where it happened. Compare the input against the grammar. This step is fairly obvious, but I include it because even experts (actually, especially experts) will sometimes overlook the obvious in a rush to use more advanced techniques.
Make sure that Marpa's
warnings named arguments for both the grammar and the recognizer are turned on. The
warnings named arguments are on by default.
When Marpa "strips" its objects, it removes data that is not needed for subsequent processing. This saves time and memory, but data that is not needed for processing can be extremely valuable for debugging. When objects are stripped, many of Marpa's tracing methods will return partial information or no information at all.
You should turn off Marpa's
strip named argument to the grammar. Grammar stripping is on by default. If you are using the recognizer's
strip method to strip the recognizer, stop doing that.
Turn on the
trace_terminals recognizer named argument. This tells you which tokens the recognizer is looking for and which ones it thinks it found. If the problem is in lexing,
trace_terminals tells you the whole story.
Even if the problem is not in the lexing, tracing terminals can tell you a lot. Marpa uses prediction-driven lexing. At any given parse location, Marpa is only looking for those tokens that it thinks could result in a successful parse. That means that the list of tokens that the recognizer is looking for tells you where the recognizer thinks it is.
It sometimes helps to look carefully at the output of
show_symbols. Check if anything there is not what you expected.
trace_actions will show you how action names resolve to semantic Perl closures. Setting the
trace_values evaluator named argument to a trace level of 1 traces the values of the parse tree nodes as they are pushed on, and popped off, the evaluation stack.
You can trace the earley sets as they are completed, but it is usually best to wait until input to the recognizer is finished. At that point call
Marpa::Recognizer::show_earley_sets . Since the items in the earley sets use QDFA states, you usually will want to call
Marpa::Grammar::show_QDFA as well. The implementation document has example outputs from the
show_QDFA methods, and explains how to read them.
A complete investigation of a parse includes the following:
Make sure the
warningsoption is turned on. It is on by default.
Turn off the
stripMarpa named argument. By default, it is on.
Make sure you're not stripping the recognizer.
Turn on the
trace_terminalsrecognizer named argument.
show_symbolson the precomputed grammar.
show_ruleson the precomputed grammar.
Turn on the
trace_actionsevaluator named argument.
trace_valuesevaluator named argument to level 1.
show_earley_setson the recognizer.
show_QDFAon the precomputed grammar.
When considering how much tracing to turn on, remember that if the input text to the grammar is large, the outputs from
trace_values can be very lengthy. You want to work with short inputs if at all possible.
Copyright 2007-2010 Jeffrey Kegler, all rights reserved. Marpa is free software under the Perl license. For details see the LICENSE file in the Marpa distribution.