The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.

Changes for version 0.006

  • Change: 7816a4894a020596e52157c05c39db0b94cdf2bf Author: Ben Tyler <benjamin.tyler@booking.com> Date : 2019-06-15 15:02:06 +0000
    • tests: escape braces in regex
  • Change: 021031425483141c5540ab1e8c252aef709b1dbc Author: Ben Tyler <benjamin.tyler@booking.com> Date : 2019-06-15 14:56:10 +0000
    • README: stray word
  • Change: 099ccc4e920bd912cd4817c892816330a7743268 Author: Ben Tyler <benjamin.tyler@gmail.com> Date : 2019-06-15 14:52:18 +0000
    • Perlish parser (#2)
    • Prototype: Perl-ish syntax for accessing vars in scope
    • This is potentially a really terrible idea, but it was fun to work on the parser. Basically: JSON::Pointer syntax is probably a little too simple -- Sawyer had good feedback on not just guessing whether something is a hash or array access. So he wrote a nice, compact lexer and parser to support quoted strings and spit out tokens with types.
    • This commit takes that general idea, and goes a little nuts: the parser now produces an AST describing a Perl-like syntax for dereferencing and accessing Perl data structures. It supports simple cases, like '$my_cool_hashref', but also fancy ones, like '$my_cool_hashref->{$some_arrayref->[-1]->{nested_hashkey}->{$some_sc alar}}';
    • It does not use Perl 'eval'; indeed, it violates normal Perl syntax rules in several ways. For example, indexing into a data structure _always_ uses '->', even if it isn't a ref, e.g. '%my_cool_hash->{foo}'. As a result, this might be uncanny valley territory.
    • TODO: tests, the interpreter, reviewing whether this is remotely a reasonable thing to do.
    • parser: export 'lex', 'parse', and ':constants'
    • ':constants' will be used for testing, as well as interpreting the op tree
    • parser: these syntax names are never used
    • lexer: die on empty or obviously bogus spec
    • Everything will always need to start with a Perl symbol name.
    • parser: split hash/array parsing into subs
    • I think this makes it a bit easier to follow the flow.
    • parser: be stricter about symbol names
    • Catch some obvious typos that will never be valid symbol names (like '$fo o').
    • parser: better error messages for Perl syntax
    • This is probably a signal that this langauge should conform more closely to real Perl syntax, but I'm not ready to invest in doing that just yet.
    • lexer/parser: require strings to be quoted
    • This allows the parser to be more informative about bad input. Legitmate use of single quotes in hash key literals can be managed via backslash escapes.
    • parser: flesh out test cases
    • Along the way the lexer was changed to return a list instead of an arrayref. This has better symmetry with the input format for 'parse', which is a list of tokens.
    • parser: tests for invalid parses
    • parser: make it an object
    • This makes it easier to introduce a 'parser' argument to Devel::Optic which is any object which ->can("parse").
    • parser: cosmetic reorganization
    • interpreter: naive recursive interpreter
    • This commit introduces a basic recursive interpreter for the AST spat out by the parser. With this, the basic functionality of Devel::Optic is back in place, now with a much higher power language for data access.
    • Ideally I'd like to change this and the parser to be non-recursive to reduce the number of sub calls -- the interpeter code, at least, might be running inside very hot parts of a system -- but I couldn't quite picture how to do this with explicit stacks, and I wanted to get something functioning to start with. Make it work, make it right, make it fast, yada yada yada.
    • TODO: adapt tests, adapt main functions to use this, more human readable error messages, performance measurements and optimizations.
    • The error messages are going to be quite a challenge -- the simple (and non-nested) JSON::Pointer syntax made it easy to tell users exactly where we were when things went pear shaped, so we could give very useful feedback to guide iteration. That's a bit trickier with recursive tree traversal.
    • tests: move 'parser' earlier than 'full_picture'
    • full_picture uses the parser in order to work, so it should come later in the sequence.
    • sweeping code reorganization
    • stuff interpreter/parser inside a 'lens' (maybe the name wasn't so bad?), which is self-contained as to what language it expects and how it gets data out of Perl
    • update docs to reflect new syntax
    • some tests are commented out because introducing a full-blown mini language for apertures has a lot of benefits, but simplicity of contextful error reporting is not among them.
    • die -> croak
    • many errors are caller errors
    • 'aperture' -> 'query'
    • KISS terminology
    • high quality error reporting
    • This commit restores the errors reported by the Perlish lens to more or less the level of quality I had in the first, naive query syntax.