The London Perl and Raku Workshop takes place on 26th Oct 2024. If your company depends on Perl, please consider sponsoring and/or attending.
 / 
Unisyn-Parse-20210830
River stage zero No dependents
/ Unisyn::Parse

Name

Unisyn::Parse - Parse a Unisyn expression.

Synopsis

Parse the Unisyn expression:

    my $expr = "𝗮𝑎𝑠𝑠𝑖𝑔𝑛⌊⟨❨𝗯𝗽❩⟩𝐩𝐥𝐮𝐬❪𝘀𝗰❫⌋⟢𝗮𝗮𝑎𝑠𝑠𝑖𝑔𝑛❬𝗯𝗯𝐩𝐥𝐮𝐬𝗰𝗰❭⟢";

using:

  create (K(address, Rutf8 $expr))->print;

to get:

  ok Assemble(debug => 0, eq => <<END);
Semicolon
  Term
    Assign: 𝑎𝑠𝑠𝑖𝑔𝑛
      Term
        Variable: 𝗮
      Term
        Brackets: ⌊⌋
          Term
            Term
              Dyad: 𝐩𝐥𝐮𝐬
                Term
                  Brackets: ❨❩
                    Term
                      Term
                        Brackets: ❬❭
                          Term
                            Term
                              Variable: 𝗯𝗽
                Term
                  Brackets: ❰❱
                    Term
                      Term
                        Variable: 𝘀𝗰
  Term
    Assign: 𝑎𝑠𝑠𝑖𝑔𝑛
      Term
        Variable: 𝗮𝗮
      Term
        Brackets: ❴❵
          Term
            Term
              Dyad: 𝐩𝐥𝐮𝐬
                Term
                  Variable: 𝗯𝗯
                Term
                  Variable: 𝗰𝗰
  END

Description

Parse a Unisyn expression.

Version "20210829".

The following sections describe the methods in each functional area of this module. For an alphabetic listing of all methods by name see Index.

Create

Create a Unisyn parse of a utf8 string.

create($address)

Create a new unisyn parse from a utf8 string.

     Parameter  Description
  1  $address   Address of utf8 source string to parse as a variable

Example:

    create (K(address, Rutf8 $Lex->{sampleText}{vav}))->print;                    # Create parse tree from source terminated with  zero  # 𝗘𝘅𝗮𝗺𝗽𝗹𝗲


    ok Assemble(debug => 0, eq => <<END);
  Assign: 𝑎
    Term
      Variable: 𝗮
    Term
      Variable: 𝗯
  END

Parse

Parse Unisyn expressions

Print

Print a parse tree

print($parse)

Create a parser for an expression described by variables.

     Parameter  Description
  1  $parse     Parse tree

Example:

    create (K(address, Rutf8 $Lex->{sampleText}{vav}))->print;                    # Create parse tree from source terminated with  zero  # 𝗘𝘅𝗮𝗺𝗽𝗹𝗲


    ok Assemble(debug => 0, eq => <<END);
  Assign: 𝑎
    Term
      Variable: 𝗮
    Term
      Variable: 𝗯
  END

Hash Definitions

Unisyn::Parse Definition

Description of parse

Output fields

address8

Address of source string as utf8

arena

Arena containing tree

fails

Number of failures encountered in this parse

parse

Offset to the head of the parse tree

size8

Size of source string as utf8

source32

Source text as utf32

sourceLength32

Length of utf32 string

sourceSize32

Size of utf32 allocation

Private Methods

getAlpha($register, $address, $index)

Load the position of a lexical item in its alphabet from the current character.

     Parameter  Description
  1  $register  Register to load
  2  $address   Address of start of string
  3  $index     Index into string

getLexicalCode($register, $address, $index)

Load the lexical code of the current character in memory into the specified register.

     Parameter  Description
  1  $register  Register to load
  2  $address   Address of start of string
  3  $index     Index into string

putLexicalCode($register, $address, $index, $code)

Put the specified lexical code into the current character in memory.

     Parameter  Description
  1  $register  Register used to load code
  2  $address   Address of string
  3  $index     Index into string
  4  $code      Code to put

loadCurrentChar()

Load the details of the character currently being processed so that we have the index of the character in the upper half of the current character and the lexical type of the character in the lowest byte.

checkStackHas($depth)

Check that we have at least the specified number of elements on the stack.

     Parameter  Description
  1  $depth     Number of elements required on the stack

pushElement()

Push the current element on to the stack.

pushEmpty()

Push the empty element on to the stack.

lexicalNameFromLetter($l)

Lexical name for a lexical item described by its letter.

     Parameter  Description
  1  $l         Letter of the lexical item

lexicalNumberFromLetter($l)

Lexical number for a lexical item described by its letter.

     Parameter  Description
  1  $l         Letter of the lexical item

new($depth, $description)

Create a new term in the parse tree rooted on the stack.

     Parameter     Description
  1  $depth        Stack depth to be converted
  2  $description  Text reason why we are creating a new term

error($message)

Die.

     Parameter  Description
  1  $message   Error message

testSet($set, $register)

Test a set of items, setting the Zero Flag is one matches else clear the Zero flag.

     Parameter  Description
  1  $set       Set of lexical letters
  2  $register  Register to test

checkSet($set)

Check that one of a set of items is on the top of the stack or complain if it is not.

     Parameter  Description
  1  $set       Set of lexical letters

reduce($priority)

Convert the longest possible expression on top of the stack into a term at the specified priority.

     Parameter  Description
  1  $priority  Priority of the operators to reduce

reduceMultiple($priority)

Reduce existing operators on the stack.

     Parameter  Description
  1  $priority  Priority of the operators to reduce

accept_a()

Assign.

accept_b()

Open.

accept_B()

Closing parenthesis.

accept_d()

Infix but not assign or semi-colon.

accept_p()

Prefix.

accept_q()

Post fix.

accept_s()

Semi colon.

accept_v()

Variable.

parseExpressionCode()

Parse the string of classified lexical items addressed by register $start of length $length. The resulting parse tree (if any) is returned in r15.

parseExpression(@parameters)

Create a parser for an expression described by variables.

     Parameter    Description
  1  @parameters  Parameters describing expression

MatchBrackets(@parameters)

Replace the low three bytes of a utf32 bracket character with 24 bits of offset to the matching opening or closing bracket. Opening brackets have even codes from 0x10 to 0x4e while the corresponding closing bracket has a code one higher.

     Parameter    Description
  1  @parameters  Parameters

ClassifyNewLines(@parameters)

Scan input string looking for opportunities to convert new lines into semi colons.

     Parameter    Description
  1  @parameters  Parameters

ClassifyWhiteSpace(@parameters)

Classify white space per: "lib/Unisyn/whiteSpace/whiteSpaceClassification.pl".

     Parameter    Description
  1  @parameters  Parameters

parseUtf8($p, @parameters)

Parse a unisyn expression encoded as utf8 and return the parse tree.

     Parameter    Description
  1  $p           Parse
  2  @parameters  Parameters

printLexicalItem($parse, $source32, $offset)

Print the utf8 string corresponding to a lexical item at a variable offset.

     Parameter  Description
  1  $parse     Parse tree
  2  $source32  Variable address of utf32 source representation
  3  $offset    Variable offset to lexical item in utf32

printBrackets($parse, $source32, $offset)

Print the utf8 string corresponding to a lexical item at a variable offset.

     Parameter  Description
  1  $parse     Parse tree
  2  $source32  Variable address of utf32 source representation
  3  $offset    Variable offset to lexical item in utf32

T($key, $expected, %options)

Test a parse.

     Parameter  Description
  1  $key       Key of text to be parsed
  2  $expected  Expected result
  3  %options   Options

Index

1 accept_a - Assign.

2 accept_b - Open.

3 accept_B - Closing parenthesis.

4 accept_d - Infix but not assign or semi-colon.

5 accept_p - Prefix.

6 accept_q - Post fix.

7 accept_s - Semi colon.

8 accept_v - Variable.

9 checkSet - Check that one of a set of items is on the top of the stack or complain if it is not.

10 checkStackHas - Check that we have at least the specified number of elements on the stack.

11 ClassifyNewLines - Scan input string looking for opportunities to convert new lines into semi colons.

12 ClassifyWhiteSpace - Classify white space per: "lib/Unisyn/whiteSpace/whiteSpaceClassification.

13 create - Create a new unisyn parse from a utf8 string.

14 error - Die.

15 getAlpha - Load the position of a lexical item in its alphabet from the current character.

16 getLexicalCode - Load the lexical code of the current character in memory into the specified register.

17 lexicalNameFromLetter - Lexical name for a lexical item described by its letter.

18 lexicalNumberFromLetter - Lexical number for a lexical item described by its letter.

19 loadCurrentChar - Load the details of the character currently being processed so that we have the index of the character in the upper half of the current character and the lexical type of the character in the lowest byte.

20 MatchBrackets - Replace the low three bytes of a utf32 bracket character with 24 bits of offset to the matching opening or closing bracket.

21 new - Create a new term in the parse tree rooted on the stack.

22 parseExpression - Create a parser for an expression described by variables.

23 parseExpressionCode - Parse the string of classified lexical items addressed by register $start of length $length.

24 parseUtf8 - Parse a unisyn expression encoded as utf8 and return the parse tree.

25 print - Create a parser for an expression described by variables.

26 printBrackets - Print the utf8 string corresponding to a lexical item at a variable offset.

27 printLexicalItem - Print the utf8 string corresponding to a lexical item at a variable offset.

28 pushElement - Push the current element on to the stack.

29 pushEmpty - Push the empty element on to the stack.

30 putLexicalCode - Put the specified lexical code into the current character in memory.

31 reduce - Convert the longest possible expression on top of the stack into a term at the specified priority.

32 reduceMultiple - Reduce existing operators on the stack.

33 T - Test a parse.

34 testSet - Test a set of items, setting the Zero Flag is one matches else clear the Zero flag.

Installation

This module is written in 100% Pure Perl and, thus, it is easy to read, comprehend, use, modify and install via cpan:

  sudo cpan install Unisyn::Parse

Author

philiprbrenan@gmail.com

http://www.appaapps.com

Copyright

Copyright (c) 2016-2021 Philip R Brenan.

This module is free software. It may be used, redistributed and/or modified under the same terms as Perl itself.