Name

Unisyn::Parse - Parse a Unisyn expression.

Synopsis

Parse the Unisyn expression:

  𝒂 ❴ 𝒃 ⟦𝒄⟨ 𝗮 𝑒𝑞𝑢𝑎𝑙𝑠 𝒅 𝗯 𝙙 𝐭𝐢𝐦𝐞𝐬 ⟪𝗰 𝐩𝐥𝐮𝐬 𝗱⟫⟢  𝗲 𝑎𝑠𝑠𝑖𝑔𝑛 𝗳 𝐬𝐮𝐛 𝗴 𝙝⟩ 𝙘 ⟧ 𝙗 ❵ 𝙖

To get:

  Suffix: 𝙖
    Term
      Prefix: 𝒂
        Term
          Brackets: ⦇⦈
            Term
              Term
                Suffix: 𝙗
                  Term
                    Prefix: 𝒃
                      Term
                        Brackets: ⦋⦌
                          Term
                            Term
                              Suffix: 𝙘
                                Term
                                  Prefix: 𝒄
                                    Term
                                      Brackets: ⦏⦐
                                        Term
                                          Term
                                            Semicolon
                                              Term
                                                Assign: 𝑒𝑞𝑢𝑎𝑙𝑠
                                                  Term
                                                    Variable: 𝗮
                                                  Term
                                                    Dyad: 𝐭𝐢𝐦𝐞𝐬
                                                      Term
                                                        Suffix: 𝙙
                                                          Term
                                                            Prefix: 𝒅
                                                              Term
                                                                Variable: 𝗯
                                                      Term
                                                        Brackets: ⦓⦔
                                                          Term
                                                            Term
                                                              Dyad: 𝐩𝐥𝐮𝐬
                                                                Term
                                                                  Variable: 𝗰
                                                                Term
                                                                  Variable: 𝗱
                                              Term
                                                Assign: 𝑎𝑠𝑠𝑖𝑔𝑛
                                                  Term
                                                    Variable: 𝗲
                                                  Term
                                                    Dyad: 𝐬𝐮𝐛
                                                      Term
                                                        Variable: 𝗳
                                                      Term
                                                        Suffix: 𝙝
                                                          Term
                                                            Variable: 𝗴

Then traverse the parse tree printing the type of each node:

  variable
  variable
  prefix_d
  suffix_d
  variable
  variable
  plus
  times
  equals
  variable
  variable
  variable
  sub
  assign
  semiColon
  brackets_3
  prefix_c
  suffix_c
  brackets_2
  prefix_b
  suffix_b
  brackets_1
  prefix_a
  suffix_a

Description

Parse a Unisyn expression.

Version "20210921".

The following sections describe the methods in each functional area of this module. For an alphabetic listing of all methods by name see Index.

Create

Create a Unisyn parse of a utf8 string.

create($address, %options)

Create a new unisyn parse from a utf8 string.

     Parameter  Description
  1  $address   Address of a zero terminated utf8 source string to parse as a variable
  2  %options   Parse options.

Example:

    create (K(address, Rutf8 $Lex->{sampleText}{vav}))->print;                    # Create parse tree from source terminated with zero  # 𝗘𝘅𝗮𝗺𝗽𝗹𝗲


    ok Assemble(debug => 0, eq => <<END);
  Assign: 𝑎
    Term
      Variable: 𝗮
    Term
      Variable: 𝗯
  END

Parse

Parse Unisyn expressions

Traverse

Traverse the parse tree

traverseParseTree($parse)

Traverse the terms in parse tree in post order and call the operator subroutine associated with each term.

     Parameter  Description
  1  $parse     Parse tree

Example:

    my $s = Rutf8 $Lex->{sampleText}{Adv};                                        # Ascii
    my $p = create K(address, $s), operators => \&printOperatorSequence;

    K(address, $s)->printOutZeroString;
  # $p->dumpParseTree;
    $p->print;

    $p->traverseParseTree;  # 𝗘𝘅𝗮𝗺𝗽𝗹𝗲


    Assemble(debug => 0, eq => <<END)
  𝗮𝗮𝑒𝑞𝑢𝑎𝑙𝑠abc 123    𝐩𝐥𝐮𝐬𝘃𝗮𝗿
  Assign: 𝑒𝑞𝑢𝑎𝑙𝑠
    Term
      Variable: 𝗮𝗮
    Term
      Dyad: 𝐩𝐥𝐮𝐬
        Term
          Ascii: abc 123
        Term
          Variable: 𝘃𝗮𝗿
  variable
  ascii
  variable
  plus
  equals
  END

    my $s = Rutf8 $Lex->{sampleText}{ws};
    my $p = create (K(address, $s), operators => \&printOperatorSequence);

    K(address, $s)->printOutZeroString;                                           # Print input string
    $p->print;                                                                    # Print parse

    $p->traverseParseTree;                                                        # Traverse tree printing terms  # 𝗘𝘅𝗮𝗺𝗽𝗹𝗲


    Assemble(debug => 0, eq => <<END)
  𝗮𝑎𝑠𝑠𝑖𝑔𝑛⌊⟨❨𝗯𝗽❩⟩𝐩𝐥𝐮𝐬❪𝘀𝗰❫⌋⟢𝗮𝗮𝑎𝑠𝑠𝑖𝑔𝑛❬𝗯𝗯𝐩𝐥𝐮𝐬𝗰𝗰❭⟢
  Semicolon
    Term
      Assign: 𝑎𝑠𝑠𝑖𝑔𝑛
        Term
          Variable: 𝗮
        Term
          Brackets: ⌊⌋
            Term
              Term
                Dyad: 𝐩𝐥𝐮𝐬
                  Term
                    Brackets: ❨❩
                      Term
                        Term
                          Brackets: ❬❭
                            Term
                              Term
                                Variable: 𝗯𝗽
                  Term
                    Brackets: ❰❱
                      Term
                        Term
                          Variable: 𝘀𝗰
    Term
      Assign: 𝑎𝑠𝑠𝑖𝑔𝑛
        Term
          Variable: 𝗮𝗮
        Term
          Brackets: ❴❵
            Term
              Term
                Dyad: 𝐩𝐥𝐮𝐬
                  Term
                    Variable: 𝗯𝗯
                  Term
                    Variable: 𝗰𝗰
  variable
  variable
  variable
  plus
  assign
  variable
  variable
  variable
  plus
  assign
  semiColon
  END

Print

Print a parse tree

print($parse)

Print a parse tree.

     Parameter  Description
  1  $parse     Parse tree

Example:

    create (K(address, Rutf8 $Lex->{sampleText}{vav}))->print;                    # Create parse tree from source terminated with zero  # 𝗘𝘅𝗮𝗺𝗽𝗹𝗲


    ok Assemble(debug => 0, eq => <<END);
  Assign: 𝑎
    Term
      Variable: 𝗮
    Term
      Variable: 𝗯
  END

dumpParseTree($parse)

Dump the parse tree.

     Parameter  Description
  1  $parse     Parse tree

SubQuark

A set of quarks describing the method to be called for each lexical operator. These routines specialize the general purpose quark methods for use on parse methods.

Nasm::X86::Arena::DescribeSubQuarks($arena)

Return a descriptor for a subQuarks in the specified arena.

     Parameter  Description
  1  $arena     Arena descriptor

Nasm::X86::Arena::CreateSubQuarks($arena)

Create quarks in a specified arena.

     Parameter  Description
  1  $arena     Arena description optional arena address

Unisyn::Parse::SubQuarks::reload($q, %options)

Reload the description of a set of sub quarks.

     Parameter  Description
  1  $q         Subquarks
  2  %options   {arena=>arena to use; tree => first tree block; array => first array block}

Unisyn::Parse::SubQuarks::put($q, $string, $sub)

Put a new subroutine definition into the sub quarks.

     Parameter  Description
  1  $q         Subquarks
  2  $string    String containing operator type and method name
  3  $sub       Variable offset to subroutine

Unisyn::Parse::SubQuarks::dumpSubs($q)

Dump a set of quarks identifying subroutines.

     Parameter  Description
  1  $q         Quarks

Unisyn::Parse::SubQuarks::subFromQuark($q, $lexicals, $number)

Given the quark number for a lexical item and the quark set of lexical items get the offset of the associated method.

     Parameter  Description
  1  $q         Sub quarks
  2  $lexicals  Lexical item quarks
  3  $number    Lexical item quark

Unisyn::Parse::SubQuarks::lexToSub($q, $alphabet, $op, $sub)

Map a lexical item to a processing subroutine.

     Parameter  Description
  1  $q         Sub quarks
  2  $alphabet  The alphabet number
  3  $op        The operator name in that alphabet
  4  $sub       Subroutine definition

Unisyn::Parse::SubQuarks::dyad($q, $text, $sub)

Define a method for a dyadic operator.

     Parameter  Description
  1  $q         Sub quarks
  2  $text      The name of the operator as a utf8 string
  3  $sub       Associated subroutine definition

Unisyn::Parse::SubQuarks::assign($q, $text, $sub)

Define a method for an assign operator.

     Parameter  Description
  1  $q         Sub quarks
  2  $text      The name of the operator as a utf8 string
  3  $sub       Associated subroutine definition

Unisyn::Parse::SubQuarks::prefix($q, $text, $sub)

Define a method for a prefix operator.

     Parameter  Description
  1  $q         Sub quarks
  2  $text      The name of the operator as a utf8 string
  3  $sub       Associated subroutine definition

Unisyn::Parse::SubQuarks::suffix($q, $text, $sub)

Define a method for a suffix operator.

     Parameter  Description
  1  $q         Sub quarks
  2  $text      The name of the operator as a utf8 string
  3  $sub       Associated subroutine definition

Unisyn::Parse::SubQuarks::ascii($q, $sub)

Define a method for ascii text.

     Parameter  Description
  1  $q         Sub quarks
  2  $sub       Associated subroutine definition

Unisyn::Parse::SubQuarks::semiColon($q, $sub)

Define a method for the semicolon operator.

     Parameter  Description
  1  $q         Sub quarks
  2  $sub       Associated subroutine definition

Unisyn::Parse::SubQuarks::variable($q, $sub)

Define a method for a variable.

     Parameter  Description
  1  $q         Sub quarks
  2  $sub       Associated subroutine definition

Unisyn::Parse::SubQuarks::bracket($q, $open, $sub)

Define a method for a bracket operator.

     Parameter  Description
  1  $q         Sub quarks
  2  $open      Opening parenthesis
  3  $sub       Associated subroutine

Alphabets

Translate between alphabets.

asciiToAssignLatin($in)

Translate ascii to the corresponding letters in the assign latin alphabet.

     Parameter  Description
  1  $in        A string of ascii

asciiToAssignGreek($in)

Translate ascii to the corresponding letters in the assign greek alphabet.

     Parameter  Description
  1  $in        A string of ascii

asciiToDyadLatin($in)

Translate ascii to the corresponding letters in the dyad latin alphabet.

     Parameter  Description
  1  $in        A string of ascii

asciiToDyadGreek($in)

Translate ascii to the corresponding letters in the dyad greek alphabet.

     Parameter  Description
  1  $in        A string of ascii

asciiToPrefixLatin($in)

Translate ascii to the corresponding letters in the prefix latin alphabet.

     Parameter  Description
  1  $in        A string of ascii

asciiToPrefixGreek($in)

Translate ascii to the corresponding letters in the prefix greek alphabet.

     Parameter  Description
  1  $in        A string of ascii

asciiToSuffixLatin($in)

Translate ascii to the corresponding letters in the suffix latin alphabet.

     Parameter  Description
  1  $in        A string of ascii

asciiToSuffixGreek($in)

Translate ascii to the corresponding letters in the suffix greek alphabet.

     Parameter  Description
  1  $in        A string of ascii

asciiToVariableLatin($in)

Translate ascii to the corresponding letters in the suffix latin alphabet.

     Parameter  Description
  1  $in        A string of ascii

asciiToVariableGreek($in)

Translate ascii to the corresponding letters in the suffix greek alphabet.

     Parameter  Description
  1  $in        A string of ascii

asciiToEscaped($in)

Translate ascii to the corresponding letters in the escaped ascii alphabet.

     Parameter  Description
  1  $in        A string of ascii

semiColon()

Translate ascii to the corresponding letters in the escaped ascii alphabet.

printOperatorSequence($parse)

Print the operator calling sequence.

     Parameter  Description
  1  $parse     Parse

Hash Definitions

Unisyn::Parse Definition

Sub quarks

Output fields

address8

Address of source string as utf8

arena

Arena containing tree

fails

Number of failures encountered in this parse

operators

Methods implementing each lexical operator

parse

Offset to the head of the parse tree

quarks

Quarks representing the strings used in this parse

size8

Size of source string as utf8

source32

Source text as utf32

sourceLength32

Length of utf32 string

sourceSize32

Size of utf32 allocation

subQuarks

The quarks used to map a subroutine name to an offset

Private Methods

getAlpha($register, $address, $index)

Load the position of a lexical item in its alphabet from the current character.

     Parameter  Description
  1  $register  Register to load
  2  $address   Address of start of string
  3  $index     Index into string

getLexicalCode($register, $address, $index)

Load the lexical code of the current character in memory into the specified register.

     Parameter  Description
  1  $register  Register to load
  2  $address   Address of start of string
  3  $index     Index into string

putLexicalCode($register, $address, $index, $code)

Put the specified lexical code into the current character in memory.

     Parameter  Description
  1  $register  Register used to load code
  2  $address   Address of string
  3  $index     Index into string
  4  $code      Code to put

loadCurrentChar()

Load the details of the character currently being processed so that we have the index of the character in the upper half of the current character and the lexical type of the character in the lowest byte.

checkStackHas($depth)

Check that we have at least the specified number of elements on the stack.

     Parameter  Description
  1  $depth     Number of elements required on the stack

pushElement()

Push the current element on to the stack.

pushEmpty()

Push the empty element on to the stack.

lexicalNameFromLetter($l)

Lexical name for a lexical item described by its letter.

     Parameter  Description
  1  $l         Letter of the lexical item

lexicalNumberFromLetter($l)

Lexical number for a lexical item described by its letter.

     Parameter  Description
  1  $l         Letter of the lexical item

lexicalItemLength($source32, $offset)

Put the length of a lexical item into variable size.

     Parameter  Description
  1  $source32  B<address> of utf32 source representation
  2  $offset    B<offset> to lexical item in utf32

new($depth, $description)

Create a new term in the parse tree rooted on the stack.

     Parameter     Description
  1  $depth        Stack depth to be converted
  2  $description  Text reason why we are creating a new term

error($message)

Write an error message and stop.

     Parameter  Description
  1  $message   Error message

testSet($set, $register)

Test a set of items, setting the Zero Flag is one matches else clear the Zero flag.

     Parameter  Description
  1  $set       Set of lexical letters
  2  $register  Register to test

checkSet($set)

Check that one of a set of items is on the top of the stack or complain if it is not.

     Parameter  Description
  1  $set       Set of lexical letters

reduce($priority)

Convert the longest possible expression on top of the stack into a term at the specified priority.

     Parameter  Description
  1  $priority  Priority of the operators to reduce

reduceMultiple($priority)

Reduce existing operators on the stack.

     Parameter  Description
  1  $priority  Priority of the operators to reduce

accept_a()

Assign.

accept_b()

Open.

accept_B()

Closing parenthesis.

accept_d()

Infix but not assign or semi-colon.

accept_p()

Prefix.

accept_q()

Post fix.

accept_s()

Semi colon.

accept_v()

Variable.

parseExpression()

Parse the string of classified lexical items addressed by register $start of length $length. The resulting parse tree (if any) is returned in r15.

MatchBrackets(@parameters)

Replace the low three bytes of a utf32 bracket character with 24 bits of offset to the matching opening or closing bracket. Opening brackets have even codes from 0x10 to 0x4e while the corresponding closing bracket has a code one higher.

     Parameter    Description
  1  @parameters  Parameters

ClassifyNewLines(@parameters)

Scan input string looking for opportunities to convert new lines into semi colons.

     Parameter    Description
  1  @parameters  Parameters

ClassifyWhiteSpace(@parameters)

Classify white space per: "lib/Unisyn/whiteSpace/whiteSpaceClassification.pl".

     Parameter    Description
  1  @parameters  Parameters

parseUtf8($parse, @parameters)

Parse a unisyn expression encoded as utf8 and return the parse tree.

     Parameter    Description
  1  $parse       Parse
  2  @parameters  Parameters

printLexicalItem($parse, $source32, $offset, $size)

Print the utf8 string corresponding to a lexical item at a variable offset.

     Parameter  Description
  1  $parse     Parse tree
  2  $source32  B<address> of utf32 source representation
  3  $offset    B<offset> to lexical item in utf32
  4  $size      B<size> in utf32 chars of item

showAlphabet($alphabet)

Show an alphabet.

     Parameter  Description
  1  $alphabet  Alphabet name

T($key, $expected, %options)

Parse some text and dump the results.

     Parameter  Description
  1  $key       Key of text to be parsed
  2  $expected  Expected result
  3  %options   Options

C($key, $expected, %options)

Parse some text and print the results.

     Parameter  Description
  1  $key       Key of text to be parsed
  2  $expected  Expected result
  3  %options   Options

Index

1 accept_a - Assign.

2 accept_b - Open.

3 accept_B - Closing parenthesis.

4 accept_d - Infix but not assign or semi-colon.

5 accept_p - Prefix.

6 accept_q - Post fix.

7 accept_s - Semi colon.

8 accept_v - Variable.

9 asciiToAssignGreek - Translate ascii to the corresponding letters in the assign greek alphabet.

10 asciiToAssignLatin - Translate ascii to the corresponding letters in the assign latin alphabet.

11 asciiToDyadGreek - Translate ascii to the corresponding letters in the dyad greek alphabet.

12 asciiToDyadLatin - Translate ascii to the corresponding letters in the dyad latin alphabet.

13 asciiToEscaped - Translate ascii to the corresponding letters in the escaped ascii alphabet.

14 asciiToPrefixGreek - Translate ascii to the corresponding letters in the prefix greek alphabet.

15 asciiToPrefixLatin - Translate ascii to the corresponding letters in the prefix latin alphabet.

16 asciiToSuffixGreek - Translate ascii to the corresponding letters in the suffix greek alphabet.

17 asciiToSuffixLatin - Translate ascii to the corresponding letters in the suffix latin alphabet.

18 asciiToVariableGreek - Translate ascii to the corresponding letters in the suffix greek alphabet.

19 asciiToVariableLatin - Translate ascii to the corresponding letters in the suffix latin alphabet.

20 C - Parse some text and print the results.

21 checkSet - Check that one of a set of items is on the top of the stack or complain if it is not.

22 checkStackHas - Check that we have at least the specified number of elements on the stack.

23 ClassifyNewLines - Scan input string looking for opportunities to convert new lines into semi colons.

24 ClassifyWhiteSpace - Classify white space per: "lib/Unisyn/whiteSpace/whiteSpaceClassification.

25 create - Create a new unisyn parse from a utf8 string.

26 dumpParseTree - Dump the parse tree.

27 error - Write an error message and stop.

28 getAlpha - Load the position of a lexical item in its alphabet from the current character.

29 getLexicalCode - Load the lexical code of the current character in memory into the specified register.

30 lexicalItemLength - Put the length of a lexical item into variable size.

31 lexicalNameFromLetter - Lexical name for a lexical item described by its letter.

32 lexicalNumberFromLetter - Lexical number for a lexical item described by its letter.

33 loadCurrentChar - Load the details of the character currently being processed so that we have the index of the character in the upper half of the current character and the lexical type of the character in the lowest byte.

34 MatchBrackets - Replace the low three bytes of a utf32 bracket character with 24 bits of offset to the matching opening or closing bracket.

35 Nasm::X86::Arena::CreateSubQuarks - Create quarks in a specified arena.

36 Nasm::X86::Arena::DescribeSubQuarks - Return a descriptor for a subQuarks in the specified arena.

37 new - Create a new term in the parse tree rooted on the stack.

38 parseExpression - Parse the string of classified lexical items addressed by register $start of length $length.

39 parseUtf8 - Parse a unisyn expression encoded as utf8 and return the parse tree.

40 print - Print a parse tree.

41 printLexicalItem - Print the utf8 string corresponding to a lexical item at a variable offset.

42 printOperatorSequence - Print the operator calling sequence.

43 pushElement - Push the current element on to the stack.

44 pushEmpty - Push the empty element on to the stack.

45 putLexicalCode - Put the specified lexical code into the current character in memory.

46 reduce - Convert the longest possible expression on top of the stack into a term at the specified priority.

47 reduceMultiple - Reduce existing operators on the stack.

48 semiColon - Translate ascii to the corresponding letters in the escaped ascii alphabet.

49 showAlphabet - Show an alphabet.

50 T - Parse some text and dump the results.

51 testSet - Test a set of items, setting the Zero Flag is one matches else clear the Zero flag.

52 traverseParseTree - Traverse the terms in parse tree in post order and call the operator subroutine associated with each term.

53 Unisyn::Parse::SubQuarks::ascii - Define a method for ascii text.

54 Unisyn::Parse::SubQuarks::assign - Define a method for an assign operator.

55 Unisyn::Parse::SubQuarks::bracket - Define a method for a bracket operator.

56 Unisyn::Parse::SubQuarks::dumpSubs - Dump a set of quarks identifying subroutines.

57 Unisyn::Parse::SubQuarks::dyad - Define a method for a dyadic operator.

58 Unisyn::Parse::SubQuarks::lexToSub - Map a lexical item to a processing subroutine.

59 Unisyn::Parse::SubQuarks::prefix - Define a method for a prefix operator.

60 Unisyn::Parse::SubQuarks::put - Put a new subroutine definition into the sub quarks.

61 Unisyn::Parse::SubQuarks::reload - Reload the description of a set of sub quarks.

62 Unisyn::Parse::SubQuarks::semiColon - Define a method for the semicolon operator.

63 Unisyn::Parse::SubQuarks::subFromQuark - Given the quark number for a lexical item and the quark set of lexical items get the offset of the associated method.

64 Unisyn::Parse::SubQuarks::suffix - Define a method for a suffix operator.

65 Unisyn::Parse::SubQuarks::variable - Define a method for a variable.

Installation

This module is written in 100% Pure Perl and, thus, it is easy to read, comprehend, use, modify and install via cpan:

  sudo cpan install Unisyn::Parse

Author

philiprbrenan@gmail.com

http://www.appaapps.com

Copyright

Copyright (c) 2016-2021 Philip R Brenan.

This module is free software. It may be used, redistributed and/or modified under the same terms as Perl itself.

1 POD Error

The following errors were encountered while parsing the POD:

Around line 3694:

=pod directives shouldn't be over one line long! Ignoring all 5 lines of content