question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Parsing slightly ambigious grammar with LALR

See original GitHub issue

First of all thanks for creating LARK. I have used bison in past life and to me it always felt challenging but it was an absolute joy when I first used Lark.

I have a slightly ambiguous grammar which works fine with early parser but fails with error when using LALR. The grammar is

parser = Lark(r"""
    expr: SIGNED_NUMBER
        | SIMPLE_STRING
        | name
        | FUNCTION_NAME "(" function_args ")"

    function_args : expr
                  | function_args "," expr
                  
    name: NAME

    NAME : /[a-zA-Z0-9_]+/
    SIMPLE_STRING : /'[^']*'/
    FUNCTION_NAME : /[a-zA-Z0-9_]+/
    
    %import common.SIGNED_NUMBER
    %import common.WS
    %ignore WS
""", start="expr", parser="lalr")

A token without any context can be either NAME or FUNCTION_NAME. While using the default parser, it resolves these ambiguities correctly

tree = parser.parse("A(B(x, y, 5))")
print(tree.pretty())

produces

expr
  A
  function_args
    expr
      B
      function_args
        function_args
          function_args
            expr
              name	x
          expr
            name	y
        expr	5

But with lalr it fails to parse with following error

UnexpectedCharacters: No terminal defined for ',' at line 1 col 6

A(B(x, y, 5))
     ^

Expecting: {'LPAR'}

Previous tokens: Token('FUNCTION_NAME', 'x')

I am sure its a very n00b question. So if its recommended to read something (books etc.) for me to understand this, I would love some suggestions. Thanks!

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:7 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
erezshcommented, Feb 22, 2021

As for a tip for next time, try to keep your terminals to a minimum, make sure there no collisions between regexps (regexp-string and string-string aren’t a problem, Lark can detect those).

We’re actually working a feature that would print a warning in this exact situation, but it’s not ready yet.

0reactions
ashish01commented, Feb 24, 2021

Excellent! Thanks, adding a explicit priority solves the issue. For reference, here is how I changed the grammar

parser = Lark(r"""
    expr: SIGNED_NUMBER_R
        | SIMPLE_STRING
        | name
        | function_name "(" function_args ")"

    function_args : expr
                  | function_args "," expr
                  
    function_name: NAME
    name: NAME

    NAME.2 : /[a-zA-Z0-9_]+/
    SIMPLE_STRING : /'[^']*'/
    SIGNED_NUMBER_R.1: SIGNED_NUMBER
    
    %import common.SIGNED_NUMBER
    %import common.WS
    %ignore WS
""", start="expr", parser="lalr")
tree = parser.parse("A(B(x, 7y, 5))")
print(tree.pretty())

which produces

expr
  function_name	A
  function_args
    expr
      function_name	B
      function_args
        function_args
          function_args
            expr
              name	x
          expr
            name	7y
        expr
          name	5
Read more comments on GitHub >

github_iconTop Results From Across the Web

Parsing ambiguous grammars using LR parser - GeeksforGeeks
LR parser can be used to parse ambiguous grammars. LR parser resolves the conflicts (shift/reduce or reduce/reduce) in parsing table of ...
Read more >
15: Miscellaneous Parsing - Stanford University
No ambiguous grammar is LL(1) or LR(1), so we must either rewrite the grammar to remove the ambiguity or resolve conflicts in the...
Read more >
Advice on solving parsing conflicts · Issue #271 · lalrpop ...
In your particular case, this is not an "LR(1)" peculiarity, but rather a truly ambiguous grammar. The error message is trying to tell...
Read more >
Lecture 7-8 — Context-free grammars and bottom-up parsing
It follows that ambiguous grammars can never be LALR(1). Show a parse tree, and corresponding s/r parse, that repre- sents left-associativity ...
Read more >
Can this seeming ambiguity be parsed in a LALR(1) parser ...
Despite the title of your post, your grammar is not ambiguous. It is simply not LR(1), for the reason you mention: the input....
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found