Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Parse tree for ambiguous grammar contains `drv` subtree?

See original GitHub issue

Hi – I recently updated the version of Lark I’m using and am trying to debug the following program (which contains a deliberately ambiguous grammar):

from lark import Lark

grammar = '''
stmt : ifthenelse
     | ifthen
     | assign

ifthen : "if" "cond" "then" stmt

ifthenelse : "if" "cond" "then" stmt "else" stmt

assign : "a:=1"

%import common.WS
%ignore WS
'''

foo = 'if cond then if cond then a:=1 else a:=1'

parser = Lark(grammar, start='stmt', ambiguity='explicit')
print(parser.parse(foo))

When I was using lark 0.2.7, I got the following output.

Once I upgraded to any version of lark 0.3.0 or above, I instead get the following output, which I’ve pretty-printed for readability:

Tree(_ambig, [
    Tree(stmt, [
        Tree(ifthenelse, [
            Tree(stmt, [
                Tree(ifthen, [
                    Tree(stmt, [Tree(drv, [Token(__ANONSTR_4, 'a:=1')])])])]), 
            Tree(stmt, [Tree(drv, [Token(__ANONSTR_4, 'a:=1')])])])]), 
    Tree(stmt, [
        Tree(ifthen, [
            Tree(stmt, [
                Tree(ifthenelse, [
                    Tree(stmt, [Tree(assign, [])]), 
                    Tree(stmt, [Tree(assign, [])])])])])])])

While I appreciate that the output is significantly shorter (it certainly does make debugging easier!), I’m also a little baffled as to why my parse tree contains the subtree Tree(drv, [Token(__ANONSTR_4, 'a:=1')]) instead of Tree(assign, []), which is what I was expecting.

Is this behavior intentional or a bug? (The documentation didn’t seem to mention anything about drv subtrees). Either way, is there any way I can get lark to return Tree(assign, []) instead of the drv thing?

Thanks!

(For context, I’m updating some code that pretty-prints ambiguous trees for debugging purposes and ran into this case. I’m sure I could modify my code to work around this issue if necessary, but figured I might as well ask first.)