LALR templates not expanding

Describe the bug

Basically, templates don’t work as well on the LALR parser. I have to manually expand the template in order for my grammar to work on LALR. This kind of defeats the purpose of templates.

To Reproduce

Grammar “A”:

%import common.WORD
%import common.NEWLINE -> _NL
%ignore " "

start: _list{WORD}

_COMMA: "," _NL?
_list{x}: _sep{x, _COMMA} _COMMA?
_sep{x, sep}: x (sep x)*

Grammar “B” (this is equivalent to grammar A but I manually expanded one of the templates):

%import common.WORD
%import common.NEWLINE -> _NL
%ignore " "

start: _list{WORD}

_COMMA: "," _NL?
_list{x}:  x (_COMMA x)* _COMMA?

Problem

Earley can parse the following examples with either grammars

Hello, world

Hello, world,

LALR can only parse both inputs using grammar B (grammar A fails when trying to parse the second input)

[erezsh]

Each template is currently expanded into its own rule. Which is why you get a shift/reduce conflict.

Btw, I already opened an issue for this a year ago, but it didn’t get any attention: https://github.com/lark-parser/lark/issues/822

[erezsh]

Templates generate trees where data is the template name. It’s very predictable.

Of course, you won’t see those starting with underscore, because they get inlined during parsing.