antlr4 4.9.2 parser calls error handler for same "foobar^4" input although nothing changed in grammar.g4

Hello,

i was thinking about posting this in discussions but since it only occurs with the latest 4.9.2 version maybe here is the correct place.

in our codebase we have the following grammar.g4 (excerpt):

PhraseLiteral
    :   '"' StringCharacters? '"'
    |   TermCharacters
    ;

fragment
StringCharacters
    :   StringCharacter+
    ;

fragment
TermCharacters
    : ~["\\ \t\r\u000C\n()^*] ~["\\ \t\r\u000C\n()^]*
    ;

we generate the needed files with:

antlr4 -Dlanguage=JavaScript /path/to/SearchGrammar2.g4

and later we use it the following (also only excerpt):

const errorHandler = new SyntaxProcessorErrorHandler();
const chars = new antlr4.InputStream(input); // input is foobar^4
const lexer = new SearchGrammar2Lexer(chars);
const tokens = new antlr4.CommonTokenStream(lexer);
const parser = new SearchGrammar2Parser(tokens);
parser.addErrorListener(errorHandler);

with version 4.9.1 we dont get an error but now with 4.9.2 we get the following error:

extraneous input '^' expecting {'^', DEC_NUMBER, PhraseLiteral}

we get the error when using the following input:

foobar^4 // maybe caret handling has been changed in 4.9.2 ?

we haven’t changed anything except updating to 4.9.2 and use JavaScript, do you have any idea what changes in 4.9.2 could cause this?

edit: another side note, which may be helpful: the generated files hasnt changed at all using 4.9.2 - the only difference is that after installing the 4.9.2 npm package we get this error, if we use the 4.9.1 npm package there is no such error

[kaby76]

@duc-talentwunder Excellent! I can reproduce the problem. Sorry for the strange reference to a similar problem. I was thinking out loud that what you are seeing is due to the same bug in the code. I will debug the JS runtime with your grammar over the weekend to verify that it is. Thanks.

[lisa-tw]

Hi there. I am working with Duc and tried our grammar with Java target. I followed the instructions here and used the jars of versions 4.9.1 and 4.9.2. With both versions, the input foobar^4 does not throw an error.

Also tried it with the plugin and no error is thrown.

If I input something like ^4, an error is thrown (as expected).

It seems like this might be an issue with the npm package - or is there anything else we should do to test our grammar file?