question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Unexpected results for strings and doubles in expression parsing with generic lexer

See original GitHub issue

Hello,

I am getting unexpected results when using this configuration:

enum TestTokens
{
    [String("'", "'")] STRING,
    [Double] DOUBLE,
    [Keyword("eq")] EQ,
}

class TestParser
{
    public static Parser<TestTokens, Expression> GetParser()
    {
        var builder = new ParserBuilder<TestTokens, Expression>();
        var result = builder.BuildParser(new TestParser(), ParserType.EBNF_LL_RECURSIVE_DESCENT, nameof(TestParser) + "_expressions");
        return result.Result;
    }

    [Operand]
    [Production("operand: STRING")]
    public Expression LiteralString(Token<TestTokens> value)
        => Expression.Constant(value.StringWithoutQuotes);

    [Operand]
    [Production("operand: DOUBLE")]
    public Expression LiteralDouble(Token<TestTokens> value)
        => Expression.Constant(value.DoubleValue);

    [Operation((int)TestTokens.EQ, Affix.InFix, Associativity.Left, 1)]
    public Expression BinaryOperator(Expression left, Token<TestTokens> operation, Expression right)
        => Expression.Equal(left, right);
}

The test and results:

var parser = TestParser.GetParser();
var result1 = parser.Parse("123.456");
// result1.Result is a ConstantExpression with Value=123456D
// -> the decimal separator was ignored?

var result2 = parser.Parse("'str1' eq 'str2'");
// result2.Result is a ConstantExpression with Value="str1'eq 'tr2"
// -> it should have been a BinaryExpression with ConstantExpression arguments with values "str1" and "str2"

Thank you

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:10 (7 by maintainers)

github_iconTop GitHub Comments

2reactions
LaXiS96commented, Sep 5, 2022

Thank you very much for both fixes, I can confirm it’s now working as expected!

1reaction
LaXiS96commented, Sep 2, 2022

The sample code is only using LINQ expressions so a using System.Linq.Expressions; should be sufficient; I don’t have other custom classes.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Why split lexing and parsing into two separate phases?
One benefit of splitting the lexer and parser is that you can write tests for the lexer that cover different lexical variations of...
Read more >
[PATCH Rust front-end v4 14/46] gccrs: Add Parser for ...
+ + const_TokenPtr t = lexer.peek_token (); + /* TODO: should the switch just ... + "unexpected token %qs when parsing literal expression", ......
Read more >
Is there a parsing library (lexer?) which can handle generic ...
I want to parse my own markup/programming language and I see it as firstly parsing characters into tokens (number, string, identifier, ...
Read more >
Unexpected ANTLR4 optimization with INT and FLOAT?
1 Answer 1 ... ANTLR breaks your input into tokens, and only after that parses the tokens. Your use of 'clasp version 3.'...
Read more >
A Guide To Parsing: Algorithms And Terminology
An in-depth coverage of parsing terminology an issues, together with an explanation for each one of the major algorithms and when to use...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found