Unexpected results for strings and doubles in expression parsing with generic lexer
See original GitHub issueHello,
I am getting unexpected results when using this configuration:
enum TestTokens
{
[String("'", "'")] STRING,
[Double] DOUBLE,
[Keyword("eq")] EQ,
}
class TestParser
{
public static Parser<TestTokens, Expression> GetParser()
{
var builder = new ParserBuilder<TestTokens, Expression>();
var result = builder.BuildParser(new TestParser(), ParserType.EBNF_LL_RECURSIVE_DESCENT, nameof(TestParser) + "_expressions");
return result.Result;
}
[Operand]
[Production("operand: STRING")]
public Expression LiteralString(Token<TestTokens> value)
=> Expression.Constant(value.StringWithoutQuotes);
[Operand]
[Production("operand: DOUBLE")]
public Expression LiteralDouble(Token<TestTokens> value)
=> Expression.Constant(value.DoubleValue);
[Operation((int)TestTokens.EQ, Affix.InFix, Associativity.Left, 1)]
public Expression BinaryOperator(Expression left, Token<TestTokens> operation, Expression right)
=> Expression.Equal(left, right);
}
The test and results:
var parser = TestParser.GetParser();
var result1 = parser.Parse("123.456");
// result1.Result is a ConstantExpression with Value=123456D
// -> the decimal separator was ignored?
var result2 = parser.Parse("'str1' eq 'str2'");
// result2.Result is a ConstantExpression with Value="str1'eq 'tr2"
// -> it should have been a BinaryExpression with ConstantExpression arguments with values "str1" and "str2"
Thank you
Issue Analytics
- State:
- Created a year ago
- Comments:10 (7 by maintainers)
Top Results From Across the Web
Why split lexing and parsing into two separate phases?
One benefit of splitting the lexer and parser is that you can write tests for the lexer that cover different lexical variations of...
Read more >[PATCH Rust front-end v4 14/46] gccrs: Add Parser for ...
+ + const_TokenPtr t = lexer.peek_token (); + /* TODO: should the switch just ... + "unexpected token %qs when parsing literal expression", ......
Read more >Is there a parsing library (lexer?) which can handle generic ...
I want to parse my own markup/programming language and I see it as firstly parsing characters into tokens (number, string, identifier, ...
Read more >Unexpected ANTLR4 optimization with INT and FLOAT?
1 Answer 1 ... ANTLR breaks your input into tokens, and only after that parses the tokens. Your use of 'clasp version 3.'...
Read more >A Guide To Parsing: Algorithms And Terminology
An in-depth coverage of parsing terminology an issues, together with an explanation for each one of the major algorithms and when to use...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Thank you very much for both fixes, I can confirm it’s now working as expected!
The sample code is only using LINQ expressions so a
using System.Linq.Expressions;
should be sufficient; I don’t have other custom classes.