What's the order of priority for Generic Tokens?
See original GitHub issueI’m having trouble getting the string to tokenize properly because certain rules take precedence over others resulting in misclassification.
Is it the Generic Token Type and then the enum value? Or is it the highest enum value that gets tokenized first?
public enum StdToken
{
[Keyword("XL")]
[Push("xlmode")]
START_X = 100,
[Sugar("@")]
[Push("dmode")]
START_D = 101,
[Lexeme(GenericToken.UpTo, ">>", "\0")]
[Mode("dmode", "xlmode")]
[Pop]
OPS = 103,
[Sugar(">>")]
INSTALL = 15,
[Sugar("(")]
LPAREN = 40,
[Sugar(")")]
RPAREN = 41,
[Keyword("Int")]
[Keyword("String")]
[Keyword("Optset")]
[Keyword("Money")]
[Keyword("Decimal")]
[Keyword("EtnRef")]
CONVERT = 104,
[Sugar(":")]
COLON = 105,
}
Given the following: “XL(Key Metrics/C455) >> String: @working_adj.amount”,
It’s coming out as START_X OPS INSTALL CONVERT OPS - the last OPS tokenized as “: @working_adj.amount”
when it should be: START_X OPS INSTALL CONVERT COLON START_D OPS
Issue Analytics
- State:
- Created 9 months ago
- Comments:9 (5 by maintainers)
Top Results From Across the Web
Who would be your top 3 token shop list priority for this ...
In this order: Thor, Drax, Beast, Gambit, Venom, Electro, Mister Negative, Dagger, Crossbones, Omega Red, Magik, Agatha, Hela, Titania, Typhoid ...
Read more >Giving priority to a token over another - java
RULE 1: priority by the order it occurs while declaring tokens. RULE 2: priority based longest match possible.
Read more >PLY (Python Lex-Yacc)
Tokens defined by strings are added next by sorting them in order of decreasing regular expression length (longer expressions are added first).
Read more >Monster Token | Yu-Gi-Oh! Wiki - Fandom
Tokens are Monsters Special Summoned out of nowhere by a card effect. These monsters are not included ... This card can be used...
Read more >FAQ: Hardware tokens and purchases
If the shipping option chosen for your order is "Regular post" (priority or with tracking) usually delivered within a week in Europe, US...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
thanks again for reporting @TamaBaka. I guess you are the first csly user to try the lexer modes feature using keywords are mode triggers.
@TamaBaka version 2.9.3.0 is now published. It solves your issue. Could you try and tell me if it’s ok for you ?