question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

What's the order of priority for Generic Tokens?

See original GitHub issue

I’m having trouble getting the string to tokenize properly because certain rules take precedence over others resulting in misclassification.

Is it the Generic Token Type and then the enum value? Or is it the highest enum value that gets tokenized first?

public enum StdToken
   {

       [Keyword("XL")]
       [Push("xlmode")]
       START_X = 100,        

       [Sugar("@")]
       [Push("dmode")]
       START_D = 101,
 
       [Lexeme(GenericToken.UpTo, ">>", "\0")]
       [Mode("dmode", "xlmode")]
       [Pop]
       OPS = 103,

       [Sugar(">>")]
       INSTALL = 15,

       [Sugar("(")]
       LPAREN = 40,

       [Sugar(")")]
       RPAREN = 41,

       [Keyword("Int")]
       [Keyword("String")]
       [Keyword("Optset")]
       [Keyword("Money")]
       [Keyword("Decimal")]
       [Keyword("EtnRef")]
       CONVERT = 104,

       [Sugar(":")]
       COLON = 105,
   }

Given the following: “XL(Key Metrics/C455) >> String: @working_adj.amount”,

It’s coming out as START_X OPS INSTALL CONVERT OPS - the last OPS tokenized as “: @working_adj.amount

when it should be: START_X OPS INSTALL CONVERT COLON START_D OPS

Issue Analytics

  • State:closed
  • Created 9 months ago
  • Comments:9 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
b3b00commented, Dec 7, 2022

thanks again for reporting @TamaBaka. I guess you are the first csly user to try the lexer modes feature using keywords are mode triggers.

1reaction
b3b00commented, Dec 7, 2022

@TamaBaka version 2.9.3.0 is now published. It solves your issue. Could you try and tell me if it’s ok for you ?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Who would be your top 3 token shop list priority for this ...
In this order: Thor, Drax, Beast, Gambit, Venom, Electro, Mister Negative, Dagger, Crossbones, Omega Red, Magik, Agatha, Hela, Titania, Typhoid ...
Read more >
Giving priority to a token over another - java
RULE 1: priority by the order it occurs while declaring tokens. RULE 2: priority based longest match possible.
Read more >
PLY (Python Lex-Yacc)
Tokens defined by strings are added next by sorting them in order of decreasing regular expression length (longer expressions are added first).
Read more >
Monster Token | Yu-Gi-Oh! Wiki - Fandom
Tokens are Monsters Special Summoned out of nowhere by a card effect. These monsters are not included ... This card can be used...
Read more >
FAQ: Hardware tokens and purchases
If the shipping option chosen for your order is "Regular post" (priority or with tracking) usually delivered within a week in Europe, US...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found