Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Allow tokens to toggle lexing modes.

See original GitHub issue

Proposal: Allow tokens to push or pop a lexing mode, depending on if we’re already in the mode.

Example API:

const CodeFence = createToken({
  name: "CodeFence",
  pattern: {
    exec: () => {
      // ...
    },
  },
  toggle_mode: "codeblock",
});

If I wanted to do this with current API I’d have to define to separate, possibly identical tokens.

Issue Analytics

State:
Created 5 years ago
Comments:7 (7 by maintainers)

Top GitHub Comments

1reaction

bd82commented, Jul 21, 2018

I am closing this for now, I’d like to see a scenario that cannot be implemented with the current level of abstraction before we implement a new abstraction 😄

0reactions

brainkimcommented, Jul 22, 2018

Makes sense. Figured out a way to do this without needing toggle_mode. Thanks!

Top Results From Across the Web

How can I use the same tokens in different lexer modes in Antlr?

Show activity on this post. You'll have to duplicate some things into 2 different modes. You could define common tokens, like the NAME...

antlr4/lexer-rules.md at master - GitHub

Lexical modes allow us to split a single lexer grammar into multiple sublexers. The lexer can only return tokens matched by rules from...

How OSH Uses Lexer Modes - Oil Shell

This is why we need lexer modes. Most lexers look like this: Token Read() // return the next token. But our modal lexer...

Using a modal lexer for parsing sublanguages

Enter modal lexers. These lexers can switch between different modes for lexing different languages. In this article, I'll run you through an example...

Can I programmatically change lexer modes? - Google Groups

You can call pushMode any time, but be aware that it will only affect the next token, not the current token (unless you...