Monarch grammar: Regex starting with ^ should only match start of source line, but it does not.

monaco-editor version: 0.19.3 Browser: Chrome 79 OS: macOS Playground code that reproduces the issue:

According to the Monarch docs:

If it starts with a ^ character, the expression only matches at the start of a source line.

I have tried both specifying a regex literal /^!/ as well as a string "^!", but neither works.

You can reproduce the issue on the special Monarch playground at https://microsoft.github.io/monaco-editor/monarch.html. In the Language syntax definition, specify:

return {
  // The main tokenizer for our languages
  tokenizer: {
    root: [
      [/^\!/, {token: 'delimiter.curly', next: 'jsonInBang', nextEmbedded: 'json'}],
    ],
    jsonInBang: [
      [/^\!/, {token: 'delimiter.curly', next: '@pop', nextEmbedded: '@pop'}],
    ],
  },
};

And then in the Language editor, specify:

!
  {
    "foo": "bar",
    "baz": 42,
    "quux": [
      {
        "foo": "bar!",
      }
    ],
    "quux2": [
      {
        "foo": "bar",
      }
    ]
  }
!

If you look carefully, the JSON inside the ! starts out being highlighted correctly, but after the ! in the string literal "bar!", it stops working. It appears the /^\!/ has matched it even though it is not the start of a source code line, which is not what the docs claim should happen.

[alexdima]

For testing changes like that, I typically use one of the generated files like at http://127.0.0.1:8080/monaco-editor/test/playground.generated/extending-language-services-custom-languages.html

and just edit the .html to contain the Monarch definition I want and the content I want and then just reload…

[bolinfest]

@alexdima I had that working for the establish test pages, but it did not seem to be working for the playground (which I cared about because I didn’t see a good test page for grammars, so I wanted to use monarch.html). Should the fast edit/refresh cycle be working for the playground, too?