Use the analysis server for semantic highlighting

On the insiders channel, VS Code now provides apis for semantic token highlighting. This would allow Dart Code to enhance the static grammar-based highlighter with highlighting information from the analyzer.

In particular, the extension could use it to:

• be more accurate at semantic highlighting (members starting with an uppercase, named get/set, complex string literals, …)
• highlight non-Dart files via analysis server plugins

To my knowledge, LSP doesn’t support syntax highlighting (yet). So I think this would have to be implemented using the DAS protocol and a contributor for now. I’ve started some work on this and I hope I can open a PR next week.

Current issues

• Not available in a stable VS Code release
• The analysis server doesn’t highlight null and throw tokens, https://dart-review.googlesource.com/c/sdk/+/131071

[DanTup]

@simolus3 LSP v3.16 is due to be finalised imminently (hurrah!). I have started preparing to land changes in the server:

https://dart-review.googlesource.com/c/sdk/+/175722/

Once LSP 3.16 is final (and the next version of VS Code) I’ll make a new preview build of Dart-Code with the new client that will work with the server changes.

I’ve included a basic plugin test in that CL, though if there are cases you think it doesn’t cover, please let me know (or feel free to contribute changes).

[DanTup]

@simolus3 still haven’t forgotten this, but it’s still unfinalized in LSP so I’ve been holding off merging or doing any more. There are also a few things I’ve asked about, like supporting multiline/nested tokens that haven’t really been resolved yet.