Semantic highlighting flickers

I’ve tried new semantic highlighting API in rust-analyzer (which was implemented by @kjeremy). I am super excited to see this feature finally in the LSP!

However, test drive shows this flickering:

I am not exactly sure, but I think the reason here is content modifies errors:

[Trace - 4:17:49 PM] Received response 'textDocument/semanticTokens - (437)' in 15ms. Request failed: content modified (-32801).

The editor asks for highlights, and then continues with sending modifications. Server than cancels the in-flight highlighting request. This causes the provider to return undefined, and that apparently clears the previous highlights. Because user continues typing, the highlighting is not re-applied until the user stops and the server is able to catch up.

It seems like the better behavior in this case is to just shift previous highlighting ranges, instead of clearing them. Not sure if this something to be fixed in vscode-languageserver-node, vscode itself, or the API for SemanticTokensProvider.

[alexdima]

@jrieken @dbaeumer Here is an idea. Currently, when a request is canceled by the client, we kind of expect and ignore (in error telemetry, etc.) any error with the message Canceled.

So what if we just extend this and allow that a Canceled error could be thrown even when the cancellation token is not triggered by the client… Then a server side cancelation could be detected by checking if the error is Canceled and if the cancelation token is not set.

[alexdima]

I don’t think vscode-languageserver-node can run a retry loop.

The VS Code request clearly expects the reply for a certain project state, the project state at which the request was made.

The whole point of the server canceling is that the server believes it makes no sense to answer the request at that certain state and the client should recreate a request at a new project state. This is nothing vscode-languageserver-node can do.

For example, for semantic tokens, when a request is made, text buffer edits are collected, such that when the response comes back, the response is adjusted against the user edits that occurred in the meantime. When a request is canceled (completed via a busy error), the collected text buffer edits are dropped and a new request is made, which is expected to return results at that new state.