Automatic classification: more aggressive debounce and perf considerations

Testing #129436

On my machine it takes around 300ms for the language classification to compute. However some of our users have a slower machine, so here are some ideas on how to make sure to not slow down when users that are just scribbling in an untitled editor.

• Do not classify when less than 20 characters present (unless to reset mode if file has auto detected already) - Done in https://github.com/microsoft/vscode-languagedetection/commit/b024b29a080c20b6875141797380dabb1bf3f672
• Increase debounce from 300ms to 600ms (this will still not be a noticeable lag when the user stops typing, pastes) - Done in https://github.com/microsoft/vscode/pull/130006
• If you have already run the classification n times and every time you are unsuccessful maybe stop trying and only on larger changes run
• If you already auto detected a language - great! Do not be so aggressive then, only if a big change to the file content happened

Both 3 and 4 require some knowledge about what is a bigger change to a file. We can figure something out. Maybe @bpasero has ideas

[rebornix]

Thanks @bpasero for bringing this to my attention. I don’t think having it running in main thread scales in terms of performance, especially now we are also adding bracket matching on typing. Adding a few ms doesn’t seem to hurt typing but if everyone is doing so, a single type can easily exceed 16ms and we can’t finish rendering in one single frame.

Note that we already have tokenization and multiple event listeners (auto save trigger), which take a couple of ms each.

[bpasero]

💯

We have to be very very careful of running an expensive operation in the same thread where the user is typing in (see https://github.com/microsoft/vscode/issues/27378 for example to get some inspiration). At the minimum we should debounce in a way that continuous typing further delays the computation up to the point where the user stops typing for a moment.

This becomes even more important if we decide to make language detection enabled by default.

Further thoughts:

• [x] would it make sense to move the language detection code into the untitled text editor model itself to possibly making the integration tighter? For example, I find the setMode change dubious with the fragile setExplicitly option which possibly would benefit from being dealt with internally. It is also weird to register a core workbench.editor.untitled.experimentalLanguageDetection setting that is only being used from a workbench contribution imho
• we seem to be creating a full string of the text model each time to detect the language - this can be very expensive for large models and at the very minimum should support stream based detection via the text model factory (see for example how backups are created for text files where we convert a snapshot of the text model to a byte stream here)
• can we explore moving the language detection into a worker and off the renderer process? I believe we do something similar when computing the diff of a file (@alexdima could possibly chime in on that matter or maybe @rebornix for how notebooks do this in src/vs/workbench/contrib/notebook/common/services/notebookWorkerServiceImpl.ts)