Compiler service: file parsing should use caching.

Is your feature request related to a problem? Please describe.

IDE interactions are slow, partly because the same files with unmodified contents are parsed multiple times, with no caching mechanism.

Given a project with three files: A.fs, B.fs, C.fs, every time I:

1. Modify A.fs,
2. Open C.fs, file B.fs is parsed again. Files A.fs and C.fs are probably parsed again too, although I’m not sure exactly when they are, and the main point is that all files for which typechecking is performed, are parsed again.

Note that I’ve only tested this in Rider.

Describe the solution you’d like

Introduce a caching mechanism with size configurable by consumers of FCS, which avoids parsing the same files again if nothing relevant has changed.

Additional context

Note that there is a parsing cache, but it’s only utilised by the dedicated file parsing endpoint, and is not used when the typechecking endpoint is called instead.

Also note that there seem to be multiple ways in which parsing is called, with different stack traces. The cache should address all of them if possible.

Also note that if this ends up using SourceText.GetHashCode, that method is quite slow as it has to scan the whole source. It might be worth considering calculating the hash once upon creation of SourceText.

Related discussion I started: https://github.com/dotnet/fsharp/discussions/14199

Related: Idea of parallel parsing in the FCS. I briefly looked at the possibility of parsing project files in parallel in FCS. However, the complexity of IncrementalBuilder makes it rather difficult. Also, once we have a caching mechanism with a large-enough size, the marginal gain of parallel parsing would be limited to speeding up the first-time typechecking after starting the IDE.

[safesparrow]

Instead a new SyntaxTree is being instantiated on every edit. Some parse calls are there for diagnostics and there is a lot of setting things up involved (side-effects, too, I believe). It does not look practical to cache every call in a centralized way.

I think we need to get on the same page. This issue is primarily about parsing files as part of a chain of parse+typecheck operations. Please have a look at the example in the description with the three file project.

Parsing on edit is different. There is little point caching those requests as the chance of seeing exactly the same source code upon multiple edits is small. Compare it to parsing all files in the project after only one of them has changed - all but one request could be cached. I believe the “active file edit” scenario is also why some endpoints have the cache argument which is sometimes set to false by the caller.

[auduchinok]

It would be nice if ParseAndCheckFileInProject could cache previous files trees in some cache indeed.

I believe the “active file edit” scenario is also why some endpoints have the cache argument which is sometimes set to false by the caller.

Yes, exactly. We’ve added it when started doing additional parsing that we didn’t want to be cached anywhere.