Scala staging not releasing memory

Compiler version

3.0.0

Minimized example

while true do
  run(iGenAnExpr)

Output

Lots of stuff in the JVM’s old gen heap

Expectation

Thrown away compiled code is freed.

@nicolasstucki

I believe the issue is in the QuotesDriver. In the above code, I generate a lot of code via runtime MSP, and then never use it. I had thought this would result in memory being freed, but apparently there’s some long-lived storage in the compiler that I haven’t found yet, because the above example causes old gen to rapidly fill.

[smarter]

Calling reset on the base context is not enough, the leak comes from val definitions.

More precisely it comes from the baseTypeCache of some denotations: https://github.com/lampepfl/dotty/issues/12543#issuecomment-849607084 which could be turned into a WeahHashMap (and also maybe shouldn’t cache termrefs at all?)

[nicolasstucki]

Not sure if we should change anything in the caching mechanism for staging. It provides the flexibility to either cache and run fast, or not cache and run a bit slower without consuming to much memory.

The optimizations we should be looking at are in the Context itself which are things that also affect the normal compiler. I saw may trees retained which are probably trees attached to symbols, these could in theory be dropped and recomputed.