Defining a closure in a macro causes a compiler exception

Compiler version

3.0.0-RC3

Minimized example

import scala.quoted.*

object TestMacro {
  def use(f: () => String): Unit = ()

  inline def test: Unit = ${testImpl}

  def testImpl(using Quotes): Expr[Unit] = {
    import quotes.reflect.*

    def resultDefBody(): Term = '{
      val result: String = "xxx"
      result
    }.asTerm
    val resultDefSymbol = Symbol.newMethod(Symbol.spliceOwner, "getResult", MethodType(Nil)(_ => Nil, _ => TypeRepr.of[String]))
    val resultDef = DefDef(resultDefSymbol, { case _ => Some(resultDefBody()) })
    val resultExpr = Block(List(resultDef), Closure(Ref(resultDefSymbol), None)).asExprOf[() => String]

    //

    val r = '{ TestMacro.use($resultExpr) }
    println(r.asTerm.show(using Printer.TreeShortCode))
    r
  }
}

and then calling the macro:

object Test extends App {
  TestMacro.test
}

Output

I’m trying to generate code which passes a function as a parameter to a method. From what I’ve seen by printing the AST of quoted code, I need a Block with a DefDef as the statement, and a Closure as the expression. In the body of the function, I define a local variable - and this seems to be a problem. The original exception from the compiler I get is:

java.util.NoSuchElementException: val result while compiling Test.scala
[error] ## Exception when compiling 25 sources to /core/target/jvm-3.0.0-RC3/test-classes
[error] java.util.NoSuchElementException: val result
[error] scala.collection.mutable.AnyRefMap$ExceptionDefault.apply(AnyRefMap.scala:508)
[error] scala.collection.mutable.AnyRefMap$ExceptionDefault.apply(AnyRefMap.scala:507)
[error] scala.collection.mutable.AnyRefMap.apply(AnyRefMap.scala:207)
[error] dotty.tools.backend.jvm.BCodeSkelBuilder$PlainSkelBuilder$locals$.load(BCodeSkelBuilder.scala:507)

while trying to minify the example, I started getting another exception, so I’m not sure if my code reproduces the problem, but maybe it reproduces some problem 😃

Here’s the exception I get now:

java.lang.IllegalArgumentException: Could not find proxy for val result: String in List(val result, val <local Test$>, module class Test$, module class softwaremill, module class com, module class <root>), encl = (...)
[error] ## Exception when compiling 6 sources to /target/jvm-3.0.0-RC3/classes
[error] java.lang.IllegalArgumentException: Could not find proxy for val result: String in List(val result, val <local Test$>, module class Test$, module class softwaremill, module class com, module class <root>), encl = (...)
[error] dotty.tools.dotc.transform.LambdaLift$Lifter.searchIn$1(LambdaLift.scala:396)
[error] dotty.tools.dotc.transform.LambdaLift$Lifter.proxy(LambdaLift.scala:409)
[error] dotty.tools.dotc.transform.LambdaLift$Lifter.proxyRef(LambdaLift.scala:427)
[error] dotty.tools.dotc.transform.LambdaLift$Lifter.addFreeArgs$$anonfun$1(LambdaLift.scala:433)
[error] scala.collection.immutable.List.map(List.scala:246)

When printing the generated code, it looks fine:

TestMacro.use((() => {
  val result: String = "xxx"

  (result: String)
}))

Expectation

A way to generate a closure and pass it to a method, in a macro.

[LPTK]

@nicolasstucki is that really a problem? If the owner is changed only once when constructing the thing, as opposed to on every transformation of the tree. It’s only a constant cost to pay.

Should macro users who just want to quickly put some trees together have to care about owners, something the compiler should in principle be able to figure out on its own?

Symbol owners and owner chain corruptions were probably the number one cause for unreliability and crashes with Scala 2 macros. The entire thing was so complicated that it was basically impossible to get it right, from my experience (though at the time I tried to do this I was less experimented). Look at this old tutorial for example. Any deviation from the compiler’s internal notion of how things must be done led to horrible crashes. This is one of the main reason people ended up simply wiping types off the trees they received in macros, and just using untyped tree manipulations throughout their macros, which has its own set of problems.

[nicolasstucki]

The issue is that if we do it automatically, then users will probably not care about creating the trees with the correct owner. Then the norm would be to have to do an owner change.