How to debug the compilation error "Cannot extract value from singleton.ops.impl.OpMacro"?

Hi @fthomas @soronpo Thanks a lot for the great work, I’m one of the supporting developer of the deep learning library kotlingrad. When we try to integrate singleton-ops for the project on some tensor operations, we encounter some strange compiler behaviours that are hard to isolate or simplify, so we have to post a link to our repository in case others want to reproduce, sorry.

In one of our test cases: https://github.com/tribbloid/shapesafe/blob/59405baedc9048a9cac1fbf2bb7708c917266a6e/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala#L167

The compiler / macro misbehave on 2 apparently equivalent code blocks:

      val v0 = DoubleVector.random(6)
      val v1 = v0.pad(3)

      {
        val result = v1.reify

        val aa = result.arity

        {
          print_@(WideTyped(result.arity).viz)
          // this works
          result.arity.internal.dummyImp(3)
          result.crossValidate()
          result.arity.internal.requireEqual(12)
        }

        {
          print_@(WideTyped(aa).viz)
          // this doesn't, how did it happened?
          aa.internal.dummyImp(3)
          aa.internal.requireEqual(12)
        }
      }

These are the compilation errors:

[Error] /home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala:188: implicit error;
!I proof (Cannot extract value from singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.+,S,Int(3),Int(0)]
showRaw==> TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(3)), FoldableConstantType(Constant(0))))): OpMacro[+, S, Int, Int]
[Error] /home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala:189: implicit error;
!I proof (Cannot extract value from singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.Require,singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.==,S,Int(12),Int(0)],String("Cannot prove requirement Require[...]"),singleton.ops.impl.NoSym]
showRaw==> TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.Require, List()), TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$eq$eq, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(12)), FoldableConstantType(Constant(0)))), FoldableConstantType(Constant(Cannot prove requirement Require[...])), TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.NoSym, List())))): OpMacro[Require, OpMacro[==, S, Int, Int], String, NoSym]
two errors found

> Task :core:compileTestScala FAILED

These are the same error if the implicit debugging plugin “splain” (https://github.com/tek/splain) is disabled:

[Error] /home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala:188: Cannot extract value from singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.+,S,Int(3),Int(0)]
showRaw==> TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(3)), FoldableConstantType(Constant(0))))
[Error] /home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala:189: Cannot extract value from singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.Require,singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.==,S,Int(12),Int(0)],String("Cannot prove requirement Require[...]"),singleton.ops.impl.NoSym]
showRaw==> TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.Require, List()), TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$eq$eq, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(12)), FoldableConstantType(Constant(0)))), FoldableConstantType(Constant(Cannot prove requirement Require[...])), TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.NoSym, List())))
two errors found

> Task :core:compileTestScala FAILED

My questions are:

1. What’s the cause of this error? It clearly won’t be caused by a missing dependent type (variable aa has a static path, but result.arity doesn’t) So how is the macro unable to figure out the type for aa?

2. What should we do for any error that has such message? Should we debug or try to print the stacktrace?

[soronpo]

You need W.`XXX`.T https://scastie.scala-lang.org/pL4vfrNISziWcKHUBlX8fA

[tribbloid]

@soronpo I’m so sorry, after 2 weeks I no longer remember how shapeless works 🤦

I think I’ve finally isolate the problem, all caused by this.type as an abstract type:

import shapeless.Witness
import singleton.ops.+

import scala.language.implicitConversions

object ErrorCase0 {

  trait Proof extends Serializable {

    type Out <: Arity
    def out: Out
  }

  trait Arity extends Proof {

    override type Out = this.type
    override def out: Out = this
  }

  object Arity {

    trait Const[S] extends Arity with Proof {

      type SS = S
    }

    case class FromLiteral[S <: Int]() extends Const[S] {}
  }

  val v1 = Arity.FromLiteral[Witness.`6`.T]()
//  val f1 = Arity(6)
  implicitly[v1.SS + Witness.`3`.T]

  val v2 = v1.out
  implicitly[v2.SS + Witness.`3`.T] // this fails
}

https://scastie.scala-lang.org/tribbloid/yRXvkpWWRiWAyGHzpI9g1Q/1

Scalac was known for fumbling on this.type inference. This issue may be one instance of such fumbling and may won’t be fixed without some major compiler overhaul. Regardless, the above case may worth further investigation to be 100% sure of the cause

For the moment I’ll just replace the abstract type with an F-bounded type parameter to circumvent it. Thank you so much for your instructions!