Allow rewriting on types

We had a discussion recently about the problem to represent the type of vector concat. If we assume a type Vector[Elem, Len <: Nat], it is straightforward to implement a typelevel

rewrite def concat[A, M, N](xs: Vec[A, M], ys: Vec[A, N]): Vec[A, _]

such that the result of expanding concat for concrete vectors is a vector whose length is known to be the sum of the lengths of the argument vectors. But, annoyingly, there’s no simple way to declare this fact in the result type of concat. One can fallback to implicits, i.e. something like

rewrite def concat[A, M, N](xs: Vec[A, M], ys: A[A, N])(implicit ev: Add[M, N]): Vec[A, ev.Result]

But that’s a rather roundabout way to achieve something that should be simple.

The proposal here is to allow rewrite not only on terms but also on types. It’s sort of the dual of #4939 in that it caters for computations that produce types whereas #4939 is a way to consume types directly in typelevel computations.

To solve the example above, one would write

rewrite type Add[M <: Nat, N <: Nat] <: Nat = type M match {
  case Z => N
  case S[type M1] => S[Add[M1, N]]
}

This assumes type matches as proposed by #4939, but of course one could also use their expansion:

rewrite type Add[M <: Nat, N <: Nat] <: Nat = rewrite erasedValue[M] match {
  case _: Z => N
  case _: S[type M1] => S[Add[M1, N]]
}

Unlike #4939, this feature is not simply encodable in the base language, so one has to explain its properties in detail:

Question: What is the syntax of a rewrite type definition?

Answer: The same as a rewrite def, except that it’s type instead of def, the (optional) declared result type is prefixed by <: instead of : and the right hand side is a type expression. Type expressions are formed from

• blocks { ... }, ending in a type expression
• rewrite, type, and implicit matches, with type expressions in each case
• rewrite conditionals, with type expressions in each branch
• types

If a declared result type is missing, Any is assumed.

Question: When is a rewrite type application expanded?

Answer: Analogous to a rewrite term application, it is expanded on each use outside a rewrite method or type. In particular this means that in

rewrite def concat[A, M, N](xs: Vec[A, M], ys: A[A, N]): Vec[A, Add[M, N]]

the Add[M, N] is expanded only when concat is applied itself. Trying to expand it at the definition side would not work anyway, as the type match could not be reduced.

Question: How are applications of rewrite types handled when seen from inside a rewrite definition? In this case they cannot be expanded, so we have to have a fallback to represent them directly.

Answer: A rewrite type application T[S1, ..., Sn] in a rewrite definition is represented as an abstract type with the declared return type of T[S1, ..., Sn] as upper bound and Nothing as lower bound.

Question: How is the conformance of the RHS to the return type checked?

Answer When checking conformance of a RHS the the declared return type, we do expand rewrite types according to the information found in the branch. E.g. if we implement concat like this:

rewrite def concat[A, M, N](xs: Vec[A, M], ys: Vec[A, N]): Vec[A, Add[M, N]] = {
  xs match {
    case x :: xs1 => x :: concat(xs1, ys)
    case Nil => ys
  }
}

we should be able to reason as follows:

• In the case x :: xs1 we have xs1: Vec[A, M1] such that M = S[M1]
• By reading off the declared result type, concat(xs1, ys): Vec[A, Add[M1, N1]]
• By interpreting ::: concat(xs, ys): Vec[A, S[Add[M1, N1]]
• By rewriting Add[M, N] = Add[S[M1], N] = S[Add[M1, N]]

The Nil case is similar but simpler.

This part looks contentious. One can hope that we will have usually sufficient reasoning power to establish that a RHS conforms to its declared type (of course casts are available as a last resort if this fails). As an alternative we could also give up, and prove the correspondence of result to result type only at expansions. But this feels a bit like cheating…

[EDIT] I think it’s likely we’ll need to cheat. Trying to do the inductive reasoning outlined above would mean that rewrite types should be simplified whenever possible. And that’s a whole different game. If we stick to the rule that, like rewrite terms, rewrite types are expanded only outside rewrite definitions, the logical consequence is that checking rewrite types is also only done when they are expanded.

[Blaisorblade]

In the context of verification or proof assistant, I think you are right. But for real-world application or at least the two applications above, I doubt that.

But whenever you use dependent types, checking that two types are compatible can require checking/proving equality of values; even for the concat in the OP qualifies you’re doing verification, it’s just a lucky example where all the proof obligations follow immediately by normalization. That fails in this example:

xs match { // Vec[A, 1 + n]
 case x :: xs => xs ++ List(x) // Vec[A, n + 1]
}

There you already need 1 + n = n + 1, which doesn’t follow by normalization.

That’s not very compelling, so let’s look at a recent example I saw on Twitter, it seems somebody tried to turn a rectangular (sized) vector into a vector of vectors. As best as I can recover the code from the error message (or make it up) and translate it into Scala, the code involved seems to be:

def chunks(xs: Vec[A, o * p]): Vec[Vec[A, p], o] = {
  // ...
  for {
    i <- (0 until o).toVec
  } yield {
    for {
      j <- (0 until p).toVec
    } {
      xs(i * p + j)
    }
  }
}

Here’s the resulting compiler error in Haskell: https://twitter.com/lexi_lambda/status/1029918793034805248

For extra fun, ensuring the above is correct goes even beyond Presburger arithmetic, so I’m not sure how to verify it.

EDIT: yet another example in the wild comes from writing mergesort: https://stackoverflow.com/q/51919602/53974. (Wasn’t even looking for either example BTW, just ran into both ones in the last couple days 😉).

[liufengyun]

If one goes that road, and you want to typecheck definitions rather than just their uses, sooner or later you’ll need the typechecker to understand 1 + (x - 1) = x, etc.

In the context of verification or proof assistant, I think you are right. But for real-world application or at least the two applications above, I doubt that. An advantage of the approach is that Dotty already has all the infrastructure to support that, no extension of core types required if I were correct.