Shapeless 2 on Dotty

This is a general meta-issue to aggregate the overall project of getting a version of Shapeless 2 working on Dotty.

Goals

• Rough source-compatibility with the Shapeless 2.x versions of any constructs which can be meaningfully expressed on Dotty
• Full source-compatibility with the Scala 2.13 variants of the same construct in the same release. In other words, it should be possible to move from 2.13 to 3.0 simply by swapping scalaVersion for any code which exclusively uses the supported constructs

Non-Goals

• Perfect binary compatibility with the 2.x line (though, the closer to this we can get, the better)
• Ports of less-used constructs, particularly those which are impossible to render on Dotty

Requirements

Each of these should be split out into a separate issue:

• Generic
• HList
• Coproduct
• Polymorphic functions

Comments and opinions very much desired on all of the above!

To be clear, I’m well aware that most of this is covered by out-of-the-box functionality in Dotty (and shapeless 3 is a nice layer on top of that). However, any project which is cross-publishing between Scala 2 and Scala 3 will benefit considerably from this type of functionality. Additionally, cross-publication is a special case of migration, and any project which is using shapeless already will either need something like this, or it will need to recreate this type of functionality on their own for bespoke cross-building (i.e. what Circe had to do).

[Katrix]

Ok, so progress is happening on this finally. So far I’ve gotten a version of Shapeless 2 compiling on both Scala 2 and 3, but where all macros are dummied out. No idea how well it works after it compiles.

One big discussion which needs to be had is what to do with everything that can’t be cleanly ported to Scala 3? This is mostly macros that do things Scala 3 macros can’t or types not representable in Scala 3. As I can see, there are a bunch of similar macros, which can be grouped and talked about together.

1. Accesses the outer scope of the macro call (cachedImplicit is one example that does this from a quick glance at it. Also not sure how feasible it would be to implement in Scala 3 with my limited knowledge of what you can do there.)

2. Type providers of all sorts. Think Union, Witness, Record, HList, Coproduct. Problem here is mainly constructing a type from a string, which is not something I think Scala 3 exposes.

3. Things that use untyped trees. Think RecordArgs, FromRecordArgs, NatProductArgs, ProductArgs, FromProductArgs, SingletonProductArgs. These macros generally convert calls to a method into calls to a different method, while manipulating the arguments. For example for RecordArgs, lhs.method(x = 23, y = "foo", z = true) becomes lhs.methodRecord("x" ->> 23 :: "y" ->> "foo", "z" ->> true). I don’t think Scala 3 macros offers a way to redirect method calls in such a dynamic manner.

4. Stuff that in some way manipulates source code in ways not exposed by the Scala 3 compiler. Think TypeOf, compileTime. This class of macros in some way serve as a poor mans inline, but manipulates the code as a string. As such I can’t see any reasonable way to support it.

There are also the type quantifiers Shapeless exposes, which can be seen here. https://github.com/milessabin/shapeless/blob/main/core/src/main/scala/shapeless/package.scala#L60

∃ I am fairly certain is not representable in Scala 3. ∀ might still be possible? If it is, it needs a new representation.

It’s clear that none of these can be included in the Scala 3 artifact, but should they still be included in the Scala 2 artifact?

My current progress on porting Shapeless 2 to Scala 3 can be found here for anyone interested. https://github.com/Katrix/shapeless/tree/feature/scala-3-port

[DmytroMitin]

https://stackoverflow.com/questions/69483366/scala-3-collection-partitioning-with-subtypes https://stackoverflow.com/questions/74355212/shapeless3-and-annotations