add Update<R> to exercise and create method result types in Java codegen

What is the problem you want to solve?

Decoding is typeclass-driven in Scala codegen, so it is possible for users to put together type-safe utilities for working with choice results, like this one:

  import com.daml.ledger.client.{binding => sb}, sb.Primitive.Update
  import com.daml.ledger.api.v1.command_service.SubmitAndWaitRequest

  /** Assuming `ua` is an exercise from Scala codegen, submit, wait, and decode
    * the result according to the declared return type of the choice.
    */
  def performUpdate[A: sb.ValueDecoder](ledger: LedgerClient, ua: Update[A]): Future[Option[A]] =
    ledger.commandServiceClient
      .submitAndWaitForTransactionTree(
        // elided other details of the request
        SubmitAndWaitRequest(Some(Commands(commands = Seq(ua.command)))),
        None,
      )
      .map { swttr =>
        for {
          tree <- swttr.transaction
          exerciseEv <- tree.rootEventIds.collectFirst(Function unlift { id =>
            tree.eventsById.get(id).flatMap(_.kind.exercised)
          })
          resultValue <- exerciseEv.exerciseResult
          decoded <- sb.Value.decode(resultValue)
        } yield decoded
      }

Exercise methods in Scala codegen return Update[R] where R is the choice’s return type, so the correct ValueDecoder instance can be implicitly selected when using such a utility. Moreover, we can inspect the command, and choose a value to decode according to whether it is a create, exercise, or create-and-exercise, giving us a full natural transformation from Update[R] to Future[F[R]] for some functor F (Option used above for demo purposes.).

On the other hand, exercise methods in Java codegen return ExerciseCommand, ExerciseByKeyCommand, or such, with no indication of what the choice return type will be. Even if a phantom type parameter was added just to place the information somewhere, users cannot look up value-to-Java-codegen decoders implicitly in Java.

What is the solution you would propose?

Introduce an Update<R> class that all generated exercise* methods will return.

public Update<IouTransfer.ContractId> exerciseIou_Transfer(...) {...}

Update is a sum type of possible commands, as it effectively is in Scala codegen. This is a breaking change for existing callers that expect specific *Command types to be returned. Static evidence that your exercise call returned a particular subtype of command is significantly less valuable than static evidence of the Daml result of executing that command.

For create, we should be able to return CreateUpdate<ContractId>, where CreateUpdate<Z> <: Update<Z> & CreateCommand, to preserve compatibility.

Update<R> will carry a #14313 decoder for R, supplied by the codegenned exercise methods.

Describe alternatives you’ve considered

1. Have users assemble and pass in a decoder for R themselves. This will simplify some aspects of the exercise method generation, but it is significantly harder for users to do that, especially since we already have the tools in the codegen implementation to assemble decoders for all possible Rs that we can emit, and also given that we do not have a convenient combinator language for assembling decoders, i.e. #14313 (notwithstanding ContractCompanion, which is currently specific to templates).
2. Leave the type parameter off of Update, introducing a method that decodes to java.lang.Object which is “guaranteed” to be a value decoded to codegen. This allows us to avoid changing the return types at all. I elide description of the downside here.
3. Change the definition of Java to support Scala-like intersection types as method result types, so that we can return Update<R> & Cmd. This will allow existing code that statically expects ExerciseCommand et al to continue to do so. Codegen users would have to upgrade to the earliest version of Java that supports this, though. Additionally, it might take some time to get this feature through the Java feature pipeline.

Additional context

Inspired by a forum discussion with @juliuswilliam-da (though I do not think this would have genuinely applied to that use case). It has come up elsewhere on the forum, such as here.

This would permit natural transformation utilities like performUpdate to be written for Java codegen users. In my opinion, the inability to conveniently, type-safely extract exercise results (and well-typed create-contract-IDs) as you can in the Daml surface language is a significant barrier to e.g. writing “triggers” in Java instead of Daml, mentioned by @garyverhaegen-da yesterday. We can proof-of-concept such utilities with this feature, but the real value is making them practical to write at all.

It would be possible to add map2, pure, and map for Update to support submitting multiple commands in a single submit. This would significantly complicate the aforementioned utilities, but require no further codegen changes.

We would prefer to return Update<Z> & Cmd, but Java doesn’t have lower bounds on type parameters. Oh well!

[cocreature]

I think I’d just parse daml-lf for that instead of trying to generated java code.

[cocreature]

@cocreature That sounds exactly like what we’re looking for with the Java codegen. In the scenario of submitWithResult, does an actual exercise have to be submitted to the ledger to retrieve the exercise result for that choice?

Yes