Inconsistent cancellation and lost exceptions

Consider following snippet:

val job = async() {
    barrier.await()
    throw IOException()
}

barrier.await()
job.cancel(TestException())
job.await() // <- 1

1 will always throw IOException and it will have eitherCompletedExceptionally or Cancelled state. TestException is always lost even when cancel call was successful.

Now slightly rewrite it:

val job = async() { 
   barrier.await()
   withContext(wrapperDispatcher(coroutineContext)) { // Just a wrapper to avoid fast paths in implementation
      throw IOException()
   }
}

barrier.await()
job.cancel(TestException())
job.await() // <- 2

Now 2 now can throw JobCancellationException: Job is being cancelled which definitely shouldn’t be a terminal exception. Depending on timing it also may lose either TestException or IOException (even when its known that IOException was thrown).

What user may expect

1. When it’s known that IOException was thrown, await() should throw it as well. If concurrent cancellation with the cause was successful, IOException should have its cause as suppressed exception
2. When it’s known that IOException wasn’t thrown (cancellation was “first”), await should throw TestException
3. JobCancellationException should never be thrown if the cause is present on all code paths and the job is in its final state
4. No intermediate JobCancellationException should be present in the final state of Job

We should design, rework and document exception handling mechanism. We can give up some performance on an exceptional path (especially when multiple exceptions are thrown), but regular code path should stay the same

[vadymhimself]

I was able to force-throw an exception out of the coroutine on android using the main thread Handler.

[qwwdfsad]

@bolein I will update our guide before merging it. The general idea is to make exception handling “synchronous” to make parallel decomposition transparent.

E.g. for code

launch(MyExceptionHandler) {
    launch(coroutineContext) {  throw ChildException() }
    throw ParentException()
}

MyExceptionHandler will receive one of the exceptions (depending on timing) and another one in suppressed exceptions. For jobs hierarchy exception is reported only once by root job, having all intermediate exceptions as suppressed, exceptions are never swallowed.

Other changes:

1. Custom exception handler can’t override parent cancellation behavior
2. JobCancellationException are always ignored and used for internal machinery

Note that it’s work in progress and anything can change.

It’s going into release as soon as it’s ready. It has a lot of corner cases and is breaking change, so we should internally evaluate it first.