Unexpected execution order and failure with duplicated test names

I tried with Kotest 5.0 and later updates

The issue seems to be related to be triggered by two tests not on the same level having the same name. The following tests should work: each load() is properly followed by a complete(), and the tests pass on Kotest 4.x. However, on 5.x they fail with two exceptions: one somewhere in JUnit code, the other in the FakeService. Looks like either InstancePerLeaf is not respected, or the execution order is just messed up because of the duplicated names.

Renaming one of the duplicated test names causes the tests to pass, also removing some its (I think).

This behavior is unexpected, because the “duplicated” test names appear on different levels. My understanding was that the names are considered duplicate on the same tests level only.

Repro:

package com.example
​
import io.kotest.core.TestConfiguration
import io.kotest.core.spec.IsolationMode
import io.kotest.core.spec.style.DescribeSpec
import kotlinx.coroutines.CompletableDeferred
import kotlinx.coroutines.Job
import kotlinx.coroutines.cancel
import kotlinx.coroutines.launch
import kotlinx.coroutines.test.TestCoroutineScope
​
fun TestConfiguration.customTestScope() = TestCoroutineScope(Job()).also { testScope ->
    afterSpec {
        testScope.cancel()
        testScope.cleanupTestCoroutines()
    }
}
​
internal class Repro : DescribeSpec() {
​
    override fun isolationMode() = IsolationMode.InstancePerLeaf
​
    private val service = FakeService()
​
    init {
        describe("test") {
            val load: suspend () -> Unit = { runCatching { service.await() } }
​
            describe("foo") {
                customTestScope().launch { load() }
​
                it("a") {}
​
                it("b") {}
​
                describe("THIS TEST NAME IS DUPLICATED") {
                    service.complete()
​
                    it("a") {}
​
                    it("b") {}
                }
​
                describe("bar") {
                    service.complete()
​
                    it("a") {}
​
                    it("b") {}
​
                    describe("baz") {
                        customTestScope().launch { load() }
​
                        it("a") {}
​
                        it("b") {}
​
                        describe("THIS TEST NAME IS DUPLICATED") {
                            service.complete()
​
                            it("a") {}
​
                            it("b") {}
                        }
                    }
                }
            }
        }
    }
}
​
class FakeService {
​
    var deferred: CompletableDeferred<Unit>? = null
​
    suspend fun await() {
        if (deferred != null) error("Can't await without completing first")
​
        val cd = CompletableDeferred<Unit>()
        deferred = cd
​
        val value = runCatching { cd.await() }
        deferred = null
        value.getOrThrow()
    }
​
    fun complete() {
        deferred!!.complete(Unit)
    }
}

[sksamuel]

Fixed in master.

[lwasyl]

Yep, looks like exactly the same issue, thanks for pointing me to it! I’ll keep both opened just in case though, as both have clear repros that can be verified.