Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Confusing block execution with isInstancePerTest=true

See original GitHub issue

I was trying to debug some test issues and was getting really confused following the code flow when isInstancePerTest is set to true. I really like isInstancePerTest as it makes ensuring state is correct a lot easier, but it’s confusing how many times each block is executed. In this example test:

internal class ClassUnderTestTest : ShouldSpec() {

    override fun isInstancePerTest(): Boolean = true

    init {
        "Scope 1" {
            println(this.context.description().name)
            "Scope 1a" {
                println(this.context.description().name)
                should("1aa") {
                    println(this.description().name)
                    1 shouldBe 1
                }
                "Scope 1ab" {
                    println(this.context.description().name)
                    should("1aba") {
                        println(this.description().name)
                        1 shouldBe 1
                    }
                }
            }
            "Scope 1b" {
                println(this.context.description().name)
                should("1ba") {
                    println(this.description().name)
                    1 shouldBe 1
                }
                "Scope 1bb" {
                    println(this.context.description().name)
                    should("1bba") {
                        println(this.description().name)
                        1 shouldBe 1
                    }
                }
            }
            "Scope 1c" {
                println(this.context.description().name)
                should("1ca") {
                    println(this.description().name)
                    1 shouldBe 1
                }
            }
        }
    }
}

With isInstancePerTest set to false, it prints:

Scope 1
Scope 1a
should 1aa
Scope 1ab
should 1aba
Scope 1b
should 1ba
Scope 1bb
should 1bba
Scope 1c
should 1ca

Which seems expected. With isInstancePerTest set to true, it prints:

Scope 1
Scope 1
Scope 1a
Scope 1
Scope 1a
should 1aa
Scope 1
Scope 1a
Scope 1ab
Scope 1
Scope 1a
Scope 1ab
should 1aba
Scope 1
Scope 1b
Scope 1
Scope 1b
should 1ba
Scope 1
Scope 1b
Scope 1bb
Scope 1
Scope 1b
Scope 1bb
should 1bba
Scope 1
Scope 1c
Scope 1
Scope 1c
should 1ca

Whereas I would have expected:

Scope 1
Scope 1a
should 1aa
Scope 1
Scope 1a
Scope 1ab
should 1aba
Scope 1
Scope 1b
should 1ba
Scope 1bb
should 1bba
Scope 1
Scope 1c
should 1ca

I’ve posted on here before about test execution order and I know it’s a tricky thing, but it’s pretty surprising to try and step through the code with the current way it executes using isInstancePerTest=true