ShouldSpec not executing a single test

class Foo : ShouldSpec({
    
    context("A") {
        should("B") {
            throw RuntimeException()
        }
    }
    
    should("C") {
        throw RuntimeException()
    }
})

When using the plugin, executing “A” runs both “A” and “B”. Executing “B” also runs “A” and “B”. Both fail as expected.

When running “C”, it won’t be executed by the plugin.

The only way to execute C is using the class, executing Foo

[lwasyl]

Same here for DescribeSpec, with a very simple repro:

Repro code

package com.example

import io.kotest.core.spec.IsolationMode
import io.kotest.core.spec.style.DescribeSpec

class TestTest : DescribeSpec() {

    init {
        describe("test") {
            println("Starting")

            describe("First describe") {
                println("first describe")

                it("should do something") {
                    println("assertion")
                }

                it("should do something else") {
                    println("second assertion")
                }
            }

            describe("Second describe") {
                println("second describe")

                it("should do something") {
                    println("assertion")
                }

                it("should do something else") {
                    println("second assertion")
                }
            }
        }
    }

    override fun isolationMode(): IsolationMode? = IsolationMode.InstancePerLeaf
}

and

debug output

invokeBeforeProject
~~~ Kotest Configuration ~~~
-> Parallelism: 1 thread(s)
-> Default test timeout: 600000ms
-> Default test order: Sequential
-> Default isolation mode: InstancePerLeaf
-> Global soft assertations: False
-> Write spec failure file: False
-> Fail on ignored tests: False
-> Spec execution order: LexicographicSpecExecutionOrder
-> Extensions
  - io.kotest.core.extensions.SystemPropertyTagExtension
  - io.kotest.core.extensions.RuntimeTagExtension
  - io.kotest.core.extensions.IgnoredSpecDiscoveryExtension
  - io.kotest.core.extensions.TagFilteredDiscoveryExtension
-> Listeners
  - com.example.test.utils.TestCoroutineDispatcherListener
  - beforeAfterAllListener$1

Submitting 1 specs
Waiting for specs execution to terminate
Executing spec class com.example.TestTest
Executing engine listener callback:specStarted class com.example.TestTest

Notifying 1 test listeners of callback 'prepareSpec'
'prepareSpec' callbacks complete
Executing engine listener callback:specInstantiated spec:com.example.TestTest@32da98d
Materialized roots: [RootTest(testCase=TestCase(description=Description(parents=[TestName(prefix=null, name=com.example.TestTest, focus=false, bang=false)], name=TestName(prefix=Describe: , name=test, focus=false, bang=false)), spec=com.example.TestTest@32da98d, test=io.kotest.core.test.TestContext.() -> kotlin.Unit, source=SourceRef(lineNumber=9, fileName=TestTest.kt), type=Container, config=TestCaseConfig(enabled=true, invocations=1, threads=1, timeout=null, invocationTimeout=null, tags=[], listeners=[], extensions=[], enabledIf=(io.kotest.core.test.TestCase) -> kotlin.Boolean), factoryId=null, assertionMode=null), order=0)]
Waiting for test case execution to terminate
invokeBeforeSpec com.example.TestTest@7818867d
Created new spec instance com.example.TestTest@7818867d
Materialized roots: [RootTest(testCase=TestCase(description=Description(parents=[TestName(prefix=null, name=com.example.TestTest, focus=false, bang=false)], name=TestName(prefix=Describe: , name=test, focus=false, bang=false)), spec=com.example.TestTest@7818867d, test=io.kotest.core.test.TestContext.() -> kotlin.Unit, source=SourceRef(lineNumber=9, fileName=TestTest.kt), type=Container, config=TestCaseConfig(enabled=true, invocations=1, threads=1, timeout=null, invocationTimeout=null, tags=[], listeners=[], extensions=[], enabledIf=(io.kotest.core.test.TestCase) -> kotlin.Boolean), factoryId=null, assertionMode=null), order=0)]
Starting root test Description(parents=[TestName(prefix=null, name=com.example.TestTest, focus=false, bang=false)], name=TestName(prefix=Describe: , name=test, focus=false, bang=false)) in search of Description(parents=[TestName(prefix=null, name=com.example.TestTest, focus=false, bang=false)], name=TestName(prefix=Describe: , name=test, focus=false, bang=false))
com.example.TestTest Describe: test is excluded by test case filters [[(io.kotest.runner.console.TestPathTestCaseFilter@396d6bd2, Exclude)]]
com.example.TestTest -- test is *not* active
invokeAfterSpec com.example.TestTest@7818867d
Executing notifyFinalizeSpec
Executing engine listener callback:specFinished class com.example.TestTest

Spec executor has terminated
invokeAfterProject

That’s with Kotest 4.1.1, IJ plugin 1.0.5 and Android Studio 4.2 Canary 4 (Build #AI-201.7846.76.42.6636798).

What’s interesting though is that if I run the test contextually (Run context configuration action, ctrl+shift+r) with cursor at a specific it test, it will run only that one test

[kharf]

Same with BehaviorSpec. kotest 4.1.1, plugin 1.0.5.

Running gutter icons “given”, “when”, “then” all create run/debug configurations, but won’t execute. Only log message you see in Test Results is:

io.kotest.runner.console.LauncherKt --spec com.package.ExampleSpec --testpath Given: something
~~~ Kotest Configuration ~~~
-> Parallelism: 1 thread(s)
-> Default test timeout: 600000ms
-> Default test order: Sequential
-> Default isolation mode: SingleInstance
-> Global soft assertations: False
-> Write spec failure file: False
-> Fail on ignored tests: False
-> Spec execution order: LexicographicSpecExecutionOrder
-> Extensions
  - io.kotest.core.extensions.SystemPropertyTagExtension
  - io.kotest.core.extensions.RuntimeTagExtension
  - io.kotest.core.extensions.IgnoredSpecDiscoveryExtension
  - io.kotest.core.extensions.TagFilteredDiscoveryExtension