Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Spec-level invocation timeouts are not overriden at the test level

See original GitHub issue

Which version of Kotest are you using 4.3.0

Reproduction

@ExperimentalTime
class TimeoutTest : FreeSpec({

    val specLevelTimeout = 5.minutes
    val specLevelInvocationTimeout = 5.seconds

    timeout = specLevelTimeout.toLongMilliseconds()
    invocationTimeout = specLevelInvocationTimeout.toLongMilliseconds()

    val sleepDuration = 10.seconds
    "all tests have spec-level invocationTimeout of $specLevelInvocationTimeout and a timeout of $specLevelTimeout" - {

        val testLevelTimeout = 2.minutes
        val testLevelInvocationTimeout = 30.seconds

        "test has $testLevelInvocationTimeout invocation timeout and $testLevelTimeout timeout"
            .config(invocationTimeout = testLevelInvocationTimeout, timeout = testLevelTimeout) {
            println("Running test. About to sleep at ${System.currentTimeMillis()} for $sleepDuration")
            Thread.sleep(sleepDuration.toLongMilliseconds())
            println("Finished sleeping and running test at ${System.currentTimeMillis()}")
        }
    }
})

Setup Spec-level invocation timeout is 5 seconds but test-level invocation timeout is 30 seconds

Expected Test to pass, as test-level invocation timeout overrides according to https://kotest.io/timeout/

Timeouts can be specified at the spec level for every test in that spec, unless overriden by the test case itself.

Actual Test fails with

Test did not complete within 120000ms
TimeoutException(duration=120000)
	at io.kotest.core.internal.TestCaseExecutor.executeAndWait(TestCaseExecutor.kt:231)
	at io.kotest.core.internal.TestCaseExecutor.invokeTestCase(TestCaseExecutor.kt:176)
	at io.kotest.core.internal.TestCaseExecutor.executeActiveTest(TestCaseExecutor.kt:145)
	at io.kotest.core.internal.TestCaseExecutor$intercept$innerExecute$1$1.invokeSuspend(TestCaseExecutor.kt:83)
	at io.kotest.core.internal.TestCaseExecutor$intercept$innerExecute$1$1.invoke(TestCaseExecutor.kt)
	at io.kotest.core.internal.TestCaseExecutor.executeIfActive(TestCaseExecutor.kt:109)
	at io.kotest.core.internal.TestCaseExecutor$intercept$innerExecute$1.invokeSuspend(TestCaseExecutor.kt:83)
	at io.kotest.core.internal.TestCaseExecutor$intercept$innerExecute$1.invoke(TestCaseExecutor.kt)
	at io.kotest.core.internal.TestCaseExecutor.intercept(TestCaseExecutor.kt:97)
	at io.kotest.core.internal.TestCaseExecutor.execute(TestCaseExecutor.kt:63)
	at io.kotest.engine.runners.SingleInstanceSpecRunner.runTest(SingleInstanceSpecRunner.kt:73)
	at io.kotest.engine.runners.SingleInstanceSpecRunner$Context.registerTestCase(SingleInstanceSpecRunner.kt:51)
	at io.kotest.core.internal.TestCaseExecutor$executeAndWait$2$1$1$3$1$contextp$1.registerTestCase(TestCaseExecutor.kt:219)
	at io.kotest.core.test.TestContext$DefaultImpls.registerTestCase(TestContext.kt:40)
	at io.kotest.core.internal.TestCaseExecutor$executeAndWait$2$1$1$3$1$contextp$1.registerTestCase(TestCaseExecutor.kt:217)
	at io.kotest.core.spec.style.scopes.TestWithConfigBuilder.config-dVAIICY(TestWithConfigBuilder.kt:52)
	at io.kotest.core.spec.style.scopes.FreeScope.config-06zEaDw(FreeScope.kt:61)
	at io.kotest.core.spec.style.scopes.FreeScope.config-06zEaDw$default(FreeScope.kt:54)
	at kotest.TimeoutTest$1$1.invokeSuspend(TimeoutTest.kt:22)
	at kotest.TimeoutTest$1$1.invoke(TimeoutTest.kt)
	at io.kotest.core.spec.style.scopes.FreeSpecRootScope$minus$1.invokeSuspend(FreeSpecRootScope.kt:20)
	at io.kotest.core.spec.style.scopes.FreeSpecRootScope$minus$1.invoke(FreeSpecRootScope.kt)
	at io.kotest.core.internal.ExecutionsKt$executeWithBehaviours$2$1.invokeSuspend(executions.kt:13)
	at io.kotest.core.internal.ExecutionsKt$executeWithBehaviours$2$1.invoke(executions.kt)
	at io.kotest.core.internal.ExecutionsKt.wrapTestWithGlobalAssert(executions.kt:39)
	at io.kotest.core.internal.ExecutionsKt$executeWithBehaviours$2.invokeSuspend(executions.kt:12)
	at io.kotest.core.internal.ExecutionsKt$executeWithBehaviours$2.invoke(executions.kt)
	at io.kotest.core.internal.ExecutionsKt.wrapTestWithAssertionModeCheck(executions.kt:23)
	at io.kotest.core.internal.ExecutionsKt.executeWithBehaviours(executions.kt:11)
	at io.kotest.core.internal.TestCaseExecutor$executeAndWait$2$1$1$3$1.invokeSuspend(TestCaseExecutor.kt:222)
	at io.kotest.core.internal.TestCaseExecutor$executeAndWait$2$1$1$3$1.invoke(TestCaseExecutor.kt)
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:91)
	at kotlinx.coroutines.CoroutineScopeKt.coroutineScope(CoroutineScope.kt:194)
	at io.kotest.core.internal.TestCaseExecutor$executeAndWait$2$1$1$3.invokeSuspend(TestCaseExecutor.kt:216)
	at io.kotest.core.internal.TestCaseExecutor$executeAndWait$2$1$1$3.invoke(TestCaseExecutor.kt)
	at io.kotest.mpp.ReplayKt.replay(replay.kt:18)
	at io.kotest.core.internal.TestCaseExecutor$executeAndWait$2$1$1.invokeSuspend(TestCaseExecutor.kt:211)
	at io.kotest.core.internal.TestCaseExecutor$executeAndWait$2$1$1.invoke(TestCaseExecutor.kt)
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturnIgnoreTimeout(Undispatched.kt:102)
	at kotlinx.coroutines.TimeoutKt.setupTimeout(Timeout.kt:120)
	at kotlinx.coroutines.TimeoutKt.withTimeout(Timeout.kt:37)
	at io.kotest.core.internal.TestCaseExecutor$executeAndWait$2$1.invokeSuspend(TestCaseExecutor.kt:210)
	at io.kotest.core.internal.TestCaseExecutor$executeAndWait$2$1.invoke(TestCaseExecutor.kt)
	at io.kotest.engine.ExecutorExecutionContext$executeWithTimeoutInterruption$$inlined$suspendCoroutine$lambda$2.invokeSuspend(ExecutorExecutionContext.kt:56)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:274)
	at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:84)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59)
	at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
	at io.kotest.engine.ExecutorExecutionContext.executeWithTimeoutInterruption(ExecutorExecutionContext.kt:55)
	at io.kotest.core.internal.TestCaseExecutor$executeAndWait$2.invokeSuspend(TestCaseExecutor.kt:209)
	at io.kotest.core.internal.TestCaseExecutor$executeAndWait$2.invoke(TestCaseExecutor.kt)
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturnIgnoreTimeout(Undispatched.kt:102)
	at kotlinx.coroutines.TimeoutKt.setupTimeout(Timeout.kt:120)
	at kotlinx.coroutines.TimeoutKt.withTimeout(Timeout.kt:37)
	at io.kotest.core.internal.TestCaseExecutor.executeAndWait(TestCaseExecutor.kt:207)
	at io.kotest.core.internal.TestCaseExecutor.invokeTestCase(TestCaseExecutor.kt:176)
	at io.kotest.core.internal.TestCaseExecutor.executeActiveTest(TestCaseExecutor.kt:145)
	at io.kotest.core.internal.TestCaseExecutor$intercept$innerExecute$1$1.invokeSuspend(TestCaseExecutor.kt:83)
	at io.kotest.core.internal.TestCaseExecutor$intercept$innerExecute$1$1.invoke(TestCaseExecutor.kt)
	at io.kotest.core.internal.TestCaseExecutor.executeIfActive(TestCaseExecutor.kt:109)
	at io.kotest.core.internal.TestCaseExecutor$intercept$innerExecute$1.invokeSuspend(TestCaseExecutor.kt:83)
	at io.kotest.core.internal.TestCaseExecutor$intercept$innerExecute$1.invoke(TestCaseExecutor.kt)
	at io.kotest.core.internal.TestCaseExecutor.intercept(TestCaseExecutor.kt:97)
	at io.kotest.core.internal.TestCaseExecutor.execute(TestCaseExecutor.kt:63)
	at io.kotest.engine.runners.SingleInstanceSpecRunner.runTest(SingleInstanceSpecRunner.kt:73)
	at io.kotest.engine.runners.SingleInstanceSpecRunner$execute$2$invokeSuspend$$inlined$invoke$lambda$1.invokeSuspend(SingleInstanceSpecRunner.kt:83)
	at io.kotest.engine.runners.SingleInstanceSpecRunner$execute$2$invokeSuspend$$inlined$invoke$lambda$1.invoke(SingleInstanceSpecRunner.kt)
	at io.kotest.engine.spec.SpecRunner$runParallel$$inlined$map$lambda$2$1.invokeSuspend(SpecRunner.kt:80)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:274)
	at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:84)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59)
	at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38)
	at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
	at io.kotest.engine.spec.SpecRunner$runParallel$$inlined$map$lambda$2.run(SpecRunner.kt:79)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

Test did not complete within 300000ms

TimeoutException(duration=300000)
	at io.kotest.core.internal.TestCaseExecutor.executeAndWait(TestCaseExecutor.kt:231)
	at io.kotest.core.internal.TestCaseExecutor.invokeTestCase(TestCaseExecutor.kt:176)
	at io.kotest.core.internal.TestCaseExecutor.executeActiveTest(TestCaseExecutor.kt:145)
	at io.kotest.core.internal.TestCaseExecutor$intercept$innerExecute$1$1.invokeSuspend(TestCaseExecutor.kt:83)
	at io.kotest.core.internal.TestCaseExecutor$intercept$innerExecute$1$1.invoke(TestCaseExecutor.kt)
	at io.kotest.core.internal.TestCaseExecutor.executeIfActive(TestCaseExecutor.kt:109)
	at io.kotest.core.internal.TestCaseExecutor$intercept$innerExecute$1.invokeSuspend(TestCaseExecutor.kt:83)
	at io.kotest.core.internal.TestCaseExecutor$intercept$innerExecute$1.invoke(TestCaseExecutor.kt)
	at io.kotest.core.internal.TestCaseExecutor.intercept(TestCaseExecutor.kt:97)
	at io.kotest.core.internal.TestCaseExecutor.execute(TestCaseExecutor.kt:63)
	at io.kotest.engine.runners.SingleInstanceSpecRunner.runTest(SingleInstanceSpecRunner.kt:73)
	at io.kotest.engine.runners.SingleInstanceSpecRunner$execute$2$invokeSuspend$$inlined$invoke$lambda$1.invokeSuspend(SingleInstanceSpecRunner.kt:83)
	at io.kotest.engine.runners.SingleInstanceSpecRunner$execute$2$invokeSuspend$$inlined$invoke$lambda$1.invoke(SingleInstanceSpecRunner.kt)
	at io.kotest.engine.spec.SpecRunner$runParallel$$inlined$map$lambda$2$1.invokeSuspend(SpecRunner.kt:80)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:274)
	at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:84)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59)
	at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38)
	at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
	at io.kotest.engine.spec.SpecRunner$runParallel$$inlined$map$lambda$2.run(SpecRunner.kt:79)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:6 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
sksamuelcommented, Oct 28, 2020

In the example code, your test invocation timeout is 2 seconds.

0reactions
sksamuelcommented, Dec 22, 2020

I’m confident this is fixed since 4.3.1 I will close but please reopen if you encounter any other issues.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Test Timeouts - Kotest
We can apply a test and/or invocation timeout for all tests in a module using project config. object ProjectConfig : AbstractProjectConfig { override...
Read more >
Troubleshoot Lambda function invocation timeout errors - AWS
Verify that your Lambda function is timing out​​ Retrieve the request IDs of any timed-out invocations by searching the function's Amazon ...
Read more >
Ginkgo test suite - GitHub Pages
Ginkgo is a testing framework for Go designed to help you write expressive tests. It is best paired with the Gomega matcher library....
Read more >
Spock Framework Reference Documentation
Spock is a testing and specification framework for Java and Groovy applications. ... argument is not needed anymore except when undoing a spec-level...
Read more >
WildFly Admin Guide
Can be overridden on a per-server basis in host.xml . If not provided in the server-group element, it must be provided for each...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

Arb.stringPattern slows down the test dramatically

Next page

Wrong error thrown when test exceeds invocation time out