More flexible listener/extension mechanism

If compare JUnit, TestNG and Kotest on extension points then Kotest will be last and worst.

1. JUnit and TestNG cook testPlan/testRun before running and then filtered, mapped and etc. Kotest works only with discovered classes and has one extension point afterScan You may create instance of this classes and cast to Spec and found root tests but rest of nested tests will be handled only during running tests. This difference isn’t bad just only for note but it could be useful have all testPlan before start. But we have what we have.
2. JUnit and TestNG works with annotations. These annotated methods are called configuration methods and prepare test fixture (create DB/MQ/network connections, prepare test data, do common work, clear test state, release resources and many many other things). They have separated ways to control test run flow by Listeners and Extensions. One more time, config methods and listeners are separated things. Kotest has only listeners and extensions SpecListener, TestListener, SpecExtension, TestCaseExtension and several specific extensions and use these listeners/extensions as configuration methods. The problem is I can’t sort the buyers from the spyers. Your Allure integration is not work properly. And I try to write it from scratch by myself but I need more flexible extension (or flow control) mechanism. For example, If I write some code for container creation in BeforeSpec listener’s method and I want to write all preparing test fixture steps who give me guarantee that all steps in other BeforeSpec listeneres will be recorded? For this purpose my Allure listener must be first in listener list. Let’s go further. Ok, if some step in BeforeSpec BeforeTest will fail what happen then? And how it handle? How to know that particular config method is failed but not the testCase? Ok, Before methods and testCase are passed but what about After methods and their results.

I think that should be separated config methods and listeners that could intercept these config methods and their results. Look at TestNG Listener interfaces (not codebase because its ugly but the idea) IConfigurationListener IClassListener IMethodInterceptor IInvokedMethodListener ISuiteListener ITestListener

JUnit 5 has power flow control mechanism too.

It useful not only for Allure integration but and for some test case management system too I can’t do such big things by myself because it isn’t my framework and maybe this feature is not needed at all

[sksamuel]

This can all be done in a single LifecycleExtension point. We could add this for 4.1 or 4.2

On Mon, 18 May 2020, 08:22 crazyk2, [e-mail] wrote:

Exactly.

interceptBeforeSpec(spec: Spec, beforeSpec: BeforeSpec) {//do something before BeforeSpec will be invokedprintln(“BeforeSpec will be invoked right now for spec ${spec::class}”)val result = beforeSpec(spec)//do something after BeforeSpec was invokedprintln(“BeforeSpec waw invoked and result is ${result.type}”) }

Where BeforeSpec need change from

typealias BeforeSpec = suspend (Spec) -> Unit

to

typealias BeforeSpec = suspend (Spec) -> SomeResultOfListenerCall

In the same way interceptAfterSpec interceptBeforeTest interceptAfterTest beforeInvocation afterInvocation

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/kotest/kotest/issues/1438#issuecomment-630179446, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFVSGRUYQQ63L2P2P56LATRSEZB3ANCNFSM4NAFMDZA .

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.