Kotest is very slow to start

See CMVE here: https://github.com/tonicsoft/slow-kotest

relevant background: #1143 #470

For the example project above, Kotest takes around 4 seconds to run an empty unit test, compared to 1 second for junit. This is a significant interruption for a fast TDD workflow where a developer might be changing one line of code and rerunning a single, simple unit test tens or hundreds of times in a row.

The cause of the slowness seems to be scanning the classpath for AbstractProjectConfigs and AutoScan extensions. Apologies if this is a red herring - I ran the tests multiple times in a debugger and paused at different points and every time there was a thread blocked on one of these classpath scanning activities.

I appreciate that auto-magical wiring is very trendy for open source projects and allows for very terse and complete code examples to be provided in documentation and getting started guides. However, I would say that extensions are an advanced feature anyway and users of these features may not mind a couple of extra lines of code to do things “properly” in exchange for getting rid of this lag on test instantiation.

When I say “properly” I mean adhering to more robust but less fancy principles of dependency injection, “holly wood principle”, or whatever you want to call it. Project wide extensions could be passed in from the outside, explicitly, which would save the need to search the classpath for them and thus match the lightning fast run times of junit. Some discussion on #1143 touches on some of the problems that classpath scanning and implicit dependency wiring can have.

Off the top of my head I propose a couple of ways this could be achieved:

1. Require (or allow if a breaking change is not acceptable) the class name(s) of the ProjectConfig to be passed in as a parameter.

• a breaking change of some description has the advantages that new users or even experienced users setting up new projects will not have to suffer the 4 second delay.
• this is no different to how the main class name is often passed in from the outside when running jvm applications, so most jvm developers should be familiar with the pattern.

1. Make AutoScan extensions an opt in feature.
2. Process AutoScan annotations at compile time, as suggested in #470. Although it’s not clear to me that this wouldn’t just create a different issue, such as burdening users with the added complexity of annotation processors in their project.
3. If all else fails, could the classpath only be scanned once instead of twice to cut the delay in half? (again apologies if this is a red herring, I did not verify that the classpath is indeed scanned twice).

Finally, to support my case I would like to draw attention to the following comments from the discussion on #470:

We went with the former initially in versions 1 and 2 and as @helmbold will attest, all it [classpath scanning] did was cause bug report after bug report.

and

It [annotation processing] seems complicated and error prone.

I support the sentiment of those comments!

Perhaps a good old fashioned input parameter would be more robust, simpler, faster and altogether less grief for everyone hoping that one day kotlin + kotest can match the epic TDD speed of java + junit.

[sksamuel]

Implemented in master and will be included in 5.1, which we hope to release this week.

[sksamuel]

I was thinking that we would add a system property to disable autoscanning of project config, and another system property to specify an FQN, so the system can directly instantiate it via Class.forName. Then you can use project config but it should take a few millis to setup, rather than however long it takes to scan the entire classpath.