Optional lazy load for scheme

Our use case: we have no existing file with the specification, only apispec. We can generate docs from it, but:

• Schemathesis must generate test cases BEFORE running any tests, on the stage of tests collecting.
• Flask app initialization happens in a fixture, before running the first test.

I thought about laziness for test cases generation but looks like there is no way to do it. Maybe, only some kind of subtests inside existing test 🤔

My decision, for now, is keeping schemathesis tests as a separate script, outside of main tests. However, it makes difficult to run it on CI: you have to run the dev server on background, and only after that run the script and test swagger spec.

Let’s think, how to make test cases generation lazy. If it’s possible on existing implementation (somehow nest test generation into test function, and don’t fail on the first assert), code example somewhere in docs would be very helpful.

Some ideas: https://github.com/kiwicom/schemathesis/pull/36#issuecomment-530325742

[Stranger6667]

Hello @orsinium 😃

The relevant feature is released in 0.6.0. Please, see the example here - https://github.com/kiwicom/schemathesis#lazy-loading

And feel free to re-open if it wouldn’t work for you

[Stranger6667]

I’m very happy to hear it 😃 Thank you!