Add support for `fail fast` option

Hi GoogleChrome team!

I was wondering if a fail fast option, same as we see in testing library like jest , could be implemented on Lighthouse CI?

My use case is pretty simple: I want to run lighthouse twice on all my HTML pages, but when the same assertion fail on each pages, the CI result is quite overloaded… making it difficult to read and fix with ease.

Having such an option could prevent lighthouse from running further on pages that will encounter similar assertion issue.

Thanks a lot for your feedback 😉

[patrickhulce]

You can accomplish this yourself today by running lhci collect --numberOfRuns=1, running your assertions with lhci assert, and then running lhci collect --numberOfRuns=N-1 --additive before uploading.

It’s potentially feasible to introduce such a mode via a flag to autorun but we’re currently focused on features for LHCI that enable solutions to unsolved problems.

[xavierfoucrier]

Hi @patrickhulce,

Thanks again for your answer. Sounds like a great argument and I completely understand your position.

I am closing the issue, feel free to reopen if more people ask for this feature to be implemented.

Have a nice day! 😎 🤖