way to exit a given test if a test case fails and carry on with other test suites

description of the problem:

Currently, the --exitonfailure CLI option allows you to exit a test with failures if a test case fails. This however doesn’t work when you are running multiple robot tests at once (such as in CI). If you are using the --exitonfailure option and a given test fails, the remaining tests will also fail. I want to run a test suite and exit a given test if a test case fails and carry on with the rest of the suites that need to run. This would benefit from finding test failures quicker and reducing the time RF tests take to run in CI. It would be nice to be able to exit a given test if an assertion fails and carry on with any remaining tests in the suite.

proposed solution:

Add a CLI option that allows a user to exit a given test if an assertion fails and carry on with any remaining tests in the suite.

[pekkaklarck]

It’s true there’s no way to stop the current suite when tests are running and continue from the next one. If a failure occurs in suite setup, tests in that suite aren’t run, but there’s no direct way to accomplish this when tests in the suite are already running. --exitonfailure stops the whole execution and Fatal Error keyword works the same way.

Something like this has been requested already earlier (there may even be an issue about it but I don’t have time to search right now). I think the proposal was to add a keyword like ´Fatal Errorthat stops the current suite and I think that's a more flexible solution than a command line option. A benefit of a keyword would be that it could be configured to tell from which suite level to continue. For example, if you have a suite structure like below and are running suite1.1, you may want to continue from suite1.2 or from mid suite 2. Possibly we could just enhanceFatal Error` to accept such configuration.

                 root suite
              /              \ 
     mid suite 1             mid suite 2
    /           \           /           \
suite1.1    suite1.2    suite2.1    suite2.2

[luke-h1]

As an update on this issue, we managed to produce a workaround that exits a test as soon as a given test case fails via injecting a custom python script that runs as part of Test Setup. You can view the full implementation here