Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

When running multiple tests and the first test fails, then all other tests also fail

See original GitHub issue

Describe the bug When running multiple tests and the first test fails, then all other tests also fail.

To Reproduce

In a folder have multiple test files (We used excel format)
Let the first test fail intentionally
Run those tests using Botium-CLI botium-cli run mochawesome --config .\botium.json --convos Tests/
The first test will fail
The other tests will also fail

Expected behavior Only the first test fails and the rest succeed

Screenshots and Log files BotiumTestResultLog.txt BotiumVerboseLog.txt

Botium Flavour: Botium CLI

Botium Version 1.0.3

Additional context We ran botium-cli using the docker image on docker hub. But we could also reproduce this problem with a locally installed botium-cli. Our current workaround is running botium-cli docker image version 0.0.77, where the issue is not present.

We use Botium to test a Chatbot running on Microsoft Bot Framework using the DirectLine connector.

Issue Analytics

State:
Created 3 years ago
Comments:6 (3 by maintainers)

Top GitHub Comments

1reaction

Elias011commented, Jan 28, 2021

We just tried and it worked! Thank you very much for the help 🙂

0reactions

codeforequity-atcommented, Jan 28, 2021

Can you please try to set the “DIRECTLINE3_WEBSOCKET” capability to false and try again

Top Results From Across the Web

Why do my unit tests (mostly) fail when I select Run All but ...

I am working in Visual Studio 2013, and most of my unit tests will fail if I select "Run All", but if I...

Second test fails when run with first, but passes when ... - GitHub

I created a Gist here that recreates the error I'm having. ... fails when running same tests several times), #747 (flakes test fail), ......

What To Do When Tests Fail? - TestProject

Test failures - how they impact our app quality, how to prevent them, what best practices exists to detect them ealy and fix...

Writing and Organizing Tests - Cypress Documentation

After a test fails, the screenshots and videos can help find the problem so it can be fixed. Pending. You can write placeholder...

Advanced googletest Topics

HasFatalFailure() in the ::testing::Test class returns true if an assertion in the current test has suffered a fatal failure. This allows functions to...