question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Issues with skipped tests in summary and html report

See original GitHub issue

I have been using karate for automated tests (starting with 0.6.0, in September 2017) and am very happy with this excellent tool. I pulled the latest 0.8.0 (for https://github.com/intuit/karate/issues/418) and noticed couple of things that I would like to bring to your notice. Let me start of by saying this issue is related to karate.abort() and related summary/html reports (the undocumented / secret unreleased feature) and can be treated as enhancement. I did look at the open issues and feel this issue was not raised before.

Now here is the issue I would like to discuss before sending a PR. For the hypothetical test given below:

Scenario: test abort
  * def result = call read('some-test.feature')
  // skip tests below if some-test.feature failed for what ever reason
  * def check2Abort = (result.response == 'failed' ? karate.abort() : {} )
  // continue with rest of the steps in this scenario

The two issues are

  1. summary shown after the test is executed doesn’t show any skipped tests
  2. and the cucumber html report doesn’t show any skipped tests as well

Please note that the steps are actually skipped (as expected), but the report doesn’t reflect these skipped steps. Let me point two one line changes that I was thinking about Change needed in https://github.com/intuit/karate/blob/master/karate-core/src/main/java/com/intuit/karate/cucumber/KarateHtmlReporter.java#L236

else if ((StepResult.ABORTED.equals(result.getStatus())) || (Result.SKIPPED.getStatus()
        .equals(result.getStatus()))) {

And https://github.com/intuit/karate/blob/master/karate-core/src/main/java/com/intuit/karate/cucumber/KarateRuntime.java#L77 need to be modified as

Result result = aborted ? Result.SKIPPED : StepResult.PASSED;

This is the first time I was looking at karate source tree (so far I have been a passive user), so please bear with me if I missed the reason for proposed karate.abort() and feel free to shoot down the issue/suggested code change.

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:11 (10 by maintainers)

github_iconTop GitHub Comments

1reaction
vmchukkycommented, Jul 20, 2018

@ptrthomas sure, I will try to work on it and raise a PR with a test case.

On a separate note: I am happy to see (from community news) that you will be presenting at Dev Conf in Bangalore (https://devconfin2018.sched.com/event/F749/karate-microservices-test-automation-made-simple), hoping to meet you in person.

0reactions
ptrthomascommented, May 2, 2019

@Adarshkrishna9099 unlikely. open a new issue and make sure you follow this process please: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue

Read more comments on GitHub >

github_iconTop Results From Across the Web

Issues with skipped tests in summary and html report on V0.9.2
This is issue with respect to karate.abort() function where I am able to successfully skip the steps once my condition is met but...
Read more >
JUnit integration reports skipped tests as passed - GitLab.org
What is the current behavior? GitLab CI pipeline test report counts skipped tests as "passed".
Read more >
Not getting SKIPPED test-cases in cucumber html report
I'm using @skip explicitly to skip the test cases. I want to see those cases in html report. I couldn't find any resource...
Read more >
HTML test report for androidTest see the skipped test cases as ...
Expected result, in Test summary should list skipped test cases number, successful should be equal = successful test cases number/ all test ......
Read more >
7. Incomplete and Skipped Tests — PHPUnit 9.5 Manual
to keep track of the tests that you have to write. The problem with empty test methods is that they are interpreted as...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found