Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[BUG] Gradle check retry not working as expected

See original GitHub issue

Describe the bug

As a part of https://github.com/opensearch-project/OpenSearch/issues/2547 and https://github.com/opensearch-project/OpenSearch/pull/2638/, a retry mechanism was introduced to rerun failed tests during gradle check
With the increased flakiness in tests, one issue that was noticed is the retries do not always run for tests on the build server

Sample runs -

To Reproduce

Run gradle check for a PR with a test failure

Expected behavior

Retries to be present within the console output with x tests completed, y failed with y > 1 in case of failed tests with a retry

Plugins

Screenshots

Host/Environment (please complete the following information):

Jenkins build server for OpenSearch

Additional context

https://github.com/opensearch-project/OpenSearch/issues/2878

Issue Analytics

State:
Created 10 months ago
Comments:11 (11 by maintainers)

Top GitHub Comments

1reaction

andrrosscommented, Nov 15, 2022

Again looking at 6825, we see the “BUILD SUCCESSFUL” result in the output (tests succeeded on retry):

BUILD SUCCESSFUL in 25m 35s
2626 actionable tasks: 2612 executed, 2 from cache, 12 up-to-date
...
Finished: UNSTABLE

The gradle check jenkins run completes with the result “UNSTABLE” and results in a failure of the GitHub action. Is this the right behavior? I think the point of the retries is to pass the gradle check when tests pass on retry so this seems wrong to me.

0reactions

kotwanikunalcommented, Nov 15, 2022

Fix: https://github.com/opensearch-project/opensearch-build/pull/2902

Top Results From Across the Web

Unable to retry test methods from parent classes with Spock 2

org.gradle.test-retry was unable to retry the following test methods, which is unexpected. Please file a bug report at ...

Jenkinsfile clear gradle cache and retry on failure

Current scenario: We have configured CI pipeline and running a jenkinsfile.groovy script for a gradle build. The pipeline is working fine ...

Learning the Basics - Gradle User Manual

There are different issues with using mavenLocal() that you should be aware of: Maven uses it as a cache, not a repository, meaning...

Flaky test detection doesn't consider multiple runs of the same ...

Gradle build is successful. When "support test retry: successful test run mutes previous test failure" is unchecked, adding teamcity.internal.

Gradle Enterprise Predictive Test Selection User Manual

In order to obtain an accurate estimate of expected Predictive Test Selection quality, it is recommended to inspect incorrect predictions and omit simulations...