Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
[1.11.0 Release] `serve_failure` failing on master and 1.11.0 branches
See original GitHub issueE.g. 1.11.0: https://buildkite.com/ray-project/periodic-ci/builds/2524#5e259950-8965-4f40-a39a-bb0f2fbf2c86/138-427 master: https://buildkite.com/ray-project/periodic-ci/builds/2503#69d841c1-1c30-4ea4-8a2e-c586ffe142c8/139-429
I was able to reproduce the issue by starting the test from my laptop. The cluster can be kept open after the failure with --no-terminate. e.g. get aws creds, then run
ANYSCALE_HOST=https://console.anyscale.com ANYSCALE_CLI_TOKEN=... ANYSCALE_CLOUD_ID=... ANYSCALE_PROJECT=... RAY_WHEELS=... RAY_TEST_REPO=https://github.com/ray-project/ray.git RAY_TEST_BRANCH=... python ray/release/e2e.py --category=... --test-config ray/release/long_running_tests/long_running_tests.yaml --test-name serve_failure --keep-results-dir --smoke-test --no-terminate
The failures to reach /api/version in the tests are due to dashboard being terminated. I checked the cluster, and gcs was also terminated. It looks like the process terminations are not due to any crash. However, because dashboard was terminated, the test cannot submit job and finish successfully.
Assigning to @simon-mo as serve oncall.
Issue Analytics
- State:
- Created 2 years ago
- Comments:7 (7 by maintainers)
Top Results From Across the Web
Top Related Medium Post
Top Related StackOverflow Question
Troubleshoot Live Code
Top Related Reddit Thread
Top Related Hackernoon Post
Top Related Tweet
Top Related Dev.to Post
Top Related Hashnode Post
Since the issue has been identified to be related to test setup, lowering to P1.
Sounds good Jiao.