Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[Core][Bug] many_task stress test regression

See original GitHub issue

Search before asking

I searched the issues and found no similar issues.

Ray Component

Ray Core

What happened + What you expected to happen

{
  "tasks_per_second": 27.508657888123608,
  "num_tasks": 10000,
  "time": 663.5219151973724,
  "success": "1",
  "_runtime": 674.2678966522217,
  "_session_url": "https://beta.anyscale.com/o/anyscale-internal/projects/prj_2xR6uT6t7jJuu1aCwWMsle/clusters/ses_XCJkRqS4HkuHLXehx7i6Fwvc",
  "_commit_url": "https://s3-us-west-2.amazonaws.com/ray-wheels/releases/1.7.0/2367a2cb9033913b68b1230316496ae273c25b54/ray-1.7.0-cp37-cp37m-manylinux2014_x86_64.whl",
  "_stable": true
}

We have seen task scheduling/execution throughput regression (27/s) since Ray 1.7. It was 500-600/s at Ray 1.4 and unfortunately the benchmark is missing from 1.5 and 1.6.

@jjyao’s change parallelizing of lease requests is expected to to speed this up

Versions / Dependencies

N/A

Reproduction script

N/A

Anything else

No response

Are you willing to submit a PR?

Yes I am willing to submit a PR!

Issue Analytics

State:
Created 2 years ago
Comments:6 (6 by maintainers)

Top GitHub Comments

1reaction

rkooo567commented, Jan 6, 2022

Yeah agreed! We can have different policies to make it better as well.

Also, my current impression of max lease requests is to alleviate load, so it might be also possible to control this from raylet. For example, raylet decides the max pending tasks (that could be high, like 5000~10000) and reject the lease request if the queue is full.

I think we can discuss the best approach when we tune the best value.

1reaction

ericlcommented, Jan 3, 2022

@rkooo567 can you take a look?

Top Results From Across the Web

Stress Testing 02: Regression Analysis - YouTube

In this video I provide a short background on regression analysis in the context of how macroeconomic variables can be used to predict...

What is Stress Test? - YouTube

PerfMatrix Post Link: http://www.perfmatrix.com/what-is- stress - test /This video covers:1. What is Stress Test ?2. What is Future Load?3.

Lasso Regressions and Forecasting Models in Applied Stress ...

This paper presents a conceptual overview of lasso regressions, explains how they fit in applied stress tests, describes its advantages over ...

Stress-testing banks – a comparative analysis

This section provides an introduction and overview of stress test programmes implemented by public authorities such as central banks and supervisors. This FSI ......

The disutility of stress testing in low-risk HEART ... - NCBI - NIH

Multivariate logistic regression was used to determine the association of age, sex, race/ethnicity, and HEAR score with stress testing.