question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[placement group] Can not create pg for fractional resources

See original GitHub issue

What is the problem?

I try to create a placement group with specific node ip as resources. I used Ray 2.0.0-dev, python 3.8.3, and tried with two ways, neither works.

First is directly creating the pg, and assuming the ray cluster_resources already has the node ip resource

import ray
from ray.util.placement_group import (
    placement_group,
    placement_group_table,
    remove_placement_group
)
ray.init()
ray.cluster_resources() # this will print {'node:192.168.1.5': 1.0, 'CPU': 12.0, 'memory': 232.0, 'object_store_memory': 80.0}
ray.state.current_node_id() # this will print 'node:192.168.1.5'

#creating the pg with ip as the resource bundle
pg=placement_group([{ray.state.current_node_id():0.01}])
ray.get(pg.ready()) # code will stuck here, never return

second is to init ray with the custom resources

import ray
from ray.util.placement_group import (
    placement_group,
    placement_group_table,
    remove_placement_group
)
ray.init(resources={'node:192.168.1.5':1.0})
pg=placement_group([{'node:192.168.1.5':0.01}])
ray.get(pg.ready()) # code will stuck here too.

Ray version and other system information (Python version, TensorFlow version, OS): ray 2.0.0.dev0, python 3.8.3, os Darwin Kernel Version 19.6.0

Reproduction (REQUIRED)

Please provide a short code snippet (less than 50 lines if possible) that can be copy-pasted to reproduce the issue. The snippet should have no external library dependencies (i.e., use fake or mock data / environments): Attached above. If the code snippet cannot be run by itself, the issue will be closed with “needs-repro-script”.

  • I have verified my script runs in a clean environment and reproduces the issue.
  • I have verified the issue also occurs with the latest wheels.

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:22 (22 by maintainers)

github_iconTop GitHub Comments

1reaction
richardliawcommented, Feb 2, 2021

Also I can confirm this bug exists on master.

0reactions
amogkamcommented, Feb 11, 2021

@richardliaw Ohhh yes good catch. That fixes the problem.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Autoscaling placement groups - Ceph Documentation
Placement groups (PGs) are an internal implementation detail of how Ceph distributes data. You may enable pg-autoscaling to allow the cluster to make...
Read more >
Placement groups - Amazon Elastic Compute Cloud
There is no charge for creating a placement group. Placement group strategies. You can create a placement group using one of the following...
Read more >
Chapter 3. Placement Groups (PGs) Red Hat Ceph Storage 4
PGs do not own OSDs. CRUSH assigns many placement groups to each OSD pseudo-randomly to ensure that data gets distributed evenly across the...
Read more >
Placement Groups — Ray 2.2.0
get(pg.ready()) ) and have the proper resources. Ray assumes that the placement group will be properly created and does not print a warning...
Read more >
Configuration - Spark 3.3.1 Documentation - Apache Spark
For instance, Windows does not support resource limiting and actual resource is not limited on MacOS. 2.4.0. spark.executor.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found