question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[autoscaler] Problem of private cluster with autoscaler

See original GitHub issue

System information

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): 8 * Ubuntu 16.04 with 2 GPU per machine
  • Ray installed from (source or binary):binary
  • Ray version:0.7.2
  • Python version:3.6.8
  • Exact command to reproduce: see below

Describe the problem

Source code / logs

I set my setting file cluster.yaml like this:

provider:
    type: local
    head_ip: host1
    worker_ips: [host2, host3, host4, host5, host6, host7, host8]

But when I exec ray up cluster.yaml, I found that the workers didn’t start ray process. And in python, ray.nodes() only return the head node.

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:10 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
richardliawcommented, Aug 2, 2019

Those are non-fatal 😃

On Fri, Aug 2, 2019 at 11:38 AM Andriy Drozdyuk notifications@github.com wrote:

I get these errors when I try to ray up the cluster:

bash: cannot set terminal process group (-1): Inappropriate ioctl for device bash: no job control in this shell

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ray-project/ray/issues/5326?email_source=notifications&email_token=ABCRZZJHUWDRBW3P4NJOAJTQCR5JVA5CNFSM4IIB3PY2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3ORAII#issuecomment-517804065, or mute the thread https://github.com/notifications/unsubscribe-auth/ABCRZZKOG6X65XCYXCLH2D3QCR5JVANCNFSM4IIB3PYQ .

0reactions
richardliawcommented, Aug 6, 2019

Closing this for now as the OP hasn’t followed up; @HanwGeek feel free to reopen if your problem isn’t resolved.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Cluster Autoscaler: How It Works and Solving Common ...
Cluster Autoscaler automatically adapts the number of Kubernetes nodes in your cluster to your requirements. When the number of pods that are pending...
Read more >
2053343 – Cluster Autoscaler not scaling down nodes which ...
Description of problem: A cluster's cluster autoscaler has been configured to scale down nodes, but does not seem to be scaling down nodes...
Read more >
Viewing cluster autoscaler events - Google Cloud
Occurs when there are unschedulable Pods in the cluster, and cluster autoscaler cannot scale the cluster up to accommodate the Pods.
Read more >
Dynamically resizing a cluster with the cluster autoscaler
Autoscale is configured on a node pool basis, i.e. you don't enable autoscaling on a full cluster, you enable it for one or...
Read more >
Autoscaling - Amazon EKS - AWS Documentation
The Kubernetes Cluster Autoscaler automatically adjusts the number of nodes in your cluster when pods fail or are rescheduled onto other nodes. The...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found