Ensure dashboard starts on the desired port

What is the problem?

We need a guarantee that the ray dashboard will get started on port 8265 when ray stop and ray start are called programmatically in quick succession.

Ray version and other system information (Python version, TensorFlow version, OS):

In Ray 1.2.0 (Python 3.7) using ray stop --force, fixes the problem the majority of the time but we’re still seeing it get started on the undesired port about 1/2000 tries.

Reproduction (REQUIRED)

Please provide a short code snippet (less than 50 lines if possible) that can be copy-pasted to reproduce the issue. The snippet should have no external library dependencies (i.e., use fake or mock data / environments):

Courtesy of @ijrsvt

import subprocess
iteration = 0
while True:
    output = subprocess.run("ray stop --force; ray start --head > /dev/null", shell=True, capture_output=True)
    assert "8265" in output.stderr.decode()
    print("Started on 8265", " -- Iteration -- ", iteration, ")
    iteration += 1

If the code snippet cannot be run by itself, the issue will be closed with “needs-repro-script”.

• I have verified my script runs in a clean environment and reproduces the issue.
• I have verified the issue also occurs with the latest wheels.

[ijrsvt]

@edoakes Sounds good, the one edge case that I personally find weird is that if I specify --dashboard-port=8265, the current behavior is to still increment (this is only because it looks like the code doesn’t differentiate from a user-provided value of 8265 and the default value of 8265).

[fyrestone]

Maybe we can set the default value of dashboard port to None (or 0).

• If the dashboard port is None,
  • then auto select a port from [8265, 8265 + 10], failed if no available port found.
• else,
  • use the dashboard port to start dashboard, failed if the port is occupied.