HTTP server doesn't closed

What happened: When opening a client twice (or more) like the following code:

from dask.distributed import Client
with Client() as client:
    ...
...
with Client() as client:
    ...

or

from dask.distributed import Client
client = Client()
...
client.close()
...
client = Client()
...
client.close()

I`m getting the following warning massage when the new client opens (after one was created before):

/.../opt/anaconda3/envs/mlflow-env/lib/python3.7/site-packages/distributed/node.py:155: UserWarning: Port 8787 is already in use.
Perhaps you already have a cluster running?
Hosting the HTTP server on port 41137 instead
  http_address["port"], self.http_server.port

What you expected to happen: When a client is closed, it should also close the http_server thus releasing the port.

Minimal Complete Verifiable Example:

from dask.distributed import Client

with Client() as client:
    print(client)

with Client() as client:
    print(client)

Environment:

• Dask version: 2.25.0
• Python version: 3.7.9
• Operating System: Linux
• Install method (conda, pip, source): conda

[TomAugspurger]

Are you relying on garbage collection to close your LocalCluster for you? It might be safer to use a context manager for it as well, given how complicated shutting down a cluster is.

[DavidKatz-il]

Are you relying on garbage collection to close your LocalCluster for you? It might be safer to use a context manager for it as well, given how complicated shutting down a cluster is.

Thanks, this solves the problem.

In [1]: from time import sleep
   ...: from dask.distributed import Client, LocalCluster
   ...: 
   ...: with LocalCluster(dashboard_address=':3258') as cluster:
   ...:     with Client(cluster) as client:
   ...:         print(client)
   ...: sleep(10)
   ...: with LocalCluster(dashboard_address=':3258') as cluster:
   ...:     with Client(cluster) as client:
   ...:         print(client)
   ...: 
<Client: 'tcp://127.0.0.1:43337' processes=4 threads=8, memory=67.42 GB>
<Client: 'tcp://127.0.0.1:35661' processes=4 threads=8, memory=67.42 GB>