Trouble shutting down local distributed client/displaying it afterwards

Appears the running Client.close consistently runs into some sort of hang up. Also it use to be the case that Client._repr_html_ worked on a Client object after it was closed. However that is no longer the case since Client.scheduler is now None. Example of this behavior in the Gist’d Jupyter Notebook linked below along with the environment used to reproduce it. This happens in Distributed 1.21.8, but not in Distributed 1.21.6.

ref: https://gist.github.com/jakirkham/b9c75e2f7fabdee5f19db8cc3bff0d9e

[mrocklin]

Yeah, I think that this started appearing when Tornado switched to Asyncio. We don’t currently have a good way around it. I spent a long while trying to track this down. In the end the solution was to just ignore this if we’re also shutting down the Python process, so in the common case for users it doesn’t show up.

I agree that it’s an issue. I would love to see it resolved.

On Mon, May 7, 2018 at 3:56 PM, jakirkham [e-mail] wrote:

Thanks @mrocklin https://github.com/mrocklin.

So there is another issue with close generating the exception in the log (as show below). If that’s already in a separate issue, happy to follow along. Could also migrate it to a new issue or reopen. Whatever works best.

Exception ignored in: <generator object add_client at 0x11e4d1f10>RuntimeError: generator ignored GeneratorExit Future exception was never retrieved future: <Future finished exception=CommClosedError(‘in <closed TCP>: Stream is closed’,)> Traceback (most recent call last): File “/zopt/conda2/envs/test/lib/python3.6/site-packages/distributed/comm/tcp.py”, line 179, in read n_frames = yield stream.read_bytes(8) File “/zopt/conda2/envs/test/lib/python3.6/site-packages/tornado/gen.py”, line 1099, in run value = future.result() tornado.iostream.StreamClosedError: Stream is closed

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File “/zopt/conda2/envs/test/lib/python3.6/site-packages/tornado/gen.py”, line 1107, in run yielded = self.gen.throw(*exc_info) File “/zopt/conda2/envs/test/lib/python3.6/site-packages/distributed/comm/tcp.py”, line 200, in read convert_stream_closed_error(self, e) File “/zopt/conda2/envs/test/lib/python3.6/site-packages/distributed/comm/tcp.py”, line 128, in convert_stream_closed_error raise CommClosedError(“in %s: %s” % (obj, exc)) distributed.comm.core.CommClosedError: in <closed TCP>: Stream is closed

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dask/distributed/issues/1963#issuecomment-387183995, or mute the thread https://github.com/notifications/unsubscribe-auth/AASszDS2ZsZd0JbVlGN0Hd6iEDopaBVqks5twKcGgaJpZM4T0Utd .

[jakirkham]

Went ahead and put it in a new issue ( https://github.com/dask/distributed/issues/1969 ) with a clearer description. Please feel free to add anything else relevant there.