question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Update Dask and Distributed minimum version to 2.12 for Python 3.8 compatibility

See original GitHub issue

System information

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Win 10
  • Modin installed from (source or binary): pip
  • Modin version: 0.6.3
  • Python version: 3.8.0
  • Exact command to reproduce: import modin.pandas as pd

Describe the problem

Fresh install of modin, failure to import. Have tried pip --force-reinstall most dependencies.

Source code / logs

UserWarning: The Dask Engine for Modin is experimental.
Traceback (most recent call last):
  File "C:\WERK\backtestStrat3\backtest.py", line 2, in <module>
    import modin.pandas as pd
  File "C:\Python\lib\site-packages\modin\pandas\__init__.py", line 226, in <module>
    client = Client()
  File "C:\Python\lib\site-packages\distributed\client.py", line 727, in __init__
    self.start(timeout=timeout)
  File "C:\Python\lib\site-packages\distributed\client.py", line 892, in start
    sync(self.loop, self._start, **kwargs)
  File "C:\Python\lib\site-packages\distributed\utils.py", line 334, in sync
    raise exc.with_traceback(tb)
  File "C:\Python\lib\site-packages\distributed\utils.py", line 318, in f
    result[0] = yield future
  File "C:\Python\lib\site-packages\tornado\gen.py", line 735, in run
    value = future.result()
  File "C:\Python\lib\site-packages\distributed\client.py", line 955, in _start
    self.cluster = await LocalCluster(
  File "C:\Python\lib\site-packages\distributed\deploy\spec.py", line 364, in _
    await self._start()
  File "C:\Python\lib\site-packages\distributed\deploy\spec.py", line 284, in _start
    self.scheduler = await self.scheduler
  File "C:\Python\lib\site-packages\distributed\scheduler.py", line 1219, in start
    await self.listen(addr, listen_args=self.listen_args)
  File "C:\Python\lib\site-packages\distributed\core.py", line 321, in listen
    await listener.start()
  File "C:\Python\lib\site-packages\distributed\comm\tcp.py", line 432, in start
    self.tcp_server.add_sockets(sockets)
  File "C:\Python\lib\site-packages\tornado\tcpserver.py", line 165, in add_sockets
    self._handlers[sock.fileno()] = add_accept_handler(
  File "C:\Python\lib\site-packages\tornado\netutil.py", line 279, in add_accept_handler
    io_loop.add_handler(sock, accept_handler, IOLoop.READ)
  File "C:\Python\lib\site-packages\tornado\platform\asyncio.py", line 99, in add_handler
    self.asyncio_loop.add_reader(fd, self._handle_events, fd, IOLoop.READ)
  File "C:\Python\lib\asyncio\events.py", line 501, in add_reader
    raise NotImplementedError
NotImplementedError
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "C:\Python\lib\contextlib.py", line 131, in __exit__
    self.gen.throw(type, value, traceback)
  File "C:\Python\lib\site-packages\distributed\utils.py", line 186, in ignoring
    yield
  File "C:\Python\lib\site-packages\distributed\deploy\spec.py", line 607, in close_clusters
    cluster.close(timeout=10)
  File "C:\Python\lib\site-packages\distributed\deploy\cluster.py", line 83, in close
    return self.sync(self._close, callback_timeout=timeout)
  File "C:\Python\lib\site-packages\distributed\deploy\cluster.py", line 162, in sync
    return sync(self.loop, func, *args, **kwargs)
  File "C:\Python\lib\site-packages\distributed\utils.py", line 334, in sync
    raise exc.with_traceback(tb)
  File "C:\Python\lib\site-packages\distributed\utils.py", line 318, in f
    result[0] = yield future
  File "C:\Python\lib\site-packages\tornado\gen.py", line 735, in run
    value = future.result()
  File "C:\Python\lib\site-packages\distributed\deploy\spec.py", line 380, in _close
    self.scale(0)
  File "C:\Python\lib\site-packages\distributed\deploy\spec.py", line 444, in scale
    v["name"] for v in self.scheduler_info["workers"].values()
KeyError: 'workers'

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:7 (4 by maintainers)

github_iconTop GitHub Comments

2reactions
PMeiracommented, Dec 30, 2019
  File "C:\Python\lib\site-packages\tornado\platform\asyncio.py", line 99, in add_handler
    self.asyncio_loop.add_reader(fd, self._handle_events, fd, IOLoop.READ)
  File "C:\Python\lib\asyncio\events.py", line 501, in add_reader
    raise NotImplementedError

That’s a Tornado + Python 3.8 issue on Windows. Python 3.8 changed some defaults. I posted a comment earlier today, Dask Distributed probably needs a small patch: https://github.com/dask/distributed/pull/3249#issuecomment-569539093

0reactions
devin-petersohncommented, Jul 28, 2020

@itamarst It is probably worth updating our minimum Dask and Distributed versions to ensure compatibility with Python 3.8.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Dask Installation - Dask documentation
This installs Dask and all common dependencies, including pandas and NumPy. Dask packages are maintained both on the default channel and on conda-forge....
Read more >
Changelog — Dask.distributed 2022.12.1 documentation
Updated minimum supported version of Dask to 2020.12.0. Added many type annotations and updates to allow for gradually Cythonizing the scheduler.
Read more >
Changelog - Dask documentation
Updated minimum supported version of pandas to 0.25.0 and NumPy to 1.15.1. Support for the pyarrow.dataset API to read_parquet .
Read more >
Install Dask.Distributed
To install the latest version of dask.distributed from the conda-forge repository using conda: ... python -m pip install dask distributed --upgrade ...
Read more >
Development Guidelines - Dask documentation
Dask supports Python versions 3.8, 3.9 and 3.10. Name changes are handled by the dask/compatibility.py file. Test¶. Dask employs extensive unit ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found