Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

ValueError while creating distributed scheduler using Client() after importing datatable library

See original GitHub issue

Description

Creating a distributed scheduler using Client() after importing datatable library causes

ValueError: I/O operation on closed file distributed.utils - ERROR - addresses should be strings or tuples, got None

Steps to reproduce

Install datatable using pip install datatable
Save the following code snippet as a file and run it from terminal.

import datatable as dt
from dask.distributed import Client
client = Client()

Environment

OS: MacOS Catalina version 15.1 Python version: 3.7.4 Dask version: 2.9.2

Output of pip freeze

Click==7.0
cloudpickle==1.2.2
colorama==0.4.3
dask==2.9.2
datatable==0.10.1
distributed==2.9.3
HeapDict==1.0.1
msgpack==0.6.2
psutil==5.6.7
PyYAML==5.3
six==1.14.0
sortedcontainers==2.1.0
tblib==1.6.0
toolz==0.10.0
tornado==6.0.3
typesentry==0.2.7
wcwidth==0.1.8
zict==1.0.0

Stack trace

Complete stack trace: log.txt

Traceback (most recent call last): File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/forkserver.py”, line 261, in main Traceback (most recent call last): File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/forkserver.py”, line 261, in main Traceback (most recent call last): File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/forkserver.py”, line 261, in main old_handlers) File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/forkserver.py”, line 297, in _serve_one old_handlers) File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/forkserver.py”, line 297, in _serve_one code = spawn._main(child_r) File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/spawn.py”, line 114, in _main old_handlers) File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/forkserver.py”, line 297, in _serve_one code = spawn._main(child_r) File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/spawn.py”, line 114, in _main prepare(preparation_data) File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/spawn.py”, line 225, in prepare prepare(preparation_data) File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/spawn.py”, line 225, in prepare code = spawn._main(child_r) File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/spawn.py”, line 114, in _main _fixup_main_from_path(data[‘init_main_from_path’]) File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/spawn.py”, line 277, in _fixup_main_from_path _fixup_main_from_path(data[‘init_main_from_path’]) File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/spawn.py”, line 277, in _fixup_main_from_path prepare(preparation_data) File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/spawn.py”, line 225, in prepare run_name=“mp_main”) File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py”, line 263, in run_path run_name=“mp_main”) File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py”, line 263, in run_path _fixup_main_from_path(data[‘init_main_from_path’]) File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/spawn.py”, line 277, in _fixup_main_from_path pkg_name=pkg_name, script_name=fname) File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py”, line 96, in _run_module_code pkg_name=pkg_name, script_name=fname) File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py”, line 96, in _run_module_code run_name=“mp_main”) File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py”, line 263, in run_path mod_name, mod_spec, pkg_name, script_name) File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py”, line 85, in _run_code mod_name, mod_spec, pkg_name, script_name) File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py”, line 85, in _run_code exec(code, run_globals) File “/Users/santhisenan/Documents/projects/test/dask/bug-test/test.py”, line 4, in <module> exec(code, run_globals) File “/Users/santhisenan/Documents/projects/test/dask/bug-test/test.py”, line 4, in <module> pkg_name=pkg_name, script_name=fname) File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py”, line 96, in _run_module_code import datatable as dt File “/Users/santhisenan/.sm-dask/lib/python3.7/site-packages/datatable/init.py”, line 27, in <module> mod_name, mod_spec, pkg_name, script_name) File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py”, line 85, in _run_code from .fread import fread, GenericReader, FreadWarning, _DefaultLogger import datatable as dt File “/Users/santhisenan/.sm-dask/lib/python3.7/site-packages/datatable/fread.py”, line 17, in <module> File “/Users/santhisenan/.sm-dask/lib/python3.7/site-packages/datatable/init.py”, line 27, in <module> exec(code, run_globals) File “/Users/santhisenan/Documents/projects/test/dask/bug-test/test.py”, line 4, in <module> from datatable.utils.typechecks import (TValueError, TTypeError, File “/Users/santhisenan/.sm-dask/lib/python3.7/site-packages/datatable/utils/typechecks.py”, line 14, in <module> import datatable as dt from .fread import fread, GenericReader, FreadWarning, _DefaultLogger File “/Users/santhisenan/.sm-dask/lib/python3.7/site-packages/datatable/init.py”, line 27, in <module> File “/Users/santhisenan/.sm-dask/lib/python3.7/site-packages/datatable/fread.py”, line 17, in <module> from datatable.utils.terminal import term File “/Users/santhisenan/.sm-dask/lib/python3.7/site-packages/datatable/utils/terminal.py”, line 160, in <module> from .fread import fread, GenericReader, FreadWarning, _DefaultLogger File “/Users/santhisenan/.sm-dask/lib/python3.7/site-packages/datatable/fread.py”, line 17, in <module> from datatable.utils.typechecks import (TValueError, TTypeError, File “/Users/santhisenan/.sm-dask/lib/python3.7/site-packages/datatable/utils/typechecks.py”, line 14, in <module> term = Terminal() File “/Users/santhisenan/.sm-dask/lib/python3.7/site-packages/datatable/utils/terminal.py”, line 55, in init from datatable.utils.typechecks import (TValueError, TTypeError, File “/Users/santhisenan/.sm-dask/lib/python3.7/site-packages/datatable/utils/typechecks.py”, line 14, in <module> from datatable.utils.terminal import term File “/Users/santhisenan/.sm-dask/lib/python3.7/site-packages/datatable/utils/terminal.py”, line 160, in <module> from datatable.utils.terminal import term self._blessed_term = blessed.Terminal() File “/Users/santhisenan/.sm-dask/lib/python3.7/site-packages/datatable/utils/terminal.py”, line 160, in <module> File “/Users/santhisenan/.sm-dask/lib/python3.7/site-packages/blessed/terminal.py”, line 165, in init term = Terminal() File “/Users/santhisenan/.sm-dask/lib/python3.7/site-packages/datatable/utils/terminal.py”, line 55, in init term = Terminal() self._keyboard_fd = sys.stdin.fileno() File “/Users/santhisenan/.sm-dask/lib/python3.7/site-packages/datatable/utils/terminal.py”, line 55, in init ValueError: I/O operation on closed file self._blessed_term = blessed.Terminal() File “/Users/santhisenan/.sm-dask/lib/python3.7/site-packages/blessed/terminal.py”, line 165, in init self._blessed_term = blessed.Terminal() File “/Users/santhisenan/.sm-dask/lib/python3.7/site-packages/blessed/terminal.py”, line 165, in init Traceback (most recent call last): File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/forkserver.py”, line 261, in main self._keyboard_fd = sys.stdin.fileno() ValueError: I/O operation on closed file self._keyboard_fd = sys.stdin.fileno() ValueError: I/O operation on closed file old_handlers) File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/forkserver.py”, line 297, in _serve_one code = spawn._main(child_r) File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/spawn.py”, line 114, in _main prepare(preparation_data) File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/spawn.py”, line 225, in prepare _fixup_main_from_path(data[‘init_main_from_path’]) File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/spawn.py”, line 277, in _fixup_main_from_path run_name=“mp_main”) File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py”, line 263, in run_path pkg_name=pkg_name, script_name=fname) File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py”, line 96, in _run_module_code mod_name, mod_spec, pkg_name, script_name) File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py”, line 85, in _run_code exec(code, run_globals) File “/Users/santhisenan/Documents/projects/test/dask/bug-test/test.py”, line 4, in <module> import datatable as dt File “/Users/santhisenan/.sm-dask/lib/python3.7/site-packages/datatable/init.py”, line 27, in <module> from .fread import fread, GenericReader, FreadWarning, _DefaultLogger File “/Users/santhisenan/.sm-dask/lib/python3.7/site-packages/datatable/fread.py”, line 17, in <module> from datatable.utils.typechecks import (TValueError, TTypeError, File “/Users/santhisenan/.sm-dask/lib/python3.7/site-packages/datatable/utils/typechecks.py”, line 14, in <module> from datatable.utils.terminal import term File “/Users/santhisenan/.sm-dask/lib/python3.7/site-packages/datatable/utils/terminal.py”, line 160, in <module> term = Terminal() File “/Users/santhisenan/.sm-dask/lib/python3.7/site-packages/datatable/utils/terminal.py”, line 55, in init self._blessed_term = blessed.Terminal() File “/Users/santhisenan/.sm-dask/lib/python3.7/site-packages/blessed/terminal.py”, line 165, in init self._keyboard_fd = sys.stdin.fileno() ValueError: I/O operation on closed file distributed.utils - ERROR - addresses should be strings or tuples, got None

Issue Analytics

State:
Created 4 years ago
Comments:6 (3 by maintainers)

Top GitHub Comments

1reaction

TomAugspurgercommented, Jan 20, 2020

Putting your code in a __main__ block is the typical recommendation for any code using multiple processes. We have another issue working to provide feedback on when this is not done, but it’s a bit tricky. Glad to hear you got it figured out.

1reaction

TomAugspurgercommented, Jan 19, 2020

Have you created an issue on the datatable repository? The datatable maintainers may have a better idea what all happens as part of import datatable.

Top Results From Across the Web

using dask without > client = Client() - python - Stack Overflow

my question is why should I run this code before? from dask.distributed import Client, progress client = Client() client. and not just import...

Python client library | Google Cloud

We can use a Table to: create() the table ... from google.cloud.bigtable.row_filters import PassAllFilter ... ValueError: If the parameters are invalid.

ray.data.read_api — Ray 2.2.0 - the Ray documentation

import logging import os from typing import TYPE_CHECKING, Any, Dict, List, Optional, Tuple, TypeVar, Union import numpy as np import ray from ray.data....

Discussion - Quickbase Community - Higher Logic

User invite email - Is there a way to set a default message/subject line instead of typing it in each time I create...

Release Notes — Airflow Documentation

Optimize TI.xcom_pull() with explicit task_ids and map_indexes (#27699) ... To create a DAG that runs whenever a Dataset is updated use the new...