[Release][Windows] Dashboard agent broken in 1.2.0 release candidate
See original GitHub issueWhat is the problem?
In the 1.2.0 release candidate, the dashboard agent seems to be completely broken and spams the console from the background. This infinite loop of errors makes Ray pretty much unusable.
(p37) C:\Users\Administrator>python
Python 3.7.9 (default, Aug 31 2020, 17:10:11) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import ray
>>> ray.init()
2021-02-10 03:05:34,529 INFO services.py:1174 -- View the Ray dashboard at [1m[32mhttp://127.0.0.1:8265[39m[22m
{'node_ip_address': '172.31.11.8', 'raylet_ip_address': '172.31.11.8', 'redis_address': '172.31.11.8:6379', 'object_store_address': 'tcp://127.0.0.1:58119', 'raylet_socket_name': 'tcp://127.0.0.1:56755', 'webui_url': '127.0.0.1:8265', 'session_dir': 'C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\2\\ray\\session_2021-02-10_03-05-33_888831_4556', 'metrics_export_port': 54734, 'node_id': 'dd5091720488d7393b326ddfac13adab6bfb97e048484f5ed87d08d8'}
>>> 2021-02-10 03:05:45,544 WARNING worker.py:1107 -- The agent on node EC2AMAZ-6T8IHN5 failed with the following error:
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\envs\p37\lib\site-packages\ray\new_dashboard/agent.py", line 312, in <module>
loop.run_until_complete(agent.run())
File "C:\ProgramData\Anaconda3\envs\p37\lib\asyncio\base_events.py", line 587, in run_until_complete
return future.result()
File "C:\ProgramData\Anaconda3\envs\p37\lib\site-packages\ray\new_dashboard/agent.py", line 188, in run
await asyncio.gather(check_parent_task,
UnboundLocalError: local variable 'check_parent_task' referenced before assignment
[2m[36m(pid=None)[0m Traceback (most recent call last):
[2m[36m(pid=None)[0m File "C:\ProgramData\Anaconda3\envs\p37\lib\site-packages\ray\new_dashboard/agent.py", line 323, in <module>
[2m[36m(pid=None)[0m raise e
[2m[36m(pid=None)[0m File "C:\ProgramData\Anaconda3\envs\p37\lib\site-packages\ray\new_dashboard/agent.py", line 312, in <module>
[2m[36m(pid=None)[0m loop.run_until_complete(agent.run())
[2m[36m(pid=None)[0m File "C:\ProgramData\Anaconda3\envs\p37\lib\asyncio\base_events.py", line 587, in run_until_complete
[2m[36m(pid=None)[0m return future.result()
[2m[36m(pid=None)[0m File "C:\ProgramData\Anaconda3\envs\p37\lib\site-packages\ray\new_dashboard/agent.py", line 188, in run
[2m[36m(pid=None)[0m await asyncio.gather(check_parent_task,
[2m[36m(pid=None)[0m UnboundLocalError: local variable 'check_parent_task' referenced before assignment
[2m[36m(pid=None)[0m --- Logging error ---
[2m[36m(pid=None)[0m Traceback (most recent call last):
[2m[36m(pid=None)[0m File "C:\ProgramData\Anaconda3\envs\p37\lib\logging\handlers.py", line 69, in emit
[2m[36m(pid=None)[0m if self.shouldRollover(record):
[2m[36m(pid=None)[0m File "C:\ProgramData\Anaconda3\envs\p37\lib\logging\handlers.py", line 183, in shouldRollover
[2m[36m(pid=None)[0m self.stream = self._open()
[2m[36m(pid=None)[0m File "C:\ProgramData\Anaconda3\envs\p37\lib\logging\__init__.py", line 1116, in _open
[2m[36m(pid=None)[0m return open(self.baseFilename, self.mode, encoding=self.encoding)
[2m[36m(pid=None)[0m NameError: name 'open' is not defined
[2m[36m(pid=None)[0m Call stack:
[2m[36m(pid=None)[0m File "C:\ProgramData\Anaconda3\envs\p37\lib\site-packages\aiohttp\client.py", line 337, in __del__
[2m[36m(pid=None)[0m self._loop.call_exception_handler(context)
[2m[36m(pid=None)[0m File "C:\ProgramData\Anaconda3\envs\p37\lib\asyncio\base_events.py", line 1645, in call_exception_handler
[2m[36m(pid=None)[0m self.default_exception_handler(context)
[2m[36m(pid=None)[0m File "C:\ProgramData\Anaconda3\envs\p37\lib\asyncio\base_events.py", line 1619, in default_exception_handler
[2m[36m(pid=None)[0m logger.error('\n'.join(log_lines), exc_info=exc_info)
[2m[36m(pid=None)[0m Message: 'Unclosed client session\nclient_session: <aiohttp.client.ClientSession object at 0x0000020B6436E488>'
[2m[36m(pid=None)[0m Arguments: ()
2021-02-10 03:05:47,482 WARNING worker.py:1107 -- The agent on node EC2AMAZ-6T8IHN5 failed with the following error:
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\envs\p37\lib\site-packages\ray\new_dashboard/agent.py", line 312, in <module>
loop.run_until_complete(agent.run())
File "C:\ProgramData\Anaconda3\envs\p37\lib\asyncio\base_events.py", line 587, in run_until_complete
return future.result()
File "C:\ProgramData\Anaconda3\envs\p37\lib\site-packages\ray\new_dashboard/agent.py", line 188, in run
await asyncio.gather(check_parent_task,
UnboundLocalError: local variable 'check_parent_task' referenced before assignment
Ray version and other system information (Python version, TensorFlow version, OS):
Reproduction (REQUIRED)
Please provide a short code snippet (less than 50 lines if possible) that can be copy-pasted to reproduce the issue. The snippet should have no external library dependencies (i.e., use fake or mock data / environments):
Use python 3.7 (for example using conda on the aws windows 2016 deep learning ami). Install the 1.2.0 release candidate with
pip install --index-url https://test.pypi.org/simple/ ray
run
import ray
ray.init()
If the code snippet cannot be run by itself, the issue will be closed with “needs-repro-script”.
- I have verified my script runs in a clean environment and reproduces the issue.
- I have verified the issue also occurs with the latest wheels.
Issue Analytics
- State:
- Created 3 years ago
- Comments:13 (10 by maintainers)
Top Results From Across the Web
InLong v1.2.0
Use the links below to download the Apache InLong from one of our mirrors. 1.2.0 release. Name, Version, Date, Downloads. InLong Source Code ......
Read more >BP Rewrites 1.2.0 maintenance release - BuddyPress.org
Immediately available is BP Rewrites 1.2.0. This maintenance release fixes two bugs. For details on the changes, please read the 1.2.0 ...
Read more >Hotfixes - AppDynamics Documentation
This page lists hotfixes in the 4.5.x release. When artifacts are updated, the version number and availability date are listed below.
Read more >SolarWinds RMM: Mac Agent 3.0 Release Candidate (RC ...
With this update, we're bringing the functionality of the standalone Mac-MSP interface into the RMM dashboard, as we continue to ensure we offer ......
Read more >1.0.0-rc.6 release notes - Announcements - Drone
When will 1.0.0-final be released? · Agent Configuration. If you are using Drone with agents you need to pass DRONE_AGENTS_ENABLED=true to the server....
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
fixed by https://github.com/ray-project/ray/pull/19575
Unfortunately it just doesn’t work on windows. We explicitely disabled it because it to contain the damage, since those errors made the rest of ray unusuable because it was spamming its errors.
Contributions are certainly welcome from people with windows expertise 😉