Issue with passing pickled parameters when using Local Mode
See original GitHub issueSystem information
- OS Platform and Distribution Linux Ubuntu 16.04:
- **Ray installed from (source) **:
- Ray version 0.7.3:
- Python version 3.6.6:
- Exact command to reproduce:
Describe the problem
Hi Ray Team!
I use local mode when running some integration tests on a module that I have written using Ray, and I have noticed the functionality of ray change between update 0.7.2 and 0.7.3.
In 0.7.3
When I pass a pickled object as a parameter to my ray worker, I used to expect that the parameter would be returned to me as the pickled object, however while using local mode, it is returned to me as a LocalModeObjectID
, and I am unable to retrieve the object using a call to ray.get
If I used 0.7.2
When I pass the same pickled object to my ray worker, it is returned to me as a the pickled object, of type bytes
Has there been an intended change in functionality in Ray’s local mode?
Source code / logs
Issue Analytics
- State:
- Created 4 years ago
- Comments:6 (4 by maintainers)
Top Results From Across the Web
pickle — Python object serialization — Python 3.11.1 ...
By default, a pickler object will not have a dispatch_table attribute, and it will instead use the global dispatch table managed by the...
Read more >Can't pickle Function - Stack Overflow
To pickle functions, you need to use copy_reg, as shown by this example. " Classes, functions, and methods cannot be pickled -- if...
Read more >How to use Pickle to save and load Variables in Python?
Method 1: Passing the variable. In dumps() method, we can pass the variable, and it will return us the binary string for the...
Read more >11.1. pickle — Python object serialization - 3.7.9 Documentation
The pickle module is not secure against erroneous or maliciously constructed data. Never unpickle data received from an untrusted or unauthenticated source.
Read more >Python Tutorial: serialization with pickle - 2020 - BogoToBogo
Serializes the data structure using a data format called the pickle protocol. ... We passed an indent parameter to the json.dump() function, and...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
UPDATE: a small example to recreate the problem:
So the issue specifically happens if you put the pickled object in the object store using
ray.put
If I don’tray.put
the object into the object store, and instead directly pass the pickled object, then the example works fine.I have only just started using ray / tune. I am experiencing the same issue / symptoms as @jasonnance. @edoakes , do you have any idea of when you might be able to look into this issue ? thanks