question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Issue with passing pickled parameters when using Local Mode

See original GitHub issue

System information

  • OS Platform and Distribution Linux Ubuntu 16.04:
  • **Ray installed from (source) **:
  • Ray version 0.7.3:
  • Python version 3.6.6:
  • Exact command to reproduce:

Describe the problem

Hi Ray Team!

I use local mode when running some integration tests on a module that I have written using Ray, and I have noticed the functionality of ray change between update 0.7.2 and 0.7.3.

In 0.7.3 When I pass a pickled object as a parameter to my ray worker, I used to expect that the parameter would be returned to me as the pickled object, however while using local mode, it is returned to me as a LocalModeObjectID, and I am unable to retrieve the object using a call to ray.get

If I used 0.7.2 When I pass the same pickled object to my ray worker, it is returned to me as a the pickled object, of type bytes

Has there been an intended change in functionality in Ray’s local mode?

Source code / logs

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:6 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
avnishncommented, Aug 5, 2019

UPDATE: a small example to recreate the problem:

import ray
import pickle
@ray.remote
class ray_worker:

    def __init__(self, pckl_obj):
        self.pckl_obj = pickle.loads(pckl_obj)

if __name__ == "__main__":
    a = 5
    a = pickle.dumps(a)
    ray.init(local_mode=True)
    a_id = ray.put(a)
    worker = ray_worker.remote(a_id)

So the issue specifically happens if you put the pickled object in the object store using ray.put If I don’t ray.put the object into the object store, and instead directly pass the pickled object, then the example works fine.

0reactions
andrewv99commented, Oct 21, 2019

I have only just started using ray / tune. I am experiencing the same issue / symptoms as @jasonnance. @edoakes , do you have any idea of when you might be able to look into this issue ? thanks

Read more comments on GitHub >

github_iconTop Results From Across the Web

pickle — Python object serialization — Python 3.11.1 ...
By default, a pickler object will not have a dispatch_table attribute, and it will instead use the global dispatch table managed by the...
Read more >
Can't pickle Function - Stack Overflow
To pickle functions, you need to use copy_reg, as shown by this example. " Classes, functions, and methods cannot be pickled -- if...
Read more >
How to use Pickle to save and load Variables in Python?
Method 1: Passing the variable. In dumps() method, we can pass the variable, and it will return us the binary string for the...
Read more >
11.1. pickle — Python object serialization - 3.7.9 Documentation
The pickle module is not secure against erroneous or maliciously constructed data. Never unpickle data received from an untrusted or unauthenticated source.
Read more >
Python Tutorial: serialization with pickle - 2020 - BogoToBogo
Serializes the data structure using a data format called the pickle protocol. ... We passed an indent parameter to the json.dump() function, and...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found