pyarrow._plasma.ObjectNotAvailable when passing multiple copies of the same object to a taks

What is the problem?

Ray no longer supports passing multiple copies of the same ObjectID to a task as of 0.8.1. This works in 0.8.0. In Modin we often reuse partitions and data to reduce our memory footprint. This issue is significant and we cannot update our Ray dependency until it is resolved.

Ray version and other system information (Python version, TensorFlow version, OS): 0.8.1+

Reproduction (REQUIRED)

Please provide a script that can be run to reproduce the issue. The script should have no external library dependencies (i.e., use fake or mock data / environments):

import ray
ray.init()

a = ray.put(1)

@ray.remote
def f(a, b):
    return a + b

ray.get(f.remote(a, a))

Traceback produced:

---------------------------------------------------------------------------
RayTaskError(TypeError)                   Traceback (most recent call last)
<ipython-input-8-fa22ff5137ff> in <module>
----> 1 ray.get(f.remote(z, z))

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ray/worker.py in get(object_ids, timeout)
   1502                     worker.core_worker.dump_object_store_memory_usage()
   1503                 if isinstance(value, RayTaskError):
-> 1504                     raise value.as_instanceof_cause()
   1505                 else:
   1506                     raise value

RayTaskError(TypeError): ray::__main__.f() (pid=42163, ip=192.168.42.4)
  File "python/ray/_raylet.pyx", line 452, in ray._raylet.execute_task
  File "<ipython-input-6-8ee2d2129fdd>", line 3, in f
TypeError: unsupported operand type(s) for +: 'int' and 'type'

The second object is of type pyarrow._plasma.ObjectNotAvailable

cc @gshimansky cc @simon-mo

• I have verified my script runs in a clean environment and reproduces the issue.
• I have verified the issue also occurs with the latest wheels.

[devin-petersohn]

@simon-mo The function signature will have to change in your example as well, and the new task will have to do a ray.get on the list passed in.

[devin-petersohn]

Thanks @edoakes and @simon-mo!