ray.get fails to get pandas series

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 16.04 Ray installed from (source or binary): binary Ray version: Latest nightly wheel Python version: 3.7 Exact command to reproduce:

Ray.get fails to get a pandas series.

import ray
import pandas as pd
_ = ray.init()

bad_series = pd.Series([pd.Timestamp('2017-03-24 19:57:49.854052+0000', tz='UTC')])
ray.get(ray.put(bad_series))
----
2019-04-16 16:53:01,942 INFO node.py:469 -- Process STDOUT and STDERR is being redirected to /tmp/ray/session_2019-04-16_16-53-01_16726/logs.
2019-04-16 16:53:02,046 INFO services.py:407 -- Waiting for redis server at 127.0.0.1:42315 to respond...
2019-04-16 16:53:02,157 INFO services.py:407 -- Waiting for redis server at 127.0.0.1:12890 to respond...
2019-04-16 16:53:02,159 INFO services.py:804 -- Starting Redis shard with 10.0 GB max memory.
2019-04-16 16:53:02,168 INFO node.py:483 -- Process STDOUT and STDERR is being redirected to /tmp/ray/session_2019-04-16_16-53-01_16726/logs.
2019-04-16 16:53:02,169 WARNING services.py:1291 -- Warning: Capping object memory store to 20.0GB. To increase this further, specify `object_store_memory` when calling ray.init() or ray start.
2019-04-16 16:53:02,169 INFO services.py:1439 -- Starting the Plasma object store with 20.0 GB memory using /dev/shm.
Traceback (most recent call last):
  File "minimal.py", line 6, in <module>
    ray.get(ray.put(bad_series))
  File "/home/daniel/mbtf/conda/envs/mbtf/lib/python3.7/site-packages/ray/worker.py", line 2194, in get
    value = worker.get_object([object_ids])[0]
  File "/home/daniel/mbtf/conda/envs/mbtf/lib/python3.7/site-packages/ray/worker.py", line 508, in get_object
    final_results = self.retrieve_and_deserialize(plain_object_ids, 0)
  File "/home/daniel/mbtf/conda/envs/mbtf/lib/python3.7/site-packages/ray/worker.py", line 424, in retrieve_and_deserialize
    serialization_context,
  File "/home/daniel/mbtf/conda/envs/mbtf/lib/python3.7/site-packages/ray/worker.py", line 473, in _deserialize_object_from_arrow
    return pyarrow.deserialize(data, serialization_context)
  File "pyarrow/serialization.pxi", line 458, in pyarrow.lib.deserialize
  File "pyarrow/serialization.pxi", line 421, in pyarrow.lib.deserialize_from
  File "pyarrow/serialization.pxi", line 272, in pyarrow.lib.SerializedPyObject.deserialize
  File "pyarrow/serialization.pxi", line 180, in pyarrow.lib.SerializationContext._deserialize_callback
  File "/home/daniel/mbtf/conda/envs/mbtf/lib/python3.7/site-packages/ray/pyarrow_files/pyarrow/serialization.py", line 163, in _deserialize_pandas_series
    deserialized = _deserialize_pandas_dataframe(data)
  File "/home/daniel/mbtf/conda/envs/mbtf/lib/python3.7/site-packages/ray/pyarrow_files/pyarrow/serialization.py", line 152, in _deserialize_pandas_dataframe
    return pdcompat.serialized_dict_to_dataframe(data)
  File "/home/daniel/mbtf/conda/envs/mbtf/lib/python3.7/site-packages/ray/pyarrow_files/pyarrow/pandas_compat.py", line 550, in serialized_dict_to_dataframe
    for block in data['blocks']]
  File "/home/daniel/mbtf/conda/envs/mbtf/lib/python3.7/site-packages/ray/pyarrow_files/pyarrow/pandas_compat.py", line 550, in <listcomp>
    for block in data['blocks']]
  File "/home/daniel/mbtf/conda/envs/mbtf/lib/python3.7/site-packages/ray/pyarrow_files/pyarrow/pandas_compat.py", line 571, in _reconstruct_block
    dtype = make_datetimetz(item['timezone'])
  File "/home/daniel/mbtf/conda/envs/mbtf/lib/python3.7/site-packages/ray/pyarrow_files/pyarrow/pandas_compat.py", line 586, in make_datetimetz
    return _pandas_api.datetimetz_type('ns', tz=tz)
TypeError: 'NoneType' object is not callable

[markgoodhead]

@robertnishihara any update on this issue? are you awaiting a fix in pyarrow or is there a newer version of pyarrow ray needs to move on to?

[simon-mo]

This is an apache arrow issue. Please re-open if this still fails.