Feature Request: .map using dicts

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Mint 19.1 (Ubuntu 18.04)
• Modin installed from (source or binary): binary
• Modin version: 0.6.0
• Python version: 3.7.3
• Exact command to reproduce:

import modin.pandas as pd

data = {
    "column": [0,1]
}

df = pd.DataFrame(data)

mapper = {
    0: "A",
    1: "B"
}

df.loc[:, "column"] = df.loc[:, "column"].map(mapper)

Describe the problem

Maybe it isn’t a problem, but just a feature that has yet to be implemented?

Source code / logs

Traceback (most recent call last):
  File "/home/jason/.virtualenvs/bert_poc/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3326, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-8-df805f17ff4c>", line 12, in <module>
    df.loc[:, "column"] = df.loc[:, "column"].map(mapper)
  File "/home/jason/.virtualenvs/bert_poc/lib/python3.7/site-packages/modin/pandas/indexing.py", line 256, in __setitem__
    super(_LocIndexer, self).__setitem__(row_lookup, col_lookup, item)
  File "/home/jason/.virtualenvs/bert_poc/lib/python3.7/site-packages/modin/pandas/indexing.py", line 145, in __setitem__
    item = self._broadcast_item(row_lookup, col_lookup, item, to_shape)
  File "/home/jason/.virtualenvs/bert_poc/lib/python3.7/site-packages/modin/pandas/indexing.py", line 172, in _broadcast_item
    item = np.array(item)
  File "/home/jason/.virtualenvs/bert_poc/lib/python3.7/site-packages/modin/pandas/series.py", line 123, in __array__
    return super(Series, self).__array__(dtype).flatten()
  File "/home/jason/.virtualenvs/bert_poc/lib/python3.7/site-packages/modin/pandas/base.py", line 3145, in __array__
    arr = self.to_numpy(dtype)
  File "/home/jason/.virtualenvs/bert_poc/lib/python3.7/site-packages/modin/pandas/series.py", line 1024, in to_numpy
    return super(Series, self).to_numpy(dtype, copy).flatten()
  File "/home/jason/.virtualenvs/bert_poc/lib/python3.7/site-packages/modin/pandas/base.py", line 2946, in to_numpy
    arr = self._query_compiler.to_numpy()
  File "/home/jason/.virtualenvs/bert_poc/lib/python3.7/site-packages/modin/backends/pandas/query_compiler.py", line 143, in to_numpy
    arr = self._modin_frame.to_numpy()
  File "/home/jason/.virtualenvs/bert_poc/lib/python3.7/site-packages/modin/engines/base/frame/data.py", line 1137, in to_numpy
    return self._frame_mgr_cls.to_numpy(self._partitions)
  File "/home/jason/.virtualenvs/bert_poc/lib/python3.7/site-packages/modin/engines/ray/generic/frame/partition_manager.py", line 20, in to_numpy
    for row in partitions
  File "/home/jason/.virtualenvs/bert_poc/lib/python3.7/site-packages/ray/worker.py", line 2247, in get
    raise value
ray.exceptions.RayTaskError: ray_worker (pid=26958, host=LH002609)
ray.exceptions.RayTaskError: ray_worker (pid=26965, host=LH002609)
  File "/home/jason/.virtualenvs/bert_poc/lib/python3.7/site-packages/modin/engines/ray/pandas_on_ray/frame/partition.py", line 197, in deploy_ray_func
    result = func(partition, **kwargs)
  File "/home/jason/.virtualenvs/bert_poc/lib/python3.7/site-packages/modin/data_management/functions/mapfunction.py", line 10, in <lambda>
    lambda x: function(x, *args, **kwargs), *call_args, **call_kwds
  File "/home/jason/.virtualenvs/bert_poc/lib/python3.7/site-packages/pandas/core/frame.py", line 6979, in applymap
    return self.apply(infer)
  File "/home/jason/.virtualenvs/bert_poc/lib/python3.7/site-packages/pandas/core/frame.py", line 6913, in apply
    return op.get_result()
  File "/home/jason/.virtualenvs/bert_poc/lib/python3.7/site-packages/pandas/core/apply.py", line 186, in get_result
    return self.apply_standard()
  File "/home/jason/.virtualenvs/bert_poc/lib/python3.7/site-packages/pandas/core/apply.py", line 292, in apply_standard
    self.apply_series_generator()
  File "/home/jason/.virtualenvs/bert_poc/lib/python3.7/site-packages/pandas/core/apply.py", line 321, in apply_series_generator
    results[i] = self.f(v)
  File "/home/jason/.virtualenvs/bert_poc/lib/python3.7/site-packages/pandas/core/frame.py", line 6977, in infer
    return lib.map_infer(x.astype(object).values, func)
  File "pandas/_libs/lib.pyx", line 2228, in pandas._libs.lib.map_infer
TypeError: ("'dict' object is not callable", 'occurred at index column')

[devin-petersohn]

Thanks @jmwoloso, there is a more concise way of converting to/from pandas for a single operation, though it’s mostly used internally (editing from your old code):

import modin.pandas as pd

data = {
    "column": [0,1]
}

df = pd.DataFrame(data)

mapper = {
    0: "A",
    1: "B"
}

df.loc[:, "column"] = df.loc[:, "column"]._default_to_pandas(lambda s: s.map(mapper))

This will do everything from converting to a pandas DataFrame, doing the operation, then converting back. Let me know if it is working for you! (It’s a little cleaner and less chance of bugs I think)

[devin-petersohn]

Hi @jmwoloso, thanks for reporting!

This functionality isn’t implemented yet, but it shouldn’t be throwing this kind of error. I don’t think the overhead for implementing it is very high compared to fixing the error message, so we’ll probably just implement for the next release. Thanks!