`.cache()` fails for MaskedArrays

joblib caching implementation for arguments that are ndarrays depends on .viewing the object as uint8. This is not possible for masked arrays, because their size will change and therefore the mask doesn’t fit anymore. See also https://github.com/numpy/numpy/issues/10074.

$ cat mwe.py 
#!/usr/bin/env python3.6

import numpy
import joblib

memory = joblib.Memory(cachedir="/tmp", verbose=1)

@memory.cache
def foo(x):
    return x+x

x = numpy.ma.MaskedArray(
        numpy.zeros(10, "uint16"),
        mask=numpy.zeros(10, "?"))

print(foo(x))
$ ./mwe.py 
Traceback (most recent call last):
  File "./mwe.py", line 16, in <module>
    print(foo(x))
  File "/dev/shm/gerrit/venv/stable-3.6/lib/python3.6/site-packages/joblib/memory.py", line 562, in __call__
    return self._cached_call(args, kwargs)[0]
  File "/dev/shm/gerrit/venv/stable-3.6/lib/python3.6/site-packages/joblib/memory.py", line 499, in _cached_call
    output_dir, argument_hash = self._get_output_dir(*args, **kwargs)
  File "/dev/shm/gerrit/venv/stable-3.6/lib/python3.6/site-packages/joblib/memory.py", line 585, in _get_output_dir
    argument_hash = self._get_argument_hash(*args, **kwargs)
  File "/dev/shm/gerrit/venv/stable-3.6/lib/python3.6/site-packages/joblib/memory.py", line 579, in _get_argument_hash
    coerce_mmap=(self.mmap_mode is not None))
  File "/dev/shm/gerrit/venv/stable-3.6/lib/python3.6/site-packages/joblib/hashing.py", line 263, in hash
    return hasher.hash(obj)
  File "/dev/shm/gerrit/venv/stable-3.6/lib/python3.6/site-packages/joblib/hashing.py", line 69, in hash
    self.dump(obj)
  File "/home/users/gholl/lib/python3.6/pickle.py", line 409, in dump
    self.save(obj)
  File "/dev/shm/gerrit/venv/stable-3.6/lib/python3.6/site-packages/joblib/hashing.py", line 243, in save
    Hasher.save(self, obj)
  File "/dev/shm/gerrit/venv/stable-3.6/lib/python3.6/site-packages/joblib/hashing.py", line 95, in save
    Pickler.save(self, obj)
  File "/home/users/gholl/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/home/users/gholl/lib/python3.6/pickle.py", line 821, in save_dict
    self._batch_setitems(obj.items())
  File "/dev/shm/gerrit/venv/stable-3.6/lib/python3.6/site-packages/joblib/hashing.py", line 147, in _batch_setitems
    Pickler._batch_setitems(self, iter(sorted(items)))
  File "/home/users/gholl/lib/python3.6/pickle.py", line 852, in _batch_setitems
    save(v)
  File "/dev/shm/gerrit/venv/stable-3.6/lib/python3.6/site-packages/joblib/hashing.py", line 212, in save
    self._getbuffer(obj_c_contiguous.view(self.np.uint8)))
  File "/dev/shm/gerrit/venv/stable-3.6/lib/python3.6/site-packages/numpy/ma/core.py", line 3146, in view
    output = ndarray.view(self, dtype)
  File "/dev/shm/gerrit/venv/stable-3.6/lib/python3.6/site-packages/numpy/ma/core.py", line 3425, in __setattr__
    self._mask.shape = self.shape
ValueError: cannot reshape array of size 10 into shape (20,)

Using Python 3.6, numpy 1.13.3, and joblib 0.11.

[Jerry-Ma]

I just hit this bug. Any update on this?

[lesteve]

I can reproduce the bug. I guess a work-around is to have the memoized function take two arrays as argument, one for the array without mask, one for the mask.

A PR with a non-regression test would be more than welcome, maybe a quick fix is to exclude masked arrays from this if clause here:

https://github.com/joblib/joblib/blob/cbabe65d9b1b52a7be14885f258052d1f26ddb6f/joblib/hashing.py#L190