TypeError: cannot pickle 'property' object

I experience a problem with Python class properties and joblib. Somehow it is not possible to pickle them. See this minimal reproducible example:

import multiprocessing

from joblib import Parallel, delayed


class Task:
    def run(self):
        print(self.hello_world)

    @property
    def hello_world(self):
        return "hello world"


if __name__ == "__main__":
    Parallel(n_jobs=multiprocessing.cpu_count())([delayed(lambda: Task().run())()])

This code fails raising TypeError: cannot pickle 'property' object. It works if you replace the property with a method get_hello_world and call self.get_hello_world() in Task.run.

Environment:

• Python 3.8.1
• joblib 0.14.1
• macOS Catalina 10.15.2

[KristianHolsheimer]

Thank you @pierreglaser

I ran into this issue in the ray package, which also packages its own copy of cloudpickle. The problem of the unpicklable property is resolved (as per your PR from earlier this year… thanks for that)

I’m now having the same issue with abstractproperty. I’ll see if I can put a quick PR ready for cloudpickle. If not, I’ll submit an issue there.

Thanks again, Kris

[pierreglaser]

Hi, thanks for the report.

FWIW I also checked whether cloudpickle (which I thought was the pickler that loky uses) correctly pickles the object.

FYI, joblib vendors its own cloudpickle (inside joblib.externals.cloudpickle), which gets updated once in a while. So cloudpickle and joblib.externals.cloudpickle may or may not behave the same way depending on which version of joblib and cloudpickle is installed on your environment.

That being said, the last joblib release vendors cloudpickle==1.2.2 which had a bug affecting the pickling of property objects on Python 3.8 (see cloudpipe/cloudpickle#329)

The future joblib release will contain cloudpickle == 1.4.1, and thus should not have this issue.