Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

FunctionTransformer Pickling Error

See original GitHub issue

Description

When pickling a FunctionTransformer transformer instance, the wrapped function does not get pickled:

AttributeError: Can’t get attribute ‘f’ on <module ‘main’ …>

Steps/Code to Reproduce

from sklearn.preprocessing import FunctionTransformer
import pickle

def f(x): return x*x
pickled = pickle.dumps(FunctionTransformer(f))

del f
unpickled = pickle.loads(pickled) # Causes the exception

Versions

Windows-7-6.1.7601-SP1 Python 3.6.5 (v3.6.5:f59c0932b4, Mar 28 2018, 16:07:46) [MSC v.1900 32 bit (Intel)] NumPy 1.14.4 SciPy 1.1.0 Scikit-Learn 0.19.1

Issue Analytics

State:
Created 5 years ago
Reactions:2
Comments:11 (5 by maintainers)

Top GitHub Comments

3reactions

urigorencommented, Jan 2, 2019

It does persists with joblib

See:

from sklearn.preprocessing import FunctionTransformer
import joblib

def f(x): return x*x
joblib.dump(FunctionTransformer(f), "t.tmp")

del f
unpickled = joblib.load("t.tmp")

2reactions

happyhumancommented, Jun 6, 2019

Was this issue resolved? I am having the same issue. scikit-learn version: 0.20.2 and 0.21.2 both.

Top Results From Across the Web

pickling error for a sklearn pipeline with custom transformations

I am trying to pickle a Pipeline with some custom transformation over my features and store it in Google cloud storage but i...

Module Main has No Attribute... (on Pipelines and Pickles)

In this post I'll present a common problem that occurs when serializing and restoring scikit-learn pipelines, as well as a solution that I've ......

The implications of pickling ML models - - Uwe's Blog

When you have trained a machine learning model (pipeline), you will make predictions directly afterwards to assess its quality.

sklearn.preprocessing.FunctionTransformer

Constructs a transformer from an arbitrary callable. A FunctionTransformer forwards its X (and optionally y) arguments to a user-defined function or function ...

FunctionTransformer in Sklearn2PMML - Google Groups

problem is that this callable must be "persistable" when the FunctionTransformer is dumped into a Pickle file; this condition is