Why do I have to retrain the model every time?

Describe the solution you’d like I want to be able to use ResidualsPlot and PredictionError classes without having to retrain the model every time. I want to be able to train the model on my own and pass in the y_actual, y_predicted, and X_test and make the plots. I have tried to use the draw() method but I get errors.

Is your feature request related to a problem? Please describe. Very frustrating to have to repeatedly train the model. I am having trouble even believing that this how it was designed, I hope I’m just missing something

Examples I want to be able to do this:

X_train, X_test, y_train, y_test = train_test_split(X, y)
lr = LinearRegression()
lr.fit(X_train, y_train)
y_pred = lr.predict(X_test)
resid = y_test - y_pred
resplot = ResidualsPlot()  
resplot.draw(y_test, y_pred)
errplot = PredictionError()
errplot.draw(y_test, y_pred)

[bbengfort]

@pedropalb I think there is a blog post on this topic, but until I find the link - I wanted to mention that I’ve opened a PR that adds this functionality to the yellowbrick contrib module: #1189

[bbengfort]

Hi @pedropalb - thanks for using Yellowbrick! We currently don’t have plans to create such an interface. The issue is that Yellowbrick often takes advantage of the properties of models, particularly the learned attributes such as coef_ in order to produce its visualizations. There are a few visualizers that can produce results without the estimator, but we’ve erred on the side of integrating into the ML workflow. (Note that we’re still looking to produce Keras estimators for Yellowbrick - #1106).

However, it is hopefully pretty simple to produce an “estimator” that can do what you’d like. I thought we actually had this already in the contrib module, but for some reason I can’t find it - I’ll investigate and get back to you. But the basic sketch is as follows:

from sklearn.base import BaseEstimator
from sklearn.metrics import f1_score

class PassthroughEstimator(BaseEstimator):

    def __init__(self, y_pred, estimator_type="classifier"):
        self._y_pred = y_pred
        self._estimator_type = estimator_type        

    def fit(self, X, y=None):
        return self
    
    def predict(self, X):
        return self._y_pred
    
    def score(self, X, y=None):
        return f1_score(y, self._y_pred)

You may get some issues if Yellowbrick looks for an learned attribute you can’t find; but you can add them to the passthrough from your Keras model to make it work. Hope that helps!