Can Autosklearn be used with SHAP?

Can the model be used with SHAP?

Currently

import shap
explainer = shap.Explainer(model)

results in

---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
<ipython-input-361-5c713ec694d6> in <module>
----> 1 explainer = shap.Explainer(model)

~/anaconda3/envs/python3/lib/python3.6/site-packages/shap/explainers/_explainer.py in __init__(self, model, masker, link, algorithm, output_names, feature_names, **kwargs)
    145                 # if we get here then we don't know how to handle what was given to us
    146                 else:
--> 147                     raise Exception("The passed model is not callable and cannot be analyzed directly with the given masker! Model: " + str(model))
    148 
    149             # build the right subclass

Exception: The passed model is not callable and cannot be analyzed directly with the given masker! Model: AutoSklearn2Classifier(delete_output_folder_after_terminate=False,
                       ensemble_size=1, memory_limit=7000, metric=f1, n_jobs=8,
                       output_folder='automl4_preds', per_run_time_limit=480,
                       time_left_for_this_task=600)

System Details (if relevant)

auto-sklearn 0.12.7 shap 0.38.1 Running on Linux?

[mfeurer]

You can also find an example in this notebook: https://github.com/automl/auto-sklearn-talks/blob/main/2021_07_28_EuroPython/Tutorial-Regression.ipynb

[eddiebergman]

Hi @roger-yu-ds,

Looking at shap source code for Explainer.__init__(), it seems that it will not work out of the box as you have given.

I would suggest using their model agnostic Explainer.

estimator = AutoSklearnClassifier(...) 
explainer = shap.KernelExplainer(estimator.predict_proba, shap.sample(X_train, 128))

I would also advise updating auto-sklearn to 14.0 as we fixed some issues with probability outputs in some scenarios. Hopefully it won’t matter but if you face issues with incorrect probability sizes, this should fix it.