Prediction explanations: display Index ID for best/worst prediction explanations

Goal If users intend to understand the best and worst predictions, the API should allow them to see the Index of the explanations.

If the user can provide the index column (perhaps through DataTables), that index ID could be displayed in the reported outputted by explain_predictions_best_worst.

If the user doesn’t provide an index column, no index ID should be displayed in the report.

Proposal Add index ID to the prediction explanation, if the user provides an index column in X.

Note the Index ID:

        Best 1 of 2

                Predicted Probabilities: [benign: 0.0, malignant: 1.0]
                Predicted Value: malignant
                Target Value: malignant
                Cross Entropy: 0.0
                Index ID: 45


                     Feature Name         Feature Value        Contribution to        SHAP Value
                                                                 Prediction
                ================================================================================
                    worst perimeter          155.30                   +                  0.10
                     worst radius             23.14                   +                  0.08
                 worst concave points         0.17                    +                  0.08
                worst fractal dimension       0.09                    -                 -0.00
                   compactness error          0.04                    -                 -0.00
                    worst symmetry            0.22                    -                 -0.00


        Best 2 of 2

                Predicted Probabilities: [benign: 0.0, malignant: 1.0]
                Predicted Value: malignant
                Target Value: malignant
                Cross Entropy: 0.0
                Index ID: 2

                    Feature Name       Feature Value   Contribution to Prediction   SHAP Value
                ==============================================================================
                  worst perimeter         166.10                   +                   0.10
                    worst radius           25.45                   +                   0.08
                worst concave points       0.22                    +                   0.08
                 compactness error         0.03                    -                  -0.00
                 worst compactness         0.21                    -                  -0.00
                   worst symmetry          0.21                    -                  -0.00


        Worst 1 of 2

                Predicted Probabilities: [benign: 0.552, malignant: 0.448]
                Predicted Value: benign
                Target Value: malignant
                Cross Entropy: 0.802
                Index ID: 7

                    Feature Name       Feature Value   Contribution to Prediction   SHAP Value
                ==============================================================================
                  smoothness error         0.00                    +                   0.04
                    mean texture           21.58                   +                   0.03
                   worst texture           30.25                   +                   0.02
                worst concave points       0.11                    -                  -0.02
                    worst radius           15.93                   -                  -0.03
                mean concave points        0.02                    -                  -0.03

[freddyaboulton]

@gsheni Yea you’re right. I agree that adding the index value to the output of explain_predictions_best_worst can add value to the user.

[gsheni]

@kmax12 yes, you are right. Fixed the printout example.

@dsherry Yes, I suppose the caller could get that information if they wanted to. It would require the caller re-run the following (outside of explain_predictions_best_worst)

(regression)

y_pred = pipeline.predict(input_features)
errors = metric(y_true, y_pred)
sorted_scores = errors.sort_values()
best = sorted_scores.index[:num_to_explain]
worst = sorted_scores.index[-num_to_explain:]

• I would then have to find the index IDs of the best/worst score.