_partial_dependence_brute uses unnecessary memory

Describe the bug

When users run _partial_dependence_brute from inspection_partial_dependence.py, the following code is executed for every grid point (usually 100 times), in lines 149-150:

 for new_values in grid:
     X_eval = X.copy()

Since each loop overwrites a single feature at a time, it would be more efficient to move the X.copy() prior to the loop. Unless the model itself mutates the dataset, I don’t think there’s a reason to make a new copy on every loop.

For large datasets, this causes extreme memory use.

Steps/Code to Reproduce

N/A

Expected Results

Expected results are actual results, but with more memory use.

Actual Results

Expected results are actual results, but with more memory use.

Versions

Since 0.23, or whenever partial_dependence was introduced.

[glemaitre]

Ups my bad, I went a bit more in-depth in the code and you are right. At this stage in the code, we are handling a single interaction and we will always overwrite the same targeted feature(s). I thought that the features here was corresponding to loop over all the possible interactions.

We can safely put the copy outside of the loop then. If the garbage collection was working as expected then we should not have any gain regarding the memory usage. However, we avoid triggering copies which could be a potential speed-up then.

[jimbudarz]

My only concern would be if the model is a pipeline which modifies the dataframe inplace during prediction.