CalibratedClassifierCV doesn't support groups parameter in fit method

Description

With classes like sklearn.model_selection.GridSearchCV and even skopt.BayesSearchCV they take a cv parameter that can be any object that produces train/test splits. In the case of using sklearn.model_selection.GroupKFold as the cv object, it requires a groups variable to be passed to fit. In both these classes, they allow groups to be passed to fit that’s then passed to the underlying GroupKFold. However, sklearn.calibration.CalibratedClassifierCV doesn’t allow the groups variable to fit even though it says it accepts any cv object.

[adrinjalali]

This is I guess another example of what could potentially be solved by PR #9566. You can check issues #11429 and #4497 for more detailed discussion on the matter. Developers are mostly busy with 0.20 release at the moment. I’m pretty sure we’ll get back to this soon after the release.

In the meantime, you can check the answer I gave to the question “sklearn GridSearchCV not using sample_weight in score function” on stackoverflow useful. I haven’t applied it to GroupKFold, but I suspect it may be applicable there as well. You may need to write a wrapper around GroupKFold which doesn’t expect groups as input.

[mdbecker]

For anyone else that runs into this same issue before SLEP006 is finished, I’ve uploaded a very simple fix based on version 0.24.1 of sklearn here. This works the same was a GridSearchCV and fails gracefully if the cv passed in does not support the groups parameter (which may not be desirable depending on your usecase). @DanLeopold