__sklearn_clone__ protocol proposal

I propose a __sk_clone__ method that clone calls (if the method exist). This way arbitrary objects can define how they want to be cloned. This is similar to mechanism behind __sk_is_fitted__ + check_is_fitted.

Purpose

There are PR that run into issues with clone, and it would be nice to allow the estimator to define how it wants to be cloned.

1. Metadata routing: https://github.com/scikit-learn/scikit-learn/pull/21284 Needed to change clone to copy an private attribute that is configured outside of __init__
2. Search spaces in estimator: https://github.com/scikit-learn/scikit-learn/pull/21784#issuecomment-979284821 the search space itself needs to be copied over when clone gets called.
3. Ability to customize cloning: https://github.com/scikit-learn/scikit-learn/issues/15371#issuecomment-546655096

[thomasjpfan]

To me the difference is developer API vs user API:

1. __sklearn_clone__: Estimators developers need to think about implementing it (optionally)
2. clone: User facing. This makes it harder to be optional since users would be expecting the method as a “scikit-learn compatible estimator”.

To me, I like namespacing methods with double underscore (__) as it is developer facing and makes it explicit that it is interacting with scikit-learn’s functions. (__sklearn_is_fitted__ -> check_is_fitted).

[jnothman]

A draft SLEP is here: https://github.com/scikit-learn/enhancement_proposals/pull/67

To some extent @GaelVaroquaux’s objections still apply, but I think the core devs largely find the inflexibility of clone to be a limiting/frustrating.