HistGradientBoosting* interaction constraints

Describe the workflow you want to enable

I’d like to use HistGradientBoostingClassifier and HistGradientBoostingRegressor with the possibility to set interaction constraints for certain features. As said in https://github.com/microsoft/LightGBM/issues/2884#issue-577348269, it is one way to make those black boxes more intuitive and interpretable. In addition, it makes it much more easy to marginalize over those features.

Additional context

LightGBM has interaction_constraints, see their docs. XGBoost has them, see their docs. Have also a look at the XGBoost tutorial on interaction constraints for a nice visualization and for potential benefits:

• Better predictive performance from focusing on interactions that work – whether through domain specific knowledge or algorithms that rank interactions
• Less noise in predictions; better generalization
• More control to the user on what the model can fit. For example, the user may want to exclude some interactions even if they perform well due to regulatory constraints

[lorentzenchr]

BTW, interaction constraints for HGBT are listed on the priorities for 2021 from the Technical Committee, point 3 in https://scikit-learn.fondation-inria.fr/technical-committee-november-5-2020-fr/.

[lorentzenchr]

@mayer79 Thank you.

A paper that uses more complicated interaction constraints for bundling location related features together is https://dx.doi.org/10.2139/ssrn.3924412.