Add `u_based_decision` optionality to `sklearn.decomposition.PCA`

scikit-learn’s PCA uses scipy.linalg.svd to compute the singular value decomposition of X.

Because the solution is unique up to a change in sign in pairs of left and right singular vectors, the .fit method then uses svd_flip to enforce deterministic output.

This consists of

Adjust[ing] the columns of u and the rows of v such that the loadings in the
columns in u that are largest in absolute value are always positive.

And svd_flip comes with a parameter u_based_decision (default True) that allows for flipping this logic if the parameter is False. However, this parameter is not present in PCA.fit() or any of its similar methods.

Why not add this parameter to PCA’s methods and then pass it to svd_flip? This will affect a critical attribute, components_, which is simply equal to V. Otherwise, the only real way to use u_based_decision=False within PCA itself is to rewrite from the ground up with a lot of code copy/pasting involved.

[lesteve]

And by the way @bsolomon1124 thanks for the good quality issue! It is always nice when we can see that the OP has made an effort to research the issue and explain where he was coming from. Let’s say this does not always happen …

[lesteve]

I am going to try to sum up the discussion here:

• sign of the coefficients in a given eigenvector does not mean anything at all and for reproducibility of what the solver does we have u_based parameter in svd_flip.
• nonetheless you are trying to interpret the sign of the loadings in some fashion, and you are complaining that the result does not match your intuition
• in your particular example being able to pass u_base=False to PCA would match what you expectations

The thing is that you seem to be in a very particular case where all of the elements of your loading have the same sign. Imagine a financial instrument negatively correlated your “market-like” component, with a high absolute value, I would bet that neither u_based=True nor u_based=False would still match your expectation at all.

If you have a good reason to interpret the sign of the loadings in your particular use case I would suggest you to do some post-processing of pca.components_ and implement the rule you are thinking of. Maybe you can inherit from PCA and do not have to rewrite that much code, maybe it’s not that convenient, I am not sure.

I am going to close this one, @bsolomon1124 feel free to shout if you strongly disagree.