Proposal: Remove `upper` argument from `eigh` and `eigvalsh`

Some functions that accept a symmetric matrix (eigh, eighvals) have an upper=False kwarg. According to the API: “If True, use the upper-triangular part to compute the eigenvalues and eigenvectors. If False, use the lower-triangular part to compute the eigenvalues and eigenvectors.”.

There are a number of reasons why we might want to remove this argument from the API.

1. This kwarg goes against Point 4 of the design principles, as it biases the implementations towards “always assuming that the input is well-formed” (symmetric in this case) when it could very much not be the case. This prevents the implementations from throwing meaningful errors when the input is not symmetric.
2. Lack of orthogonality: The API does not offer a parameter of this kind for other functions that also accept a symmetric matrix as inputs such as cholesky. In fact, cholesky has a boolean parameter also named upper which does something completely different to eigh’s upper.
3. Makes the semantics of autograd counterintuitive: This is not that relevant for this API as this API does not specify differentiation behaviour, but this API will potentially be followed by a number of frameworks that implement differentiation. It is not clear whether one should assume that the input is symmetric, or that the upper (resp. lower) triangular part of the input matrix input matrix represents a symmetric matrix via the transformation A.triu() + A.triu(1).T . This is problematic when computing the gradient, as it is not clear whether the gradient should be symmetric or upper triangular. PyTorch makes the gradient symmetric to be mathematically consistent with the semantics of the operation, rather than with the optimisation that this kwarg suggests, which is semantically wrong but more intuitive. We have not received any issue about this, which hints to the fact that almost no one uses this optimisation to pack two symmetric matrices into one unconstrained matrix (and an extra vector).
4. Lack of Orthogonality 2. It provides an optimisation that can be accomplished via the current API. If a user wants to pack two symmetric matrices with zero diagonal as the upper and lower-triangular part of a square matrix they could write L1, Q1 = linalg.eigh(A.triu(1) + A.triu(1).T); L2, Q2 = linalg.eigh(A.tril(-1) + A.tril(-1).T). Doing this and implementing symmetric gradients (which is what is mathematically correct for eigh) would give the correct semantics with respect to the upper (resp. lower) part of A.
5. Related to Point 3. eigh is defined in the API as “Returns the eigenvalues and eigenvectors of a symmetric matrix (or a stack of symmetric matrices) x.”, but this kwarg suggests that x should be a stack of upper (resp. lower) triangular matrices that encode symmetric matrices.

In summary, this is a feature that seemed to have made its way from LAPACK into numpy, and a number of other frameworks have implemented it for numpy compatibility. This was certainly the case of PyTorch. It could be of interest to deprecate it, as it is too low level for a generic API, and its behaviour can be implemented in a couple of lines, as described above.

cc @rgommers @mruberry

[lezcano]

This may not be the case if libraries implement LazyTensors.

That being said, there’s also the point that, in PyTorch, we have been returning symmetric gradients for quite a few versions already, and we have not had any issue raised about this. This may indicate that not that many people use this feature.

[kgryte]

I’ve opened gh-237 to discuss adding tril and triu to the specification.