question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Incorrect shape handling in `scipy.stat.multivariate_t.rvs` method

See original GitHub issue

The rvs method of multivariate_t works only when size parameter is strictly 1 dimensional. This is not the case with multivariate_normal.

Reproducing code example:

>>> import sys, scipy, numpy; print(scipy.__version__, numpy.__version__, sys.version_info)
1.6.3 1.20.3 sys.version_info(major=3, minor=7, micro=7, releaselevel='final', serial=0)
>>> import numpy as np
>>> from scipy.stats import multivariate_normal, multivariate_t
>>> mean, cov = np.zeros(2), np.eye(2)
>>> multivariate_normal.rvs(mean=mean, cov=cov, size=None).shape
(2,)
>>> multivariate_t.rvs(loc=mean, shape=cov, size=None)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/sayam/miniconda3/envs/pymc_dev/lib/python3.7/site-packages/scipy/stats/_multivariate.py", line 4148, in rvs
    samples = loc + z / np.sqrt(x)[:, None]
IndexError: invalid index to scalar variable.

>>> multivariate_normal.rvs(mean=mean, cov=cov, size=(10, 3)).shape
(10, 3, 2)
>>> multivariate_t.rvs(loc=mean, shape=cov, size=(10, 3))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/sayam/miniconda3/envs/pymc_dev/lib/python3.7/site-packages/scipy/stats/_multivariate.py", line 4148, in rvs
    samples = loc + z / np.sqrt(x)[:, None]
ValueError: operands could not be broadcast together with shapes (10,3,2) (10,1,3) 

>>> multivariate_t.rvs(loc=mean, shape=cov, size=(3))
array([[ 1.54502738, -0.23866557],
       [-0.79040924, -1.72035808],
       [-0.28060545, -0.69889876]])

I guess the errors arise from this line - https://github.com/scipy/scipy/blob/f856e3e852a59120d44133a0ce550cceb19f9244/scipy/stats/_multivariate.py#L4084

This issue popped up while adding MvStudentT to PyMC3 https://github.com/pymc-devs/pymc3/pull/4731. Its random sampling method makes use of scipy.stats.multivariate_t.

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:8 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
Sayam753commented, Jun 6, 2021

Unfortunately, I am on MacOS and was following this tutorial.

  • In step 1, I installed miniconda instead of Anaconda (trying to be minimalist here)
  • In step 11, this command conda install mkl openblas cython numpy pytest pybind11 pythran=0.9.11 -c conda-forge helped me to follow next steps.

Sorry, I was not aware you were working on this.

My bad, I should have mentioned this earlier. Thanks for the helping hands anyways.

1reaction
tirthasheshpatelcommented, Jun 6, 2021

I was also thinking about the use of ellipsis to fix this issue but struggled my way through setting up scipy dev environment locally. FInally, I got it working.

Happy to hear that you got it working! You can refer to https://docs.scipy.org/doc/scipy/reference/building/linux.html#debian-ubuntu anytime if you are on Ubuntu/Debian. If you are on windows, this wiki page might help set it up easily. Sorry, I was not aware you were working on this.

Now, I’ll try to understand the shape handling in scipy in near future crossed_fingers

Sure, @Sayam753 😃! It would be very helpful if you could resolve some of the shape bugs I mentioned above. If you face any problems, feel free to ping me.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Various distribution functions disobey the size parameter when it ...
stats multivariate distributions seem to be vectorized w.r.t. shape parameters, I don't think we need to choose between the two rules yet. With...
Read more >
scipy.stats.multivariate_normal — SciPy v1.9.3 Manual
A multivariate normal random variable. The mean keyword specifies the mean. The cov keyword specifies the covariance matrix. Parameters. meanarray_like, default ...
Read more >
SciPy 1.9.0 Release Notes — SciPy v1.9.3 Manual
A new global optimizer, scipy.optimize.direct (DIviding RECTangles algorithm) was added. For problems with inexpensive function evaluations, like the ones ...
Read more >
scipy.stats.uniform — SciPy v1.9.3 Manual
A uniform continuous random variable. In the standard form, the distribution is uniform on [0, 1] . Using the parameters loc and scale...
Read more >
scipy.stats.ttest_ind — SciPy v1.9.3 Manual
Calculate the T-test for the means of two independent samples of scores. This is a test for the null hypothesis that 2 independent...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found