Add Hypothesis Testing

It may be beneficial to add hypothesis testing to some of our tests. Hypothesis testing allows for us to verify mathematical properties such as commutativity:

from hypothesis import given
import hypothesis.strategies as st

@given(st.integers(), st.integers())
def test_ints_are_commutative(x, y):
    assert x + y == y + x

This allows the CI to look for edge cases for us. This would be useful testing metrics. https://hypothesis.readthedocs.io/en/latest/

[thomasjpfan]

@Zac-HD Thank you for the link to the tutorial and your offer to help out! There are certainty a few interesting places for us to use property-based testing.

[Zac-HD]

👋 hi all, I’m not much of an ML person but I am particularly interested in open source and in testing scientific code (as my tutorial at SciPy might indicate). So if getting a Hypothesis core dev to review or discuss some tests would ever be useful, ping me and let me know how I can help!

Along with - or before - testing all the mathematical properties you might think of, I highly recommend getting Hypothesis to generate any valid data (i.e. including unusual formats) and just calling your code. No assertions needed, in my experience you’ll find a bunch of crashes, quickly improve your input validation, and probably fix some bugs too. Once you have that experience, testing ‘metamorphic properties’ is ludicrously effective…