Testing exception messages

Re discussion in #1052, what is the general consensus on testing for exception messages? The way I see it, there are a few ways we can handle this (in order of work required):

1. Ignore exception messages
2. Test only for exception messages that are explicitly included in the canonical data for the exercise
3. When exception messages are not explicitly stated in the canonical data, require that the exercise contributor invent messages for all expected exceptions

Additionally, if we are to test for exception messages at all, we need to have a conventional method of doing so in both Python2 and Python3. assertRaisesRegexp is replaced by assertRaisesRegex in Python3, so as mentioned in #1052 there are a few ways of handling this difference.

[N-Parsons]

If we do implement checks for the presence of error messages, I think we should definitely add something about it to the README insert so that it’s clear that we’re expecting them (and to encourage people to make them meaningful).

Exception types are controlled by the test-suite, so there’s not really much problem on the learner side, but we should try to have some consistency between the test-suites. We could create something along the lines of the Choosing Exceptions to Raise doc pandas has to give everyone something to refer to.

[cmccandless]

For future record, here is the proper way to handle exception testing in this repository:

class TestClass(unittest.TestCase):
	def test_raises_exception(self):
        with self.assertRaisesWithMessage(ValueError):
            method_that_raises_ValueError()

    # Utility functions
    def setUp(self):
        try:
            self.assertRaisesRegex
        except AttributeError:
            self.assertRaisesRegex = self.assertRaisesRegexp

    def assertRaisesWithMessage(self, exception):
        return self.assertRaisesRegex(exception, r".+")

Edit: if Python 2.7 is no longer supported, the above snippet may be simplified to the following:

class TestClass(unittest.TestCase):
    def test_raises_exception(self):
        with self.assertRaisesWithMessage(ValueError):
            method_that_raises_ValueError()

    # Utility functions
    def assertRaisesWithMessage(self, exception):
        return self.assertRaisesRegex(exception, r".+")