TST: Converting to Pytest Idiom

Our testing framework is a literal hodgepodge of different 3rd-party framework standards (i.e. nosetest, pytest, unittest) that needs to be cleaned up.

A couple of points are clear:

• Use assert instead of self.assert*...
• When possible, replace for loops with pytest.mark.parametrize
• All new test classes should inherit from object instead of tm.TestCase

However, we also have this massive testing module helper testing.py that is populated with tons of useful comparison methods as well as wrappers around basic assert methods that are used inconsistently throughout the code-base. Thus, the question is whether or not we want to keep using those OR remove them entirely and replace them with basic assert statements.

The advantage with using tm.assert statements is that they come directly with error messages instead of an empty AssertionError, which is what I don’t like about raw assert statements that are considered more idiomatic with pytest.

Thus, it’s a question of greater clarity (i.e. our error messages will be more useful by default when we get test failures) vs. idiomatic (raw assert with no wrapping is the idiomatic way to go but risks presenting less helpful failure messages).

<strike>Another point I wanted to bring up is with regards to the introduction of decorators in #15952 to capture stdout and stderr. pytest has fixtures to capture those, but incorporating it into the code-base is very difficult due to our heavy reliance still on unittest.TestCase, which causes incorporation to fail. The question is: do we eventually want to use the pytest fixture or continue using the decorator?</strike>

Thoughts?

---

From the discussion that follows, here are the functions that should be removed and replaced:

• self (tm).assertRaises --> pytest.raises
• (NO ACTION) self (tm).assertRaisesRegexp -->

with pytest.raises(cls) as exc_info:
   ...
exc_info.match(regex)

Per discussion in #16089 (comment), we are leaving as is.

• self.assert* (examples follow)
  • self.assertIs
  • self.assertEqual
  • …and many others
• self.assert_* (examples follow)
  • self.assert_numpy_array_equal
  • self.assert_series_equal
  • self.assert_frame_equal
  • self.assert_index_equal
  • …and many others
• tm.assert_equal
• tm.assertIs(Not)
• tm.assert(Not)In
• tm.assertIs(Not)None
• tm.assert(Not)IsInstance

[jorisvandenbossche]

I don’t think we need to change any of the asserts in util/testing.py. They’re already doing the right thing.

I think @gfyoung meant the assert_equal, assertIsInstance, … wrappers we have in pandas.util.testing. And for those, yes, I think we should clean that up and remove those.

[jreback]

yes the only things would change are

• assert_equal (this is generally for scalar / lists /dicts)
• self.assertEqual
• self.assertTrue/False
• assertDictEqual
• assertIs
• assertIsInstance

and use pytest.raises