Remove test command and tests_require

In this comment, @graingert proposes that we may be able to completely remove support for tests_require instead of transitioning that tooling from easy_install to pip install.

While he didn’t directly propose removal, the effort would only benefit that ticket if the install functionality were completely removed, so let’s explore what that will entail.

While I agree that tox is an excellent, powerful, robust solution, it’s more heavy than tests_require, requiring that the user have tox installed in advance. As a more thorough solution, it also is subject to bugs and constraints that a simpler test runner is not. There are several advantages to setup.py test and tests_require over tox:

• Doesn’t have any prerequisites (except the implicit one on Setuptools, which is generally assumed).
• As a result of not having any prerequisites, invocation is often a simple invocation (setup.py test) instead of multiple steps (e.g. pip install --user tox; python -m tox).
• setup.py test allows for invocation under a number of different Python versions naturally (i.e. python3.3 setup.py test or py -3.3 setup.py test) whereas tox offers “run under the python in which tox is installed” or “run for explicitly-declared python versions”.
• As setup.py test doesn’t use virtualenv, it’s not subject to virtualenv bugs or other constraints imposed by virtualenv (such as version 14 dropping support for Python 3.2.).
• As setup.py test doesn’t rely on pip, it’s not subject to the bugs of pip (such as issues with --editable installs or namespace packages) or other constraints imposed by pip (such as dropping support for Python 3.2).

I consider these advantages small and easy enough to overcome, especially now that many of these issues have been resolved in setuptools, pip, and virtualenv. If we can get to a place that tox can broadly supplant the uses cases of setup.py test and pytest-runner (and thus tests_require) in practice, then yes, deprecating and removing it would be in order. Given the amount of activity and bugs I see around these tools, I’d asses they’re still in active use.

Before flagging this functionality as deprecated, I’d like to survey the community about the possibility to see if there are use cases that would prove difficult to support with tox.

@graingert, would you be interested in being the champion for this effort (removing tests_require), starting with the outreach on distutils-sig and then implementing the deprecation/removal changes?

[mcepl]

Eliminate where or how? For your own projects or for projects you are packaging? tox is similar to make test in that each project can specify its own test commands. python -munittest discover will not work for a large number of packages, which intend for you to use pytest, and presumably also intend to have the dependencies that tox installs.

Well, the problem with whole virtual environments tox promotes is that it is completely useless in the Linux packaging world: we have all builds working in their own chrooted completely isolated environments so whatever issues virtenv is trying to solve are completely irrelevant for us. Also, we have different mechanisms (full blown isolated builds in separate virtual machines or something of that calibre) for testing with different versions of Python or something.

So, we usually end up with digging into tox.ini and ripping out only unittest (or pytest, it doesn’t matter) line and using only that.

[RonnyPfannschmidt]

@jaraco instead of test_requires, i’d like to see something that allows pip install -e .[test]