RuntimeError: Python version >= 3.5 required.

Environment

• pip version: 19.1.1
• Python version: 2.7.12
• OS: Ubuntu 16.04

Description Fails to install packages from source where install_requires includes versions have incompatible python_requires.

Expected behavior Source package should be installed with compatible versions of packages listed in install_requires from setup.py. Eg: Installing pandas in Python 2.7 should result in numpy version 1.16.4 being installed (the last 2.7 compatible release before 1.17.0rc1).

How to Reproduce

1. Run pip install --no-binary=pandas pandas in a Python 2.7 environment.
2. An error occurs.

Output

$ mkvirtualenv -ppython2.7 pip27
Running virtualenv with interpreter /usr/bin/python2.7
New python executable in /home/tclugg/.virtualenvs/pip27/bin/python2.7
Also creating executable in /home/tclugg/.virtualenvs/pip27/bin/python
Installing setuptools, pip, wheel...done.
virtualenvwrapper.user_scripts creating /home/tclugg/.virtualenvs/pip27/bin/predeactivate
virtualenvwrapper.user_scripts creating /home/tclugg/.virtualenvs/pip27/bin/postdeactivate
virtualenvwrapper.user_scripts creating /home/tclugg/.virtualenvs/pip27/bin/preactivate
virtualenvwrapper.user_scripts creating /home/tclugg/.virtualenvs/pip27/bin/postactivate
virtualenvwrapper.user_scripts creating /home/tclugg/.virtualenvs/pip27/bin/get_env_details
Python 2.7.12
(pip27) $ pip install -U pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Requirement already up-to-date: pip in /home/tclugg/.virtualenvs/pip27/lib/python2.7/site-packages (19.1.1)
(pip27) $ pip install --no-binary=pandas pandas
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Collecting pandas
  Using cached https://files.pythonhosted.org/packages/b2/4c/b6f966ac91c5670ba4ef0b0b5613b5379e3c7abdfad4e7b89a87d73bae13/pandas-0.24.2.tar.gz
    ERROR: Complete output from command python setup.py egg_info:
    ERROR: Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-rC8dFo/pandas/setup.py", line 746, in <module>
        **setuptools_kwargs)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/__init__.py", line 144, in setup
        _install_setup_requires(attrs)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/__init__.py", line 139, in _install_setup_requires
        dist.fetch_build_eggs(dist.setup_requires)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/dist.py", line 717, in fetch_build_eggs
        replace_conflicting=True,
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 782, in resolve
        replace_conflicting=replace_conflicting
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1065, in best_match
        return self.obtain(req, installer)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1077, in obtain
        return installer(requirement)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/dist.py", line 784, in fetch_build_egg
        return cmd.easy_install(req)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 679, in easy_install
        return self.install_item(spec, dist.location, tmpdir, deps)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 705, in install_item
        dists = self.install_eggs(spec, download, tmpdir)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 890, in install_eggs
        return self.build_and_install(setup_script, setup_base)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1158, in build_and_install
        self.run_setup(setup_script, setup_base, args)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1144, in run_setup
        run_setup(setup_script, args)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 253, in run_setup
        raise
      File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
        self.gen.throw(type, value, traceback)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
        yield
      File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
        self.gen.throw(type, value, traceback)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 166, in save_modules
        saved_exc.resume()
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 141, in resume
        six.reraise(type, exc, self._tb)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 154, in save_modules
        yield saved
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
        yield
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 250, in run_setup
        _execfile(setup_script, ns)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 45, in _execfile
        exec(code, globals, locals)
      File "/tmp/easy_install-sfO_WV/numpy-1.17.0rc1/setup.py", line 31, in <module>
    
    RuntimeError: Python version >= 3.5 required.
    ----------------------------------------
ERROR: Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-rC8dFo/pandas/

Paste the output of the steps above, including the commands themselves and
pip's output/traceback etc.

[pradyunsg]

Uhhhhh. I’ve also hidden your comment since I don’t want to be reading that again.

I understand that this can get a little frustrating. Let me clarify what @xavfernandez and I have been saying, since I don’t think we’ve been super verbose/clear with our reasoning here. Apologies for that.

---

This error is originating from setuptools’s logic for handling setup_requires. This was a mechanism that existed to help ensure dependencies for running setup.py are available. If you have a keen eye, you’ll notice that this means running setup.py to get information for what’s needed to run setup.py. That’s why setup_requires is not a good thing. Further, setuptools does not expose the information about setup_requires in any way for pip to satisfy that requirement before executing the setup.py script.

pip developers, setuptools developers and a lot more people, worked together on PEP 518 to help with exactly this situation: specifying build dependencies. That’s why one actionable item here is that pandas adopt the use of PEP 518 (i.e. use pyproject.toml). I understand that there were growing pains to adopting this new standard. None the less, we’re almost across the bridge on that; and @TomAugspurger has stated that pandas is open to doing so. 😃

---

I feel that pip should do one of the following:

I appreciate your suggestions but I think neither of them are actionable suggestions for pip.

• Stop delegating to external tools (ie: the buck stops here).

pip does not have any logic for converting a source distribution to an installable artifact. It depends on external “build backends” for doing that. So, well, this is out of scope for this tool. 😃

• Present a big warning before and/or after delegating to external tools, so that package developers (eg: the pandas team) are aware of issues with their packages (ie: pass the buck).

That’s status quo and what we do already.

The fact that pip depends on build backends is known to packager since they have to use the backend and tell it how to package things. Further, pip already states when it invokes a build backend. From your output in the first post:

ERROR: Complete output from command python setup.py egg_info: ERROR: Command “python setup.py egg_info” failed with error code 1 in /tmp/pip-install-rC8dFo/pandas/

To reiterate, pip’s delegating to and printing the output from pandas’s setup.py. Because of that, pip can’t really do much here. The most I can think that pip can do is, change the error message to say “hey, talk to the maintainers of pandas about this failure” but that’s not exactly a thing we want to do without considering the implications of that.

---

Further, both pandas and setuptools are in a position to improve the output for this scenario:

• pandas can either handle this case explicitly in their setup.py and print a friendlier message; or adopt pyproject.toml so that pip handles this for them.
  • If, for some reason, the new standard does not fit their use case, they can poke us at https://discuss.python.org/c/packaging and we can figure out the details there.
• setuptools can improve error messaging when a setup_requires; though I’m not sure how setuptools wants to handle that option, in a world where pyproject.toml is a thing.

---

I’m going to step back from this issue now. Don’t @ me. 😃

[TomAugspurger]

the final verdict was that they didn’t feel using pyproject.toml was necessary.

Just to clarify, pandas wants to adopt pyproject.toml, but it’s not a blocker for our next release.