A way to specify build-dependencies
See original GitHub issuePipenv might need a way to specify built-dependencies and tooling packages that need to be installed in order for pipenv to do it’s job correctly. Even a pyporject.toml
is not sufficient, because you’d have to pass pip the option --no-use-pep517
in order to install build dependencies and then have pip build from setup.py
the old-fashioned way.
Example
In my company we use a wrapper around setuptools to simplify writing setup.py
scripts for our packaging. Let’s call this wrapper stwrapper
.
A typical setup.py
would look like this:
# setup.py for "package-that-uses-stwrapper-in-its-setup-py"
from stwrapper import setup
setup(
name='package-that-uses-stwrapper-in-its-setup-py',
# ... some more arguments
)
Where the function stwrapper.setup
parses arguments, does some extra work and then calls the real setuptools.setup
. Effectively, the stwrapper
package is sort of a built dependency for us.
Lets say we have the following Pipfile
:
[dev-packages]
stwrapper = "*"
package-that-uses-stwrapper-in-its-setup-py = { path = ".", editable = true, requires=["stwrapper"] }
and do a
pipenv lock --verbose
then, with the current version of Pipenv, we get:
Traceback:
...
pipenv.patched.notpip._internal.exceptions.InstallationError: Command "python setup.py egg_info" failed with error code 1 in '.'
Because stwarpper
is not installed at the time pipenv tries to parse the setup.py
of package-that-uses-stwrapper-in-its-setup-py
. That is even the case, when stwrapper
is installed in the system’s python installation. Pipenv wants to have it inside the new virtual environment it creates. It’s a Catch-22.
Our current workaround is to let pipenv create the virutal environment. Then do a pipenv run pip install stwrapper
inside it and then the pipenv lock
. That way it works.
Another, better workaround will be possible one https://github.com/pypa/pip/pull/6370 is merged into pip. Then the behavior of pip-option --no-use-pep517
will be to install requirements specified in a pyproject.toml
and proceed with pre-pep517 installation using setup.py
.
Possible Solutions
1.) Pipenv gets an equivalent to setup_requires
in setup files or requires=[...]
in pyproject.toml
’s [backend]
section that is global or we use the existing one that is currently only used for python_version
:
[requires]
python_version = "2.7"
stwrapper = "*"
[packages]
...
2.) Pipenv uses the system python’s installation of stwrapper
(I’m aware of the --site-packages
option but that may have unwanted side effects).
3.) Specify a per-package requires
, i.e.
[packages]
mypackage = { version="2.0.1", requires=['stwrapper', 'gitpython'] }
4.) Pass --no-use-pep517
to pip for certain packages (similar syntax to 3.) and use that feature in conjunction with pyproject.toml
for these packages
[packages]
mypackage = { version="2.0.1", pip_options=['--no-use-pep517'] }
5.) Specify an order/priority of package installation, so that the dependency is fullfilled when the dependent package is being installed/
Scope
I guess the problem applies to all projects that use non-standard-libaray packages in their setup.py
files and/or use setuptools entry points to modify its behavior. You can not put such dependencies in the Pipfile, because they are needed for successfully locking.
Related Issues
Issue Analytics
- State:
- Created 4 years ago
- Comments:16 (6 by maintainers)
I wonder if this can be fixed more simply by passing
--no-use-pep517
during the retry phase of installation.Okay, I just learned that the
build-system
solution, does not work in case of editable installs, unless pip is called with--no-use-pep517
.See PEP 517:
So the issue persists. It seems something like this, might be useful: