Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

shouldn't pipenv rather depend on the packages in vendor than providing them directly?

See original GitHub issue


I’m currently packaging pipenv for Debian and I wonder what’s the purpose of the vendor directory and its contents? I understand the purpose of patched but shouldn’t pipenv rather depend on those packages in vendor (e.g. via install_requires in setup py) than providing them?

From a packaging point of view this is also very hard as I’ll probably have to un-bundle the vendor packages one by one and make pipenv depend on the corresponding Debian packages.

Assuming the packages in vendor are unpatched, would you accept pull requests that try to un-bundle those packages in vendor?

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:14 (10 by maintainers)

github_iconTop GitHub Comments

ncoghlancommented, Apr 1, 2018

As @gsemet notes, the key problem for tools like pip/pipenv/setuptools/etc is that they’re part of bootstrapping the dependency management system, and we can’t necessarily assume that that is already working in an end users’ environment, just because they have a Python install.

Debian & Ubuntu, for example, ship without a working ensurepip and venv by default - users need admin privileges to get things working (see ).

Fedora ships a working ensurepip and venv, but they do so by relying on pip’s dependency bundling at the virtual environment level and then keeping the pip package itself up to date.

The Mac OS X system Python is infamously broken by default (in more ways than just not supporting pip), as Apple decided to head down the Swift path instead.

pip and setuptools have the extra problem of actually ending up in the run and/or build environments of the projects being developed, giving them an extra incentive to hide their dependencies, whereas pipenv is mainly avoiding polluting the user level site-packages directory. However, having a random upgrade in the user site-packages directory break pipenv would also be a major problem when it came to keeping pipenv managed environments up to date, so there’s a significant resilience concern there as well.

So the answer to @venthur’s original question that started the issue is “No, it shouldn’t, as doing so would make bootstrapping more difficult, and introduce a greater chance of an end user accidentally breaking their ability to update their pipenv managed environments in a way they can’t readily recover from”.

I’m going to close this particular issue on that basis - if folks wanted to open to open separate issues for switching to pip-style vendoring (with clear dependency declarations, and automated updates for vendored dependencies), and then configuring to monitor those dependencies for new releases, that would be a good thing.

More generally, I’d ask that folks avoid making the assumption that we aren’t already well aware of the risks and trade-offs between fully integrated systems (fewer security updates due to the use of shared components, but either requiring more coordinated integration testing before an update is released, or else increasing the risks of failure on end user systems due to untested combinations), and isolated application silos (multiple app updates needed for the same security update, but each app can be developed and tested independently without a complex distro-style coordination layer).

The key design assumption that pipenv makes that a lot of other developer tools don’t is that we expect a large proportion of our user base to be students, educators, and research scientists that are looking to write and run their first Python applications. We default them to running the latest version of everything, and require them to explicitly opt out and pin old versions if that’s what they want to do instead, but we mostly assume that they’re writing software for themselves.

Even for more experienced application developers, we mostly assume that they’re either working on deployment to hosted PaaS environments (so dependency updates are just a git push away), app store application development (so they have to bundle anything beyond the base platform APIs regardless), or else they’re using pipenv to manage the dependencies for their test suite and development environment, rather than for the deployed component.

This is going to be irritating for folks that prefer to treat their systems as carefully crafted interlocked webs of dependencies, and don’t want to delegate responsibility for any part of that system to anyone else. However, even the traditional Linux operating system developers have conceded that the “giant ball of mud” approach to operating system development breaks down once you reach the level of thousands or tens of thousands of intertwined components, and are switching to technologies like Kubernetes, Flatpak, and Snappy to help break that scaling barrier, while mitigating the harm that can be caused by outdated components in application bundles that haven’t updated appropriately.

venthurcommented, Apr 1, 2018

Well, then let’s open another issue requesting the pip-style vendoring, shall we?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Pipenv: promises a lot, delivers very little | Chris Warrick
Pipenv only concerns itself with managing dependencies. It isn't a packaging tool. If you want your thing up on PyPI, Pipenv won't help...
Read more >
pipenv Documentation - Read the Docs
Pipenv is a tool that aims to bring the best of all packaging worlds (bundler, composer, npm, cargo, yarn, etc.) to the. Python...
Read more >
How to resolve Python package dependencies with pipenv?
First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again. Alternatively, you can use $ pipenv...
Read more >
Should You Use Upper Bound Version Constraints?
In this discussion I would like to explain why always providing an upper limit causes far more harm than good even for true...
Read more >
pipenv Documentation
Do not touch Pipfile early and rely on it so that one can do ... led to direct invocations of virtualenv, rather than...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found