question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Add a way to avoid installing dependencies provided by other means

See original GitHub issue

What’s the problem this feature will solve?

When packaging applications in containers for different architectures or non-glibc-distros, there is often no binary pip package available. Often distros like Alpine Linux provide packages matching the required version range. Because there is currently no way to tell pip to exclude this packages, you have to install all the build tools and build all the dependencies manually, which leads to excessive build times and uses way more resources.

Describe the solution you’d like

I would like to have an option --ignore pkgA,pkgB, or --ignores ignored_packages.txt. Packages listed there will be completely ingnored by pip and treated as installed. Optionally, it is required to add a --yes-please-shoot-me-in-the-foot-i-know-what-i-am-doing flag to enable this behaviour to scare away novice users.

Alternative Solutions

Provide prebuild packages for all architectures and musl.

Additional context

I know this is a feature useful to expert users and potentially harmful to novice users.

Code of Conduct

Issue Analytics

  • State:closed
  • Created 10 months ago
  • Comments:13 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
pradyunsgcommented, Dec 16, 2022

This isn’t a problem with just trying to install numpy – this is a problem with trying to generate an installable artifact for Shapely; due to being unable to install a package required to generate that artifact.

The proposed solution does not solve the user’s problem in this case because you won’t be able to build Shapely into an installable artifact. You either need to (a) install the dependencies needed to build numpy or (b) install numpy and all the other build dependencies of Shapely and generate a wheel for it and then ask pip to install that wheel.

TBH, doing (a) will be easier than (b). At the end of the day, this is a “how do I build + install X” problem and I suggest you seek assistance in a support forum for how to install X in an alpine container.

Even if this wasn’t a build-time issue, I don’t think having pip blanket ignore parts of the dependency graph is a tractable approach since we can’t know the dependencies of X and this can’t account for them during installation; if we don’t have metadata about X to look at.

1reaction
pfmoorecommented, Nov 18, 2022

From the way you describe the use case, it sounds like Alpine are providing a distribution package foo-python that installs a copy of the foo Python package.

If pip isn’t recognising that foo is installed, that implies that the Alpine package isn’t including the necessary metadata to describe the package in its foo-python distribution. The required metadata is described here. If the metadata is correctly included, pip should see that foo is already present, and not try to build it.

If that’s not the issue, can you provide a more detailed description of a case where the proposed flag would be needed?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Prevent pip from installing some dependencies - Stack Overflow
One way is to list all the modules and use pip --no-deps but that means a constantly keeping track of the dependencies manually,...
Read more >
Dependency Management with Pip, Python's Package Manager
One way to avoid dependency conflicts is to use an alternative Python package manager, like conda, poetry or ActiveState's State Tool.
Read more >
How to Install and Correct Dependencies Issues in Ubuntu
Enable all repositories · Update the software · Upgrade the software · Clean the package dependencies · Clean cached packages · Remove "on-hold"...
Read more >
Dependency Resolution - pip documentation v22.3.1
The process of determining which version of a dependency to install is known as dependency resolution. This behaviour can be disabled by passing...
Read more >
Maven – Optional Dependencies and Dependency Exclusions
Optional dependencies save space and memory. They prevent problematic jars that violate a license agreement or cause classpath issues from being bundled into...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found