Add ability to have custom package sets. pipenv --install --pack=dev --pack=test

Feature: Custom Package Sets

It would be nice to be able to create custom package sets. For instance, pipenv already has support for two package sets “default” and “dev”. However, I can see use cases where it would be nice to have multiple package sets.

$ pipenv install --pack=test    # Custom "test" package set; [test-packages]
$ pipenv install --pack=dev     # Existing "dev" package set; [dev-packages]
$ pipenv install --pack=default # Existing "default" package set; [packages]
$ pipenv install --pack=        # "; [packages]

Description of Problem

For me, I have a use case where I want to test the changes made in a another package, DependencyProject, to see the effects it in MainProject before I release it (DependencyProject).

In MainProject, I want everyone to have DependencyProject.

$ pipenv install dependency-project

For --dev work, I want contributors to install testing tools.

$ pipenv install --dev pylint pytest tox

But, normal contributors (nor the CI release process) shouldn’t have pre-alpha releases of DependencyProject installed. This should be reserved experienced contributors (or, “coordinated” contributors), so they should use a different package set; not “default”, not “dev”.

$ pipenv install --pack=dep-test -e git+https://github.com/fictional-org/dependency-project#egg=dependency-project

Current Workaround

To be honest, I haven’t attempted a workaround yet. I’ve used Pipenv sparingly over the past year, but over the past month, I have made it part of my standard process. And, I’m seeing this scenario (as described above) coming. So, at the moment, it is a hypothetical problem, but I do have a real workaround.

Use pipenv run pip install ... when I need to do alpha testing.

$ pipenv run pip install -e git+https://github.com/fictional-org/dependency-project#egg=dependency-project

Then run pipenv install --dev to go back to the standard development environment.

$ pipenv install --dev

I’ve tested this scenario by installing requests and requests==2.9.1. (NOTE: The that the current version of Requests ==2._19_.1 is very similar to the older version ==2._9_.1.)

$ pipenv install requests

$ pipenv run pip freeze
requests=2.19.1

$ pipenv run pip install requests==2.9.1

$ pipenv run pip freeze
requests=2.9.1

$ pipenv install

$ pipenv run pip freeze
requests=2.19.1

Existing Problems

In my hypothetical problem, I am trying to add two different version of the same package into the Pipfile. Because Pipenv does have two package set options (default and dev) I can attempt the scenario using them. As it turns out, it isn’t currently possible to have differing versions between the two package sets. Again, I’ll demonstrate with Requests.

Explicit Default, Relaxed Dev

$ pipenv install requests==2.9.1
# [packages]
# requests = "==2.9.1"

$ pipenv install --dev requsets
# [packages]
# requests = "==2.9.1"
# [dev-packages]
# requests = "*"

$ pipenv run pip freeze
requests==2.9.1

Explicit Dev, Relaxed Default

$ pipenv install --dev requests==2.9.1
# [dev-packages]
# requests = "==2.9.1"

$ pipenv install requests
# [packages]
# requests = "*"
# [dev-packages]
# requests = "==2.9.1"

$ pipenv run pip freeze
requests==2.19.1

$ pipenv install --dev
$ pipenv run pip freeze
requests==2.9.1

NOTE: This problem is, likely, a combination problem, so I should test scenarios for “Relaxed Dev, Explicit Default” and “Relaxed Default, Explicit Dev” as well (And, there’s likely more cases that I haven’t conceptualized yet.) But, I think the two scenarios show an inconsistency.

So, just to get to my desired feature, this problem will have to be solved first. (I have not researched the Issues about this potential bug, so I haven’t created a new one. I can, and will, upon request. ~“When Duplicate Dependencies exists, the dependency defined in the Default package set always wins.”)

Conclusion

Despite my specific use case (and the current limitations that prevent it), there’s potential for this feature.

I suppose, conceptually, my request is most similar to extra_requires as found in setuptools. Most notably projects like IPython (setup.py) depend on this functionality to provide different levels of dependencies to their users; ex: pip install ipython['all'].

A project like Tox allows for multiple “package sets”, so conceptually the expectation exists. Although, I’ll concede that Tox implements this scenario by having multiple virtualenvs. That’s a potential solution to the problem, but by separating sets into virtualenvs, that creates a combination problem between each set. (Does pipenv install --pack=test imply installing --pack=dev too?)

… I’m doing a terrible job of concluding this request because now I have more questions pertaining to resolution scenarios between multiple sets. So, in order to stop my digressions, I’ll stop here to say, …

I think this would be a great feature. I can’t possibly think up of all the cases where it would be amazing. And, I can’t conceive of all the cases where it becomes hard to implement, so I’m just putting it out there. Let’s discus.

Due Dilligence

I haven’t been able to find any mention of this capability in the Basic Usage, Advanced Usage, nor in the Issues, (is:issue label:enhancement dev packages), so I created this request.

[marco-silva0000]

is there any progress on this? is there any repo where a draft implementation is being worked on?

[uranusjr]

The number does not need to be the lowest possible, you can choose whatever you think is appropriate. The maintainers may request you to change the number before it is merged.