question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

multiple parallel environments

See original GitHub issue

Hi,

thanks for this wonderful tool!! It’s well crafted and extremely useful, but I feel that there is exactly one feature missing to make it just perfect for a workflow I frequently encounter while developing python packages:

One often needs to switch python versions in order to test behaviour on different python versions. Technically, pipenv --python=XXX can be used to switch between different python versions. However, this replaces the current environment and is therefore inconveniently slow in practice.

I would really like to be able to install multiple environments in parallel for the same project and select which one to use on the pipenv command line. This can speed up the time required to switch python versions for the same project significantly and therefore we would have a powerful (IMO alround better) replacement for virtualenvwrapper and the like.

This could work by e.g. providing an environment name on the command line:

# setup environments
pipenv --name 35 --python 3.5 install
pipenv --name 36 --python 3.6 install

# run commands
pipenv --name 35 run python

One issue could be that of different lockfiles in different python versions…

If you would be fine with such an option and think that it is possible to implement for s.o. unacquainted with the code, I’m of course willing to help on the implementation.

This issue is related to #368 and #137 who I think were trying to ask the same question, but I’m unsure whether you rejected the idea altogether or were happy with an intermediate solution at the time.

Best, Thomas

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Reactions:27
  • Comments:20 (8 by maintainers)

github_iconTop GitHub Comments

14reactions
techalchemycommented, Nov 16, 2017

Thanks for the suggestion but deterministic virtualenv locations is an inherent part of what pipenv is, so I don’t see us implementing this. I’m gonna close this out for now!

13reactions
jtratnercommented, Mar 5, 2018

To be super clear, you can still get your own custom environments set up just by sourcing virtualenvs.

virtualenv 35 --python=python3.5
virtualenv 36 --python=python3.6
source 35/bin/activate && pipenv install
source 36/bin/activate && pipenv install
source 35/bin/activate && pipenv run <whatever>

a tiny bit of additional visual clutter to the commands but is pretty straightforward

Read more comments on GitHub >

github_iconTop Results From Across the Web

Parallel Environments (PE) | User Guides | High Performance ...
Parallel Environments (PE). Below you will find information for each of the parallel environments that can used when submitting jobs to HPCC clusters....
Read more >
Do we need multiple parallel environments to train in batches ...
We don't need multiple environments. On-policy algorithms require that new training samples are collected with the newest policy, ...
Read more >
Is it mandatory to have several parallel environments ... - Reddit
Hello, I'm wondering whether having several environments is mandatory to train a successful policy when using PPO ? Couldn't one generate as ...
Read more >
Efficient Multiple Gym Environments - Squadrick
Using multiprocessing for parallel gym environments was a definite improvement, however it's useful only for a single PC with multiple cores.
Read more >
Parallel Environments : TechWeb - Boston University
In MATLAB, there are two complementary ways to perform parallel computations: ... puts you in a Single Program Multiple Data, or spmd, parallel...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found