pipenv not compatible with virtualenv and miniconda

I have both virtualenv and miniconda installed.

When I run pipenv install it generates a Pipefile but returns error:

Creating a Pipfile for this project...
Creating a virtualenv for this project...
⠋Using base prefix '/Users/<my_username>/miniconda3'
New python executable in /Users/<my_username>/.virtualenvs/AIND-Pacman-pIsSzxmg/bin/python
dyld: Library not loaded: @rpath/libpython3.6m.dylib
  Referenced from: /Users/<my_username>/.virtualenvs/AIND-Pacman-pIsSzxmg/bin/python
  Reason: image not found
ERROR: The executable /Users/<my_username>/.virtualenvs/AIND-Pacman-pIsSzxmg/bin/python is not functioning
ERROR: It thinks sys.prefix is '/Users/<my_username>/.virtualenvs' (should be '/Users/<my_username>/.virtualenvs/AIND-Pacman-pIsSzxmg')
ERROR: virtualenv is not compatible with this system or executable

Virtualenv location: 
No package provided, installing all dependencies.
Pipfile found at /Users/<my_username>/code/aind-projects/AIND-Pacman/Pipfile. Considering this to be the project home.
Creating a virtualenv for this project...
⠙Using base prefix '/Users/<my_username>/miniconda3'
New python executable in /Users/<my_username>/.virtualenvs/AIND-Pacman-pIsSzxmg/bin/python
dyld: Library not loaded: @rpath/libpython3.6m.dylib
  Referenced from: /Users/<my_username>/.virtualenvs/AIND-Pacman-pIsSzxmg/bin/python
  Reason: image not found
ERROR: The executable /Users/<my_username>/.virtualenvs/AIND-Pacman-pIsSzxmg/bin/python is not functioning
ERROR: It thinks sys.prefix is '/Users/<my_username>/.virtualenvs' (should be '/Users/<my_username>/.virtualenvs/AIND-Pacman-pIsSzxmg')
ERROR: virtualenv is not compatible with this system or executable

Virtualenv location: 
Pipfile.lock not found, creating...
Traceback (most recent call last):
  File "/Users/<my_username>/miniconda3/bin/pipenv", line 11, in <module>
    sys.exit(cli())
  File "/Users/<my_username>/miniconda3/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/<my_username>/miniconda3/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/<my_username>/miniconda3/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/<my_username>/miniconda3/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/<my_username>/miniconda3/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/<my_username>/miniconda3/lib/python3.6/site-packages/pipenv/cli.py", line 755, in install
    do_init(dev=dev, allow_global=system, ignore_hashes=ignore_hashes)
  File "/Users/<my_username>/miniconda3/lib/python3.6/site-packages/pipenv/cli.py", line 560, in do_init
    do_lock(no_hashes=no_hashes)
  File "/Users/<my_username>/miniconda3/lib/python3.6/site-packages/pipenv/cli.py", line 409, in do_lock
    do_purge(downloads=True, bare=True)
  File "/Users/<my_username>/miniconda3/lib/python3.6/site-packages/pipenv/cli.py", line 506, in do_purge
    shutil.rmtree(project.download_location)
  File "/Users/<my_username>/miniconda3/lib/python3.6/site-packages/pipenv/project.py", line 92, in download_location
    mkdir_p(self._download_location)
  File "/Users/<my_username>/miniconda3/lib/python3.6/site-packages/pipenv/utils.py", line 149, in mkdir_p
    os.mkdir(newdir)
PermissionError: [Errno 13] Permission denied: '/downloads'

And when when I instead run sudo pipenv install it returns:

Creating a Pipfile for this project...
Creating a virtualenv for this project...
⠋Traceback (most recent call last):
  File "/Users/<my_username>/miniconda3/bin/pew", line 11, in <module>
    sys.exit(pew())
  File "/Users/<my_username>/miniconda3/lib/python3.6/site-packages/pew/pew.py", line 737, in pew
    return command(sys.argv[2:])
  File "/Users/<my_username>/miniconda3/lib/python3.6/site-packages/pew/pew.py", line 253, in new_cmd
    args.requirements, rest)
  File "/Users/<my_username>/miniconda3/lib/python3.6/site-packages/pew/pew.py", line 215, in mkvirtualenv
    check_call(["virtualenv", envname] + rest, cwd=str(workon_home))
  File "/Users/<my_username>/miniconda3/lib/python3.6/subprocess.py", line 286, in check_call
    retcode = call(*popenargs, **kwargs)
  File "/Users/<my_username>/miniconda3/lib/python3.6/subprocess.py", line 267, in call
    with Popen(*popenargs, **kwargs) as p:
  File "/Users/<my_username>/miniconda3/lib/python3.6/subprocess.py", line 707, in __init__
    restore_signals, start_new_session)
  File "/Users/<my_username>/miniconda3/lib/python3.6/subprocess.py", line 1326, in _execute_child
    raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/<my_username>/.local/share/virtualenvs'

Virtualenv location: 
No package provided, installing all dependencies.
Pipfile found at /Users/<my_username>/code/aind-projects/AIND-Pacman/Pipfile. Considering this to be the project home.
Creating a virtualenv for this project...
⠙Traceback (most recent call last):
  File "/Users/<my_username>/miniconda3/bin/pew", line 11, in <module>
    sys.exit(pew())
  File "/Users/<my_username>/miniconda3/lib/python3.6/site-packages/pew/pew.py", line 737, in pew
    return command(sys.argv[2:])
  File "/Users/<my_username>/miniconda3/lib/python3.6/site-packages/pew/pew.py", line 253, in new_cmd
    args.requirements, rest)
  File "/Users/<my_username>/miniconda3/lib/python3.6/site-packages/pew/pew.py", line 215, in mkvirtualenv
    check_call(["virtualenv", envname] + rest, cwd=str(workon_home))
  File "/Users/<my_username>/miniconda3/lib/python3.6/subprocess.py", line 286, in check_call
    retcode = call(*popenargs, **kwargs)
  File "/Users/<my_username>/miniconda3/lib/python3.6/subprocess.py", line 267, in call
    with Popen(*popenargs, **kwargs) as p:
  File "/Users/<my_username>/miniconda3/lib/python3.6/subprocess.py", line 707, in __init__
    restore_signals, start_new_session)
  File "/Users/<my_username>/miniconda3/lib/python3.6/subprocess.py", line 1326, in _execute_child
    raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/<my_username>/.local/share/virtualenvs'

Virtualenv location: 
Pipfile.lock not found, creating...
Locking [dev-packages] dependencies...
⠼Locking [packages] dependencies...
⠦Updated Pipfile.lock!
Installing dependencies from Pipfile.lock...
An error occured while installing!
/bin/sh: /bin/pip: No such file or directory

You can supply the --ignore-hashes option to 'pipenv install' to bypass this feature.

I am using OSX El Capitan 10.11.5, conda 4.3.11, and I try removing virtualenv from ~/.bash_profile and then running source ~/.bash_profile in terminal:

export WORKON_HOME=$HOME/.virtualenvs
source /Library/Frameworks/Python.framework/Versions/2.7/bin/virtualenvwrapper.sh

However, after doing so, when I run which virtualenv it returns /Users/<my_username>/miniconda3/bin/virtualenv, and when I try uninstalling virtualenv with with conda uninstall virtualenv it returns PackageNotFoundError: Package not found: '' no packages found to remove from environment: /Users/<my_username>/miniconda3

When I delete ~/.virtualenv with rm -rf ~/.virtualenvs it doesn’t make any difference and I get the same errors.

Is pipenv a replacement for both virtualenv AND miniconda such that I need to remove both?

[shanipribadi]

The main issue here is that virtualenv could not find the libpython3.6.m.dylib that is provided by miniconda and required by the python binary from miniconda.

There’s a workaround on Linux with miniconda:

LD_LIBRARY_PATH=$HOME/miniconda3/lib pipenv --three to allow creation of the virtualenv overriding the library path with miniconda lib path and then

cp $HOME/miniconda3/lib/libpython3.6m.so.1.0 $(pipenv --venv)/lib to copy the dynamic library to the virtual env directory so that the python binary can find it in ../lib

you could try DYLD_LIBRARY_PATH or DYLD_FALLBACK_LIBRARY_PATH on Mac, it might behave similar to the above. https://stackoverflow.com/questions/3146274/is-it-ok-to-use-dyld-library-path-on-mac-os-x-and-whats-the-dynamic-library-s

from the look of it, virtualenv does not copy the shared library since the assumption is that the python binary it found is the one available globally in the system.

[nateprewitt]

Great, this is all good stuff. So it definitely looks like there’s something wrong with your virtualenv setup for python3 in microconda3. The fact that both pew and mkvirtualenv fail with python3 means this is outside of the scope of pipenv. When you don’t explicitly create a new project, pipenv install will by default use python3 to create everything. This is why your setup seems to work with the standard python command but not pipenv.

So, my only advice here is to first attempt to see if pip3 or pip3.6 exists on your system. If it does, you can try installing pipenv there and that may possibly fix this issue. Otherwise, you either have a bad install of miniconda3, or there is a linking problem from minoconda3 to python3.6. This line (dyld: Library not loaded: @rpath/libpython3.6m.dylib) makes me think it’s probably the latter, but I can’t say for sure.

Since this is out of the purview of pipenv I’m not sure there’s much more we can do here. I’m going to close this out for now since it’s not a defect in pipenv but please feel free to add any additional notes on things you find if you choose to debug your environment further. Thanks again for checking in on this.