How to install setup.py optional dependencies for a package with extras?

Possibly related issues: https://github.com/kennethreitz/pipenv/issues/858, https://github.com/kennethreitz/pipenv/issues/870, https://github.com/kennethreitz/pipenv/issues/1014.

I’m attempting to install a package which uses the extras_require kwarg in its setup function.

The package is in a repo I created here. You can see that the setup.py provides two extras: dev and test. I’m attempting to use pipenv to create a virtual env for this package.

In a virtual environment I would run pip install .[dev] but running the same in pipenv either doesn’t seem to work or I’m doing something wrong.

Describe your environment

1. macOS 10.13.1
2. Python version: Python 3.6.3
3. Pipenv version: pipenv, version 8.3.2

Expected result

I expected a virtual environment to be created by pipenv.

Actual result

Installing .[dev]…
Traceback (most recent call last):
  File "/Users/taylor/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pkg_resources/_vendor/packaging/requirements.py", line 90, in __init__
    req = REQUIREMENT.parseString(requirement_string)
  File "/Users/taylor/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 1617, in parseString
    raise exc
  File "/Users/taylor/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 1607, in parseString
    loc, tokens = self._parse( instring, 0 )
  File "/Users/taylor/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 1379, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/Users/taylor/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 3376, in parseImpl
    loc, exprtokens = e._parse( instring, loc, doActions )
  File "/Users/taylor/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 1379, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/Users/taylor/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 3698, in parseImpl
    return self.expr._parse( instring, loc, doActions, callPreParse=False )
  File "/Users/taylor/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 1379, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/Users/taylor/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 3359, in parseImpl
    loc, resultlist = self.exprs[0]._parse( instring, loc, doActions, callPreParse=False )
  File "/Users/taylor/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 1383, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/Users/taylor/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 2670, in parseImpl
    raise ParseException(instring, loc, self.errmsg, self)
pkg_resources._vendor.pyparsing.ParseException: Expected W:(abcd...) (at char 0), (line:1, col:1)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/taylor/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2873, in __init__
    super(Requirement, self).__init__(requirement_string)
  File "/Users/taylor/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pkg_resources/_vendor/packaging/requirements.py", line 94, in __init__
    requirement_string[e.loc:e.loc + 8]))
pkg_resources.extern.packaging.requirements.InvalidRequirement: Invalid requirement, parse error at "'.[dev]'"

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/taylor/.pyenv/versions/3.6.3/bin/pipenv", line 11, in <module>
    load_entry_point('pipenv==8.3.2', 'console_scripts', 'pipenv')()
  File "/Users/taylor/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/taylor/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/taylor/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/taylor/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/taylor/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/taylor/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pipenv/cli.py", line 1876, in install
    c = pip_install(package_name, ignore_hashes=True, allow_global=system, no_deps=False, verbose=verbose, pre=pre)
  File "/Users/taylor/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pipenv/cli.py", line 1355, in pip_install
    if [x for x in requirements.parse(package_name.split('--hash')[0].split('--trusted-host')[0])][0].vcs:
  File "/Users/taylor/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pipenv/cli.py", line 1355, in <listcomp>
    if [x for x in requirements.parse(package_name.split('--hash')[0].split('--trusted-host')[0])][0].vcs:
  File "/Users/taylor/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pipenv/vendor/requirements/parser.py", line 50, in parse
    yield Requirement.parse(line)
  File "/Users/taylor/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pipenv/vendor/requirements/requirement.py", line 220, in parse
    return cls.parse_line(line)
  File "/Users/taylor/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pipenv/vendor/requirements/requirement.py", line 198, in parse_line
    pkg_req = Req.parse(line)
  File "/Users/taylor/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2918, in parse
    req, = parse_requirements(s)
  File "/Users/taylor/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2866, in parse_requirements
    yield Requirement(line)
  File "/Users/taylor/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2875, in __init__
    raise RequirementParseError(str(e))
pkg_resources.RequirementParseError: Invalid requirement, parse error at "'.[dev]'"

Steps to replicate

Provide the steps to replicate (which usually at least includes the commands and the Pipfile).

1. git clone [e-mail]:astronomerio/boa.git
2. cd boa
3. pipenv install .[dev]

---

Workaround

Here’s my current workaround for others that stumble upon this issue:

1. cd into project directory
2. pipenv shell
3. Inside that virtual env, run pip install -e .[dev]

(This works but kind of feels like “doing it wrong”.)

[romantolkachyov]

I found that zsh trying to interpret .[dev] part so under zsh you must run pipenv install -e '.[dev]' (escape it somehow)

[techalchemy]

@mdeguzis in the meantime you can do pip install --user --upgrade git+https://github.com/kennethreitz/pipenv.git@master#egg=pipenv, but you will have to reinstall from PyPI when the new one is released