Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Using VCS dependencies throws KeyError when writing Pipfile.lock

See original GitHub issue

When converting an existing project from requirements.txt to use pipenv, I performed a pipenv install, which created the following Pipfile:

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
boto = "==2.36.0"
boto3 = "==1.4.5"
braintree = "==3.37.2"
cachetools = "==1.1.6"
celery = "==4.2.1"
certifi = "*"
cffi = "==1.5.2"
cloudinary = "==1.5.0"
coverage = "==4.0.3"
customerio = "==0.1.10"
dj-static = "==0.0.6"
django-ace = "==1.0.2"
django-admin-sortable = "==2.0.20"
django-admin-tools = "==0.8.1"
django-admin-view-permission = "==1.8"
django-allauth = "==0.32.0"
django-annoying = "==0.8.7"
django-cors-headers = "==2.0.2"
django-dbsettings = {ref = "2eb24b58b50c3d47c185f09c17b99c36a4ec2d76",git = "git://github.com/Anton-Shutik/django-dbsettings.git",editable = true}
django-debug-toolbar = "==1.6"
django-filter = "==1.0.2"
django-mailgun = "==0.8.0"
django-nose = "==1.4.4"
django-oauth-toolkit = "==0.10.0"
django-pylibmc = "==0.6.1"
django-reversion = "==1.10.1"
django-seo-js = {ref = "9a0db3856c2647fe6a09dd0cf4103f9f8b1740f5",git = "git://github.com/groveco/django-seo-js.git",editable = true}
django-storages = "==1.6.5"
djangorestframework = "==3.8.2"
djangorestframework-csv = "==1.4.1"
djangorestframework-jsonapi = "==2.6.0"
django-watson = "==1.3.1"
drf-extensions = "==0.3.1"
freezegun = "==0.3.6"
elasticsearch = "<7.0.0,>=6.0.0"
elasticsearch-dsl = "==6.1.0"
gunicorn = "==19.4.5"
hasoffers = "==0.2.1"
jsmin = "==2.2.0"
kombu = "==4.2.0"
librato-metrics = "==0.8.6"
lxml = "==3.5.0"
mailchimp = "==2.0.9"
mock = "==1.3.0"
psycopg2 = "==2.7.3.2"
pylibmc = "==1.5.1"
pytz = "==2018.5"
raven = "==5.32.0"
redis = "==2.10.5"
requests-aws4auth = "==0.9"
requests = "==2.9.1"
selenium = "==3.3.1"
sauceclient = "==0.2.1"
slacker = "==0.9.9"
sqlparse = "==0.2.2"
stripe = "==1.56.0"
tinys3 = "==0.1.11"
user-agents = "==1.1.0"
wsgiref = "==0.1.2"
zenpy = "==2.0.7"
django-sprinklers = {editable = true,git = "https://github.com/chrisclark/django-sprinklers",ref = "e53053f53e0c5a3ff45a3c6c6b8c70af374854b1"}
django-segments = {ref = "00c9591ab72b099b567f855ca0c3b86367164910",git = "https://github.com/groveco/django-segments",editable = true}
sailthru-client = "==2.3.1"
nose-exclude = "==0.5.0"
ddtrace = "==0.12.0"
zeep = "==2.4.0"
flake8 = "==3.5.0"
urllib3 = "==1.22"
Avalara = "==18.5.2"
Django = "==1.11.6"
Beaker = "==1.9.0"
factory_boy = "==2.8.1"
django_celery_beat = "==1.3.0"
Pillow = "==3.3.1"
rest_condition = "==1.0.3"
XlsxWriter = "==0.8.5"
nose_ittr = "==0.0.4"
Explorer = {ref = "7373383598ce9bba2c6e43e3d32342fef11035ba",git = "https://github.com/groveco/django-sql-explorer",editable = true}

[requires]
python_version = "2.7"

Then pipenv attempts to generate the Pipenv.lock file and generates the following error:

Locking [dev-packages] dependencies…
Locking [packages] dependencies…
✔ Success!
Traceback (most recent call last):
  File "/usr/local/Cellar/pipenv/2018.11.26/libexec/bin/pipenv", line 11, in <module>
    load_entry_point('pipenv==2018.11.26', 'console_scripts', 'pipenv')()
  File "/usr/local/Cellar/pipenv/2018.11.26/libexec/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/Cellar/pipenv/2018.11.26/libexec/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/local/Cellar/pipenv/2018.11.26/libexec/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/Cellar/pipenv/2018.11.26/libexec/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/Cellar/pipenv/2018.11.26/libexec/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/Cellar/pipenv/2018.11.26/libexec/lib/python3.7/site-packages/pipenv/vendor/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/Cellar/pipenv/2018.11.26/libexec/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/Cellar/pipenv/2018.11.26/libexec/lib/python3.7/site-packages/pipenv/vendor/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/Cellar/pipenv/2018.11.26/libexec/lib/python3.7/site-packages/pipenv/cli/command.py", line 332, in lock
    pypi_mirror=state.pypi_mirror,
  File "/usr/local/Cellar/pipenv/2018.11.26/libexec/lib/python3.7/site-packages/pipenv/core.py", line 1068, in do_lock
    lockfile=lockfile
  File "/usr/local/Cellar/pipenv/2018.11.26/libexec/lib/python3.7/site-packages/pipenv/utils.py", line 682, in venv_resolve_deps
    lockfile[lockfile_section][k].update(v)
KeyError: 'idna'

I have run pipenv lock --clear multiple times and have gotten different KeyError failures for chardet as well, but mostly for idna. The stack trace is the same, regardless of the key that fails the lookup. When I used pipenv install --skip-lock, all the dependencies installed except for one that needed native headers from libmemcached. The error message that pip reported from that failure was very different and Google-able, so I was able to resolve and get it installed. But I still have no Pipfile.lock and pipenv lock --clear any number of times just generates the same error as above. Help?

$ pipenv --support

Pipenv version: '2018.11.26'

Pipenv location: '/usr/local/Cellar/pipenv/2018.11.26/libexec/lib/python3.7/site-packages/pipenv'

Python location: '/usr/local/Cellar/pipenv/2018.11.26/libexec/bin/python3.7'

Python installations found:

3.7.1: /usr/local/bin/python3
3.7.1: /usr/local/bin/python3.7m
2.7.15: /Users/al_the_x/.local/share/virtualenvs/grove.co-dcWwlTov/bin/python
2.7.15: /usr/local/bin/python
2.7.15: /usr/local/bin/pythonw
2.7.10: /usr/bin/python
2.7.10: /usr/bin/pythonw
2.7.10: /usr/bin/python2.7

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.7.1',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '18.2.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 18.2.0: Mon Nov 12 20:24:46 PST '
                     '2018; root:xnu-4903.231.4~2/RELEASE_X86_64',
 'python_full_version': '3.7.1',
 'python_version': '3.7',
 'sys_platform': 'darwin'}

System environment variables:

PATH
GREP_COLOR
LESS_TERMCAP_mb
TERM_PROGRAM
LESS_TERMCAP_md
NVM_CD_FLAGS
LESS_TERMCAP_me
PIP_PYTHON_PATH
SHELL
TERM
TMPDIR
Apple_PubSub_Socket_Render
TERM_PROGRAM_VERSION
DIRENV_DIR
TERM_SESSION_ID
LESS_TERMCAP_ue
NVM_DIR
USER
LS_COLORS
GREP_COLORS
RPROMPT
SSH_AUTH_SOCK
__CF_USER_TEXT_ENCODING
VIRTUAL_ENV
VIRTUAL_ENV_DISABLE_PROMPT
PAGER
WORKON_HOME
TMUX
DIRENV_WATCHES
LESS_TERMCAP_us
VIRTUALENVWRAPPER_PYTHON
PIPENV_ACTIVE
_git_log_oneline_format
_
PWD
EDITOR
LANG
TMUX_PANE
XPC_FLAGS
PYTHONDONTWRITEBYTECODE
XPC_SERVICE_NAME
HOME
SHLVL
LESS
LOGNAME
PIP_DISABLE_PIP_VERSION_CHECK
VISUAL
LESS_TERMCAP_so
TMUX_PLUGIN_MANAGER_PATH
NVM_BIN
BROWSER
_git_log_medium_format
DIRENV_DIFF
SECURITYSESSIONID
LESS_TERMCAP_se
PIP_SHIMS_BASE_MODULE
PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

PIPENV_ACTIVE: 1

Debug–specific environment variables:

PATH: /usr/local/Cellar/pipenv/2018.11.26/libexec/tools:/usr/local/opt/nvm/versions/node/v10.4.1/bin:/Users/al_the_x/.local/share/virtualenvs/grove.co-dcWwlTov/bin:/Users/al_the_x/bin:/usr/local/Cellar/pipenv/2018.11.26/libexec/tools:/Users/al_the_x/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin
SHELL: /usr/local/bin/zsh
EDITOR: /usr/local/bin/nvim
LANG: en_US.UTF-8
PWD: /Users/al_the_x/Projects/grove.co
VIRTUAL_ENV: /Users/al_the_x/.local/share/virtualenvs/grove.co-dcWwlTov

Contents of Pipfile (‘/Users/al_the_x/Projects/grove.co/Pipfile’):

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
boto = "==2.36.0"
boto3 = "==1.4.5"
braintree = "==3.37.2"
cachetools = "==1.1.6"
celery = "==4.2.1"
certifi = "*"
cffi = "==1.5.2"
cloudinary = "==1.5.0"
coverage = "==4.0.3"
customerio = "==0.1.10"
dj-static = "==0.0.6"
django-ace = "==1.0.2"
django-admin-sortable = "==2.0.20"
django-admin-tools = "==0.8.1"
django-admin-view-permission = "==1.8"
django-allauth = "==0.32.0"
django-annoying = "==0.8.7"
django-cors-headers = "==2.0.2"
django-dbsettings = {ref = "2eb24b58b50c3d47c185f09c17b99c36a4ec2d76",git = "git://github.com/Anton-Shutik/django-dbsettings.git",editable = true}
django-debug-toolbar = "==1.6"
django-filter = "==1.0.2"
django-mailgun = "==0.8.0"
django-nose = "==1.4.4"
django-oauth-toolkit = "==0.10.0"
django-pylibmc = "==0.6.1"
django-reversion = "==1.10.1"
django-seo-js = {ref = "9a0db3856c2647fe6a09dd0cf4103f9f8b1740f5",git = "git://github.com/groveco/django-seo-js.git",editable = true}
django-storages = "==1.6.5"
djangorestframework = "==3.8.2"
djangorestframework-csv = "==1.4.1"
djangorestframework-jsonapi = "==2.6.0"
django-watson = "==1.3.1"
drf-extensions = "==0.3.1"
freezegun = "==0.3.6"
elasticsearch = "<7.0.0,>=6.0.0"
elasticsearch-dsl = "==6.1.0"
gunicorn = "==19.4.5"
hasoffers = "==0.2.1"
jsmin = "==2.2.0"
kombu = "==4.2.0"
librato-metrics = "==0.8.6"
lxml = "==3.5.0"
mailchimp = "==2.0.9"
mock = "==1.3.0"
psycopg2 = "==2.7.3.2"
pylibmc = "==1.5.1"
pytz = "==2018.5"
raven = "==5.32.0"
redis = "==2.10.5"
requests-aws4auth = "==0.9"
requests = "==2.9.1"
selenium = "==3.3.1"
sauceclient = "==0.2.1"
slacker = "==0.9.9"
sqlparse = "==0.2.2"
stripe = "==1.56.0"
tinys3 = "==0.1.11"
user-agents = "==1.1.0"
wsgiref = "==0.1.2"
zenpy = "==2.0.7"
django-sprinklers = {editable = true,git = "https://github.com/chrisclark/django-sprinklers",ref = "e53053f53e0c5a3ff45a3c6c6b8c70af374854b1"}
django-segments = {ref = "00c9591ab72b099b567f855ca0c3b86367164910",git = "https://github.com/groveco/django-segments",editable = true}
sailthru-client = "==2.3.1"
nose-exclude = "==0.5.0"
ddtrace = "==0.12.0"
zeep = "==2.4.0"
flake8 = "==3.5.0"
urllib3 = "==1.22"
Avalara = "==18.5.2"
Django = "==1.11.6"
Beaker = "==1.9.0"
factory_boy = "==2.8.1"
django_celery_beat = "==1.3.0"
Pillow = "==3.3.1"
rest_condition = "==1.0.3"
XlsxWriter = "==0.8.5"
nose_ittr = "==0.0.4"
Explorer = {ref = "7373383598ce9bba2c6e43e3d32342fef11035ba",git = "https://github.com/groveco/django-sql-explorer",editable = true}

[requires]
python_version = "2.7"