Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Pipenv lock still takes way too long to complete

See original GitHub issue

pipenv lock is still taking up to 30 minutes to complete for my medium-sized project.

While #4403 was closed as fixed and indeed no full tree copies are made any more, pipenv lock is still unworkably slow. What used to take 50 seconds under 2018.11.26, now takes 30 minutes to figure out that nothing changed:

$ export PIPENV_INSTALL_TIMEOUT=10000
$ cp Pipfile.lock Pipfile.lock.before
$ time pipenv lock
Locking [dev-packages] dependencies…
Building requirements...
Resolving dependencies...
✔ Success!
Locking [packages] dependencies…
Building requirements...
Resolving dependencies...
✔ Success!
Updated Pipfile.lock (0e1198)!

real	29m19.701s
user	28m39.300s
sys	4m8.222s
$ diff -su Pipfile.lock.old Pipfile.lock
Files Pipfile.lock.old and Pipfile.lock are identical

This is making it untenable to keep using Pipenv.

Full pipenv lock -v output in a gist.

Pipfile:

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]
flake8 = "*"
flake8-bugbear = "*"
black = "==19.10b0"
pre-commit = "*"
pytest = "*"
pytest-cov = "*"

[packages]
project-name = {editable = true,path = "."}
"zc.buildout" = "*"

[requires]
python_version = "3.6"

where project-name (redacted name) lists the following install-requires lines:

install_requires =
    apache-airflow[aws,celery,postgres,redis] >= 1.10.11
    airflow_multi_dagrun
    airflow-prometheus-exporter

$ pipenv --support

Pipenv version: '2020.8.13'

Pipenv location: '/usr/local/Cellar/pipenv/2020.8.13/libexec/lib/python3.8/site-packages/pipenv'

Python location: '/usr/local/Cellar/pipenv/2020.8.13/libexec/bin/python3.8'

Python installations found:

3.9.0: /Users/mj/bin/python3.9
3.8.5: /usr/local/bin/python3
3.8.5: /usr/local/bin/python3.8
3.8.3: /Users/mj/bin/python3.8
3.7.7: /Users/mj/bin/python3.7
3.7.6: /opt/miniconda3/bin/python3
3.7.6: /opt/miniconda3/bin/python3.7m
3.7.6: /opt/miniconda3/bin/python3.7
3.7.3: /Users/mj/.pyenv/versions/3.7.3/bin/python3
3.6.10: /Users/mj/bin/python3.6
3.6.1: /Users/mj/bin/pypy3
3.5.9: /Users/mj/bin/python3.5
3.4.10: /Users/mj/bin/python3.4
3.3.7: /Users/mj/bin/python3.3
3.2.6: /Users/mj/bin/python3.2
2.7.18: /Users/mj/bin/python2.7
2.7.16: /usr/bin/python2.7
2.6.9: /Users/mj/bin/python2.6
2.5.6: /Users/mj/bin/python2.5
2.4.6: /Users/mj/bin/python2.4

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.8.5',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '17.7.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 17.7.0: Wed May 27 17:00:02 PDT '
                     '2020; root:xnu-4570.71.80.1~1/RELEASE_X86_64',
 'python_full_version': '3.8.5',
 'python_version': '3.8',
 'sys_platform': 'darwin'}

System environment variables:

PATH
AUTOJUMP_ERROR_PATH
NVM_INC
TERM_PROGRAM
GIT_PROMPT_END
rvm_bin_path
AUTOJUMP_SOURCED
GEM_HOME
NVM_CD_FLAGS
SHELL
TERM
HISTSIZE
TMPDIR
COPYFILE_DISABLE
IRBRC
Apple_PubSub_Socket_Render
CONDA_SHLVL
PERL5LIB
TERM_PROGRAM_VERSION
CONDA_PROMPT_MODIFIER
TERM_SESSION_ID
MY_RUBY_HOME
PERL_MB_OPT
LC_ALL
USER
HISTFILESIZE
NVM_DIR
COMMAND_MODE
CONDA_EXE
rvm_path
SSH_AUTH_SOCK
__CF_USER_TEXT_ENCODING
PTPIMG_API_KEY
_CE_CONDA
rvm_prefix
_
CONDA_PREFIX
PWD
HOMEBREW_GITHUB_API_TOKEN
EDITOR
LANG
ITERM_PROFILE
PYTHONSTARTUP
XPC_FLAGS
ITERM_ORIG_PS1
XPC_SERVICE_NAME
HISTCONTROL
_CE_M
FIGNORE
rvm_version
COLORFGBG
HOME
SHLVL
LC_TERMINAL_VERSION
ITERM_PREV_PS1
ITERM_SESSION_ID
PERL_LOCAL_LIB_ROOT
LOGNAME
CONDA_PYTHON_EXE
LC_CTYPE
GEM_PATH
GIT_PROMPT_ONLY_IN_REPO
CONDA_DEFAULT_ENV
NVM_BIN
DISPLAY
LC_TERMINAL
HGSRC
SECURITYSESSIONID
RUBY_VERSION
PERL_MM_OPT
COLORTERM
PIP_DISABLE_PIP_VERSION_CHECK
PYTHONDONTWRITEBYTECODE
PIP_SHIMS_BASE_MODULE
PIP_PYTHON_PATH
PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

Debug–specific environment variables:

PATH: /usr/local/Cellar/pipenv/2020.8.13/libexec/tools:/usr/local/google-sdks/google-cloud-sdk/bin:/Users/mj/perl5/bin:/Users/mj/.nvm/versions/node/v12.14.1/bin:/Users/mj/.rvm/gems/ruby-2.3.0/bin:/Users/mj/.rvm/gems/ruby-2.3.0@global/bin:/Users/mj/.rvm/rubies/ruby-2.3.0/bin:/opt/miniconda3/bin:/opt/miniconda3/condabin:/Users/mj/bin:/Users/mj/Development/Library/zopatista_setup/bin:/Users/mj/Development/Library/zopatista_setup/audio:/usr/local/Cellar/ruby/2.7.1_2/bin:/Users/mj/.cargo/bin:/Users/mj/.poetry/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/usr/local/MacGPG2/bin:/usr/local/sbin:/opt/X11/bin:/Applications/Postgres.app/Contents/Versions/latest/bin:/Users/mj/.local/bin:/Users/mj/.rvm/bin:/Applications/Visual Studio Code.app/Contents/Resources/app/bin
SHELL: /bin/bash
EDITOR: vim
LANG: en_GB.UTF-8
PWD: /Users/mj