Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Error importing requirements.txt file containing VCS URL with environment variable

See original GitHub issue

Issue description

I just tried to import the following the file, but failed.

  • requirements.txt
-e git+https://${GITHUB_TOKEN}@github.com/skyoo2003/myprivate.git@v1.0.0#egg=myprivate

Expected result

Expect the dependency to be included in “Pipfile” and successful installation

Actual result

I typed the enter key twice because it didn’t seem to stop, and it failed with the error message below.

$ pipenv install -r requirements.txt --verbose
Requirements file provided! Importing into Pipfile…
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Building requirements...
Resolving dependencies...
⠹ Locking...
⠴ Locking...
⠏ Locking...ERROR:pip.subprocessor:Command errored out with exit status 128:
 command: git clone -q 'https://****@github.com/skyoo2003/myprivate.git' /tmp/pipenv-zlxgv8gf-src/myprivate
     cwd: None
Complete output (2 lines):
remote: Invalid username or password.
fatal: Authentication failed for 'https://****@github.com/skyoo2003/myprivate.git/'
----------------------------------------
✘ Locking Failed! 
Traceback (most recent call last):
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/resolver.py", line 807, in <module>
    main()
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/resolver.py", line 803, in main
    parsed.requirements_dir, parsed.packages, parse_only=parsed.parse_only)
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/resolver.py", line 785, in _main
    resolve_packages(pre, clear, verbose, system, write, requirements_dir, packages)
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/resolver.py", line 753, in resolve_packages
    requirements_dir=requirements_dir,
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/resolver.py", line 736, in resolve
    req_dir=requirements_dir
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/utils.py", line 1379, in resolve_deps
    req_dir=req_dir,
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/utils.py", line 1084, in actually_resolve_deps
    deps, index_lookup, markers_lookup, project, sources, req_dir, clear, pre
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/utils.py", line 649, in create
    pre=pre, clear=clear
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/utils.py", line 463, in get_metadata
    req, resolver=transient_resolver
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/utils.py", line 540, in get_deps_from_req
    req_list, lockfile = get_vcs_deps(reqs=[req])
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/utils.py", line 1867, in get_vcs_deps
    with temp_path(), locked_repository(requirement) as repo:
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/utils.py", line 2029, in locked_repository
    with requirement.req.locked_vcs_repo(src_dir=src_dir) as repo:
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 2141, in locked_vcs_repo
    vcsrepo = self.get_vcs_repo(src_dir=src_dir)
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 2097, in get_vcs_repo
    vcsrepo.obtain()
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/vendor/requirementslib/models/vcs.py", line 77, in obtain
    self.repo_backend.obtain(self.checkout_directory, self.parsed_url)
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/patched/notpip/_internal/vcs/versioncontrol.py", line 527, in obtain
    self.fetch_new(dest, url, rev_options)
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/patched/notpip/_internal/vcs/git.py", line 225, in fetch_new
    self.run_command(make_command('clone', '-q', url, dest))
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/patched/notpip/_internal/vcs/versioncontrol.py", line 667, in run_command
    log_failed_cmd=log_failed_cmd)
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/patched/notpip/_internal/utils/subprocess.py", line 242, in call_subprocess
    raise InstallationError(exc_msg)
pipenv.patched.notpip._internal.exceptions.InstallationError: Command errored out with exit status 128: git clone -q 'https://****@github.com/skyoo2003/myprivate.git' /tmp/pipenv-zlxgv8gf-src/myprivate Check the logs for full command output.
ERROR:pip.subprocessor:Command errored out with exit status 128:
 command: git clone -q 'https://****@github.com/skyoo2003/myprivate.git' /tmp/pipenv-zlxgv8gf-src/myprivate
     cwd: None
Complete output (2 lines):
remote: Invalid username or password.
fatal: Authentication failed for 'https://****@github.com/skyoo2003/myprivate.git/'
----------------------------------------
Traceback (most recent call last):
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/resolver.py", line 807, in <module>
    main()
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/resolver.py", line 803, in main
    parsed.requirements_dir, parsed.packages, parse_only=parsed.parse_only)
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/resolver.py", line 785, in _main
    resolve_packages(pre, clear, verbose, system, write, requirements_dir, packages)
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/resolver.py", line 753, in resolve_packages
    requirements_dir=requirements_dir,
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/resolver.py", line 736, in resolve
    req_dir=requirements_dir
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/utils.py", line 1379, in resolve_deps
    req_dir=req_dir,
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/utils.py", line 1084, in actually_resolve_deps
    deps, index_lookup, markers_lookup, project, sources, req_dir, clear, pre
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/utils.py", line 649, in create
    pre=pre, clear=clear
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/utils.py", line 463, in get_metadata
    req, resolver=transient_resolver
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/utils.py", line 540, in get_deps_from_req
    req_list, lockfile = get_vcs_deps(reqs=[req])
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/utils.py", line 1867, in get_vcs_deps
    with temp_path(), locked_repository(requirement) as repo:
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/utils.py", line 2029, in locked_repository
    with requirement.req.locked_vcs_repo(src_dir=src_dir) as repo:
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 2141, in locked_vcs_repo
    vcsrepo = self.get_vcs_repo(src_dir=src_dir)
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 2097, in get_vcs_repo
    vcsrepo.obtain()
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/vendor/requirementslib/models/vcs.py", line 77, in obtain
    self.repo_backend.obtain(self.checkout_directory, self.parsed_url)
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/patched/notpip/_internal/vcs/versioncontrol.py", line 527, in obtain
    self.fetch_new(dest, url, rev_options)
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/patched/notpip/_internal/vcs/git.py", line 225, in fetch_new
    self.run_command(make_command('clone', '-q', url, dest))
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/patched/notpip/_internal/vcs/versioncontrol.py", line 667, in run_command
    log_failed_cmd=log_failed_cmd)
  File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/patched/notpip/_internal/utils/subprocess.py", line 242, in call_subprocess
    raise InstallationError(exc_msg)
pipenv.patched.notpip._internal.exceptions.InstallationError: Command errored out with exit status 128: git clone -q 'https://****@github.com/skyoo2003/myprivate.git' /tmp/pipenv-zlxgv8gf-src/myprivate Check the logs for full command output.

Steps to replicate

Provide the steps to replicate (which usually at least includes the commands and the Pipfile).

  1. Prepare a requirements.txt file containing VCS URL with environment variable and initialized Pipfile
  2. Execute a command such as pipenv install -r requirements.txt --verbose
$ pipenv --support

Pipenv version: '2020.4.1b1'

Pipenv location: '/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv'

Python location: '/home/lukas/.pyenv/versions/3.6.10/bin/python3.6'

Python installations found:

  • 3.6.10: /home/lukas/.pyenv/versions/3.6.10/bin/python3.6
  • 3.6.10: /home/lukas/.pyenv/versions/3.6.10/bin/python3.6m
  • 3.6.10: /home/lukas/.pyenv/versions/3.6.10/bin/python3
  • 3.6.10: /home/lukas/.pyenv/versions/3.6.10/bin/python3.6
  • 3.6.9: /usr/bin/python3.6
  • 3.6.9: /usr/bin/python3.6m
  • 3.6.9: /usr/bin/python3
  • 2.7.17: /usr/bin/python2
  • 2.7.17: /usr/bin/python2.7

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.6.10',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '5.3.0-51-generic',
 'platform_system': 'Linux',
 'platform_version': '#44~18.04.2-Ubuntu SMP Thu Apr 23 14:27:18 UTC 2020',
 'python_full_version': '3.6.10',
 'python_version': '3.6',
 'sys_platform': 'linux'}

System environment variables:

  • CLUTTER_IM_MODULE
  • LS_COLORS
  • LC_MEASUREMENT
  • LC_PAPER
  • LC_MONETARY
  • XDG_MENU_PREFIX
  • LANG
  • LESS
  • DISPLAY
  • PYENV_ROOT
  • OLDPWD
  • GNOME_SHELL_SESSION_MODE
  • MAVEN_HOME
  • COLORTERM
  • DESKTOP_AUTOSTART_ID
  • PYENV_VIRTUALENV_INIT
  • USERNAME
  • JAVA_HOME
  • PYENV_HOOK_PATH
  • XDG_VTNR
  • ZSH
  • SSH_AUTH_SOCK
  • MANDATORY_PATH
  • LC_NAME
  • XDG_SESSION_ID
  • USER
  • PYENV_DIR
  • PAGER
  • LSCOLORS
  • DESKTOP_SESSION
  • QT4_IM_MODULE
  • TEXTDOMAINDIR
  • GNOME_TERMINAL_SCREEN
  • DEFAULTS_PATH
  • PWD
  • HOME
  • TEXTDOMAIN
  • SSH_AGENT_PID
  • PYENV_VERSION
  • QT_ACCESSIBILITY
  • XDG_SESSION_TYPE
  • XDG_DATA_DIRS
  • XDG_SESSION_DESKTOP
  • LC_ADDRESS
  • LC_NUMERIC
  • GTK_MODULES
  • PAPERSIZE
  • WINDOWPATH
  • VTE_VERSION
  • TERM
  • SHELL
  • QT_IM_MODULE
  • XMODIFIERS
  • IM_CONFIG_PHASE
  • XDG_CURRENT_DESKTOP
  • GPG_AGENT_INFO
  • GNOME_TERMINAL_SERVICE
  • PYENV_SHELL
  • XDG_SEAT
  • SHLVL
  • LANGUAGE
  • LC_TELEPHONE
  • GDMSESSION
  • GNOME_DESKTOP_SESSION_ID
  • LOGNAME
  • DBUS_SESSION_BUS_ADDRESS
  • XDG_RUNTIME_DIR
  • XAUTHORITY
  • XDG_CONFIG_DIRS
  • PATH
  • LC_IDENTIFICATION
  • SESSION_MANAGER
  • GTK_IM_MODULE
  • LC_TIME
  • PIP_DISABLE_PIP_VERSION_CHECK
  • PYTHONDONTWRITEBYTECODE
  • PIP_SHIMS_BASE_MODULE
  • PIP_PYTHON_PATH
  • PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

Debug–specific environment variables:

  • PATH: /home/lukas/.pyenv/versions/3.6.10/bin:/home/lukas/.pyenv/libexec:/home/lukas/.pyenv/plugins/python-build/bin:/home/lukas/.pyenv/plugins/pyenv-virtualenv/bin:/home/lukas/.pyenv/plugins/pyenv-update/bin:/home/lukas/.pyenv/plugins/pyenv-installer/bin:/home/lukas/.pyenv/plugins/pyenv-doctor/bin:/usr/lib/jvm/adoptopenjdk-8-openj9-amd64/bin:/usr/lib/maven/apache-maven-3.6.3/bin:/home/lukas/.pyenv/plugins/pyenv-virtualenv/shims:/home/lukas/.pyenv/shims:/home/lukas/.pyenv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/lukas/.fzf/bin
  • SHELL: /usr/bin/zsh
  • LANG: ko_KR.UTF-8
  • PWD: /home/lukas/pipenv-test

Contents of Pipfile (‘/home/lukas/pipenv-test/Pipfile’):

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
myprivate = {editable = true, git = "https://****@github.com/skyoo2003/myprivate.git", ref = "v1.0.0"}

[requires]
python_version = "3.6"

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:8 (1 by maintainers)

github_iconTop GitHub Comments

2reactions
CaselITcommented, Jul 29, 2020

It also does not work when using basic auth to authenticate to a git provider.

The problem seems to be that the command that is run has the sensible parts (token/password) sanitized.

Example is a line in the requirement file with a repository and an user and password in base auth http form, like

https://gitlab+deploy-token-123:foobar@gitlab.com/some/project.git#egg=some-egg

The command that is run by subprocess is

['git', 'clone', '-q', 'https://gitlab+deploy-token-123:****@gitlab.com/some/project.git', '/tmp/pipenv-asf23-src/project']

that clearly does not work That command line is from this file https://github.com/pypa/pipenv/blob/b29a4884e24dd2dd72f379d64b23779a676d2ef9/pipenv/patched/notpip/_internal/utils/subprocess.py#L186-L191

0reactions
matteiuscommented, Aug 21, 2022

Can this be checked with the new pipenv requirements command which replaced the pipenv lock -r? Version pipenv==2022.8.19

Read more comments on GitHub >

github_iconTop Results From Across the Web

How to state in requirements.txt a direct github source
I got a "should either be a path to a local project or a VCS url beginning with svn+, git+, hg+, or bzr+"...
Read more >
User Guide - pip documentation v22.3.1
“Requirements files” are files containing a list of items to be installed using pip install like so: Unix/macOS. python -m pip install -r...
Read more >
pipenv Changelog - pyup.io
Fixed edge case where a non-editable file, url or vcs would overwrite the value ``no_deps`` ... Support expanding environment variables in requirement URLs....
Read more >
Notebook-scoped Python libraries | Databricks on AWS
Any subdirectories in the file path must already exist. If you run %pip freeze > /dbfs/<new-directory>/requirements.txt , ...
Read more >
Installing Packages - Python Packaging User Guide - Python.org
Upgrading packages. Installing to the User Site. Requirements files. Installing from VCS. Installing from other Indexes. Installing from a local src tree.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

powershell: `pipenv shell` still spawns CMD

Next page

Pipenv 2020 error's out not finding the requirement.txt