process-dependency-links not respected for subdependencies when generating pipfile.lock

When trying to install a private dependency with a private subdependency that has a private subdependency of its own, pipenv doesn’t follow the specified dependency_links for the “subsubdependency” when generating Pipfile.lock.

$ python -m pipenv.help output

Pipenv version: '11.10.1'

Pipenv location: '/home/user/.local/lib/python2.7/site-packages/pipenv'

Python location: '/usr/bin/python'

Other Python installations in PATH:

• 2.7: /usr/bin/python2.7

• 2.7: /usr/bin/python2.7

• 3.6: /usr/bin/python3.6m

• 3.6: /usr/bin/python3.6

• 2.7.15: /usr/bin/python

• 2.7.15: /usr/bin/python2

• 3.6.5: /usr/bin/python3

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '0',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '4.15.17-300.fc27.x86_64',
 'platform_system': 'Linux',
 'platform_version': '#1 SMP Thu Apr 12 18:19:17 UTC 2018',
 'python_full_version': '2.7.15',
 'python_version': '2.7',
 'sys_platform': 'linux2'}

System environment variables:

• PYTHONDONTWRITEBYTECODE
• PIP_PROCESS_DEPENDENCY_LINKS
• PIP_PYTHON_PATH

Pipenv–specific environment variables:

Debug–specific environment variables:

• PATH: /usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/user/.local/bin:/home/user/bin
• SHELL: /bin/bash
• LANG: en_US.UTF-8
• PWD: /home/user/test/

---

Contents of Pipfile (‘/home/user/test/Pipfile’):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[dev-packages]

[packages]
private_package = {ref = "master", git = "ssh://[e-mail]/private/dependency.git", editable = true}

[requires]
python_version = "2.7"

---

Expected result

The Pipfile.lock should be correctly generated.

Actual result

Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
Could not find a version that matches privatesubsubdependency
There are incompatible versions in the resolved dependencies.

Steps to replicate

Create three repos, the first of which (privatedependency) points to the second (privatesubdependency) in its dependency_links, and the second of which points to the third (privatesubsubdependency).

pipenv install -e 'git+ssh://[e-mail]/private/dependency.git@master#egg=dependency'

Everything will be installed correctly, but pipenv will complain when attempting to generate the Pipfile.lock. If I separately add privatesubdependency to the Pipfile, things start working OK, but it defeats the point of pipenv when I start having to manually manage my subdependencies.

I know that process-dependency-links is deprecated but the "official best ™️ " solution (maintain a private PyPI-compatible server) simply isn’t feasible for my team right now. We’re closely following threads like https://github.com/pypa/pip/issues/3610 and https://github.com/pypa/pip/issues/4187, but until a better alternative is proposed, we rely on this feature pretty heavily.

Thanks!

[wreed4]

I’m not sure pipenv has absorbed the change in latest pip correctly. When trying to use the new syntax (which works correctly in pip), I get the following error:

env/versions/3.6.3/lib/python3.6/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 235, in get_dependencies
    legacy_results = self.get_legacy_dependencies(ireq)
  File "/home/william/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 415, in get_legacy_dependencies
    results, ireq = self.resolve_reqs(download_dir, ireq, wheel_cache)
  File "/home/william/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 297, in resolve_reqs
    results = resolver._resolve_one(reqset, ireq)
  File "/home/william/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pipenv/patched/notpip/_internal/resolve.py", line 335, in _resolve_one
    add_req(subreq, extras_requested=available_requested)
  File "/home/william/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pipenv/patched/notpip/_internal/resolve.py", line 288, in add_req
    wheel_cache=self.wheel_cache,
  File "/home/william/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pipenv/patched/notpip/_internal/req/req_install.py", line 171, in from_req
    "dependencies" % req
pipenv.patched.notpip._internal.exceptions.InstallationError: Direct url requirement (like privatedependency@ git+https://[TOKEN]@[URL_REDACTED].git) are not allowed for dependencies

[techalchemy]

Closing, this is removed in the latest pip, fixed before that in pipenv, thanks all!