Pipenv 2020.11.15 does not resolve local package dependencies if complex setup.py

Issue description

Given

• a local package
• with a complex setup.py file (that is, its dependencies can’t be easily inferred from parsing the file, it must be run)
• ~non-empty dev-packages section (I’m unsure if there are other ways to trigger the issue, or if this is relevant at all, but this seems to do the trick. Apologies if this turns out to be a misleading clue)~
• there is a second dependency

When the Pipfile.lock is removed then Pipenv 2020.11.15 won’t resolve the local package dependencies again.

Pipenv 2020.11.4 is unaffected

Expected result

The local package dependencies are installed and locked

Actual result

They are not!

Steps to replicate

I’ll use an open source Tryton server framework module as an example.

pipenv install [--dev] pudb
git clone -b 5.8 --depth=1 https://github.com/tryton/party
pipenv install ./party
pipenv --rm
rm Pipfile.lock
pipenv install #[--dev]

Now trytond-party dependencies are missing. Namely: the Tryton server framework and some modules read from a file at a precise version range. Tryton modules’ tipical setup.py is rather convoluted for this purpose.

Obviously transitive dependencies (werkzeug, lxml, etc) are missing as well, but note that some aren’t (python-sql and python-stdnum) because the setup.py of the example contains this code line

requires = ['python-sql >= 0.4', 'python-stdnum >= 1.13']
requires.extend(
  #...
)

https://github.com/tryton/party/blob/develop/setup.py#L54

---

$ pipenv --support

Pipenv version: '2020.11.15'

Pipenv location: '/usr/local/lib/python3.9/site-packages/pipenv'

Python location: '/usr/local/bin/python'

Python installations found:

• 3.9.0: /usr/local/bin/python3.9
• 3.9.0: /usr/local/bin/python3
• 3.7.3: /usr/bin/python3.7m
• 3.7.3: /usr/bin/python3.7
• 3.7.3: /usr/bin/python3
• 2.7.16: /usr/bin/python2
• 2.7.16: /usr/bin/python2.7

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.9.0',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '4.15.0-123-generic',
 'platform_system': 'Linux',
 'platform_version': '#126-Ubuntu SMP Wed Oct 21 09:40:11 UTC 2020',
 'python_full_version': '3.9.0',
 'python_version': '3.9',
 'sys_platform': 'linux'}

System environment variables:

• HOSTNAME
• PYTHON_VERSION
• PWD
• HOME
• LANG
• GPG_KEY
• TERM
• SHLVL
• PYTHON_PIP_VERSION
• PYTHON_GET_PIP_SHA256
• PYTHON_GET_PIP_URL
• PATH
• OLDPWD
• _
• PIP_DISABLE_PIP_VERSION_CHECK
• PYTHONDONTWRITEBYTECODE
• PIP_SHIMS_BASE_MODULE
• PIP_PYTHON_PATH
• PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

Debug–specific environment variables:

• PATH: /usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
• LANG: C.UTF-8
• PWD: /tmp/prova

---

Contents of Pipfile (‘/tmp/prova/Pipfile’):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
trytond-party = {path = "./party"}

[dev-packages]
pudb = "*"

[requires]
python_version = "3.9"

Contents of Pipfile.lock (‘/tmp/prova/Pipfile.lock’):

{
    "_meta": {
        "hash": {
            "sha256": "ad53243de5ad0029a5740c63854f3f5f58bd5132735cf8e750aaf7828c4b3221"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.9"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "python-sql": {
            "hashes": [
                "sha256:0f8181af764d4f41671788c86cfc4b2ad06b3f63038793a0e7d16855d7f8d394",
                "sha256:306999bd311fbf50804d76f346655af0a6ff18881ce46c1329256fee40f492c0",
                "sha256:5023f172222fe0c8d6097f705608b77ebb21eb6a0bd711ae0ee00af189209946"
            ],
            "version": "==1.2.1"
        },
        "python-stdnum": {
            "hashes": [
                "sha256:6389a1e7658e39c37e4f10b42d7a51ce620e031bdeae05158519c218e14ff3b5",
                "sha256:fd3a92b8ec82a159c41dbaa3c5397934d090090c92b04e346412e0fd7e6a1b1c"
            ],
            "version": "==1.14"
        },
        "trytond-party": {
            "path": "./party",
            "version": "==5.8.1"
        }
    },
    "develop": {
        "pudb": {
            "hashes": [
                "sha256:e8f0ea01b134d802872184b05bffc82af29a1eb2f9374a277434b932d68f58dc"
            ],
            "index": "pypi",
            "version": "==2019.2"
        },
        "pygments": {
            "hashes": [
                "sha256:381985fcc551eb9d37c52088a32914e00517e57f4a21609f48141ba08e193fa0",
                "sha256:88a0bbcd659fcb9573703957c6b9cff9fab7295e6e76db54c9d00ae42df32773"
            ],
            "markers": "python_version >= '3.5'",
            "version": "==2.7.2"
        },
        "urwid": {
            "hashes": [
                "sha256:588bee9c1cb208d0906a9f73c613d2bd32c3ed3702012f51efe318a3f2127eae"
            ],
            "version": "==2.1.2"
        }
    }
}

[matteius]

@n1ngu I think that is a good plan, closing this for now. I re-opened #4895 and #4812 as I think they do a good job representing what the current issues are with 2021.11.23 and local editable packages.

[n1ngu]

Indeed, the reproduction steps I filed on December 2020 don’t reproduce the issue on 2021.11.23! 🎉

The issues about the reqlib-metadata folder might be more related to the package being installed in editable mode? While my initial goal was installing local packages in editable mode, this was not necessary to trigger issues in the 2020.* series. So maybe it is time to close this issue and refile it with new knowledge and reproduction instructions and also link any potentially related issue? Including a reference here, because it has been hard to keep track of what is what and future readers will enjoy any followup.