Poetry cannot resolve scikit-image dependencies from a private PyPi server

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: MacOS 11
• Poetry version: 1.1.4

Issue

Given a project that depends on scikit-image, and using an external (private) repository that does not implement PyPi’s JSON api, running poetry debug resolve scikit-image resolves scikit-image to have no dependencies (output at the end). However this is incorrect, and the console output clearly shows there is an exception while trying to invoke the PEP517 module.

Unfortunately this manifests itself as a poetry lock working just fine, but incorrectly specifying that there are no dependencies and thus scikit-image fails to import.

I’ve tried various steps to debug this:

1. Creating different venvs with different python versions
2. Explicitly setting the scikit-image dependency with source="pypi"
3. Trying to download the source package and invoking pep517 manually to see what happens
4. Upgrading pip, wheel and setuptools everywhere I can. Side note: I cannot see where WARNING: You are using pip version 20.2.3; however, version 20.3.1 is available. is coming from 😱

I’m willing to help create a fix for this, but I need some pointers as I’m in the dark about what could be going wrong here.

The output:

❯ poetry debug resolve scikit-image -vvv
Resolving dependencies...
   1: fact: document-model-gateway is 1.0.0
   1: derived: document-model-gateway
   1: fact: document-model-gateway depends on scikit-image (^0.17.2)
   1: selecting document-model-gateway (1.0.0)
   1: derived: scikit-image (^0.17.2)
onfido: 1 packages found for scikit-image >=0.17.2,<0.18.0
PyPI: No release information found for scikit-image-0.12.dev0, skipping
PyPI: No release information found for scikit-image-0.9.2, skipping
PyPI: 1 packages found for scikit-image >=0.17.2,<0.18.0
onfido: Downloading sdist: scikit-image-0.17.2.tar.gz
PackageInfo: PEP517 build failed: Command ['/var/folders/k3/splrymbj4wn9fgzcfgkzy93h0000gn/T/tmpegwzhoyc/.venv/bin/python', '-'] errored with the following return code 1, and output:
WARNING: You are using pip version 20.2.3; however, version 20.3.1 is available.
You should consider upgrading via the '/private/var/folders/k3/splrymbj4wn9fgzcfgkzy93h0000gn/T/tmpegwzhoyc/.venv/bin/python -m pip install --upgrade pip' command.
Traceback (most recent call last):
  File "<stdin>", line 6, in <module>
  File "/private/var/folders/k3/splrymbj4wn9fgzcfgkzy93h0000gn/T/tmpegwzhoyc/.venv/lib/python3.9/site-packages/pep517/meta.py", line 53, in build
    _prep_meta(hooks, env, dest)
  File "/private/var/folders/k3/splrymbj4wn9fgzcfgkzy93h0000gn/T/tmpegwzhoyc/.venv/lib/python3.9/site-packages/pep517/meta.py", line 28, in _prep_meta
    reqs = hooks.get_requires_for_build_wheel({})
  File "/private/var/folders/k3/splrymbj4wn9fgzcfgkzy93h0000gn/T/tmpegwzhoyc/.venv/lib/python3.9/site-packages/pep517/wrappers.py", line 160, in get_requires_for_build_wheel
    return self._call_hook('get_requires_for_build_wheel', {
  File "/private/var/folders/k3/splrymbj4wn9fgzcfgkzy93h0000gn/T/tmpegwzhoyc/.venv/lib/python3.9/site-packages/pep517/wrappers.py", line 255, in _call_hook
    self._subprocess_runner(
  File "/private/var/folders/k3/splrymbj4wn9fgzcfgkzy93h0000gn/T/tmpegwzhoyc/.venv/lib/python3.9/site-packages/pep517/wrappers.py", line 75, in quiet_subprocess_runner
    check_output(cmd, cwd=cwd, env=env, stderr=STDOUT)
  File "/usr/local/Cellar/python@3.9/3.9.0_5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 420, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/local/Cellar/python@3.9/3.9.0_5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/private/var/folders/k3/splrymbj4wn9fgzcfgkzy93h0000gn/T/tmpegwzhoyc/.venv/bin/python', '/private/var/folders/k3/splrymbj4wn9fgzcfgkzy93h0000gn/T/tmpegwzhoyc/.venv/lib/python3.9/site-packages/pep517/_in_process.py', 'get_requires_for_build_wheel', '/var/folders/k3/splrymbj4wn9fgzcfgkzy93h0000gn/T/tmpptu68ng0']' returned non-zero exit status 1.
input was : import pep517.build
import pep517.meta

path='/var/folders/k3/splrymbj4wn9fgzcfgkzy93h0000gn/T/tmpl54mmc0o/scikit-image-0.17.2'
system=pep517.build.compat_system(path)
pep517.meta.build(source_dir=path, dest='/var/folders/k3/splrymbj4wn9fgzcfgkzy93h0000gn/T/tmpegwzhoyc/dist', system=system)

   1: selecting scikit-image (0.17.2)
   1: Version solving took 17.438 seconds.
   1: Tried 1 solutions.

Resolution results:

scikit-image 0.17.2

The output from poetry debug resolve against PyPi: https://gist.github.com/orf/b91ba6ac62583523cb8f6999ef69fe96

[orf]

I guess there are two bugs here: the first is that the PEP517 process fails to run on scikit-image, and the second is that Poetry seems to ignore PEP517 failures during resolution.

[neersighted]

This is now obsolete for several reasons:

Poetry should now successfully report PEP 517 build errors – I cannot reproduce a silent PEP 517 failure when resolving using a (bad) sdist from a custom source on 1.2.x.

Poetry now uses wheels wheels whenever possible to gather metadata: #6547.

In addition, PEP 691 and PEP 658 have gained acceptance as the standard ecosystem-wide way to support serving metadata in a PEP 503 repository. This will significantly improve this process moving forward.