setup_requires equivalent for dependencies required by other modules

• I have searched the issues of this repo and believe that this is not a duplicate.
• I have searched the documentation and believe that my question is not covered.

Question

I’m trying to use pyjnius which requires Cython to be installed first. Note that it probably isn’t loaded as an inner dependency by it (which I think should change, and I’m looking at https://github.com/kivy/pyjnius for solutions to it), but adding both pyjnius and cython in .toml file and doing a poetry install is resulting in failure, since pyjnius apparently requires Cython to be installed before its setup.

Complete output from command python setup.py egg_info:


    You need Cython to compile Pyjnius.


    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/wp/ss8wgxqs2pz2csrshmbkzvs80000gn/T/pip-install-pq9f9wz0/pyjnius/setup.py", line 65, in <module>
        from Cython.Distutils import build_ext
    ModuleNotFoundError: No module named 'Cython'

This is similar to https://github.com/sdispater/poetry/issues/37 since https://pypi.org/pypi/pyjnius/json doesn’t list cython as a required dependency but even adding cython manually to the toml file doesn’t work here.

The only thing that did work was doing poetry install twice, once with just cython and other time with pyjnius added, but of course, I can’t have this in an automated setup.

So is there a way to automate this weird pre-setup dependency case for this module with poetry ?

[sangaline]

The same thing happens with scikit-learn v0.21.3.

      File "sklearn/utils/setup.py", line 8, in configuration
        from Cython import Tempita
    ModuleNotFoundError: No module named 'Cython'

[nirvana-msu]

I’m having a similar issue while trying to install ray using a Nexus PyPI mirror. Ray depends on gpustat, for which installation fails with the error:

distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('pytest-runner')

Poetry fails to infer this dependency via parsing setup.py due to the way it is defined:

if sys.version_info >= (3, 5):
    tests_requires += ['pytest>=5.4.1']
    setup_requires += ['pytest-runner>=5.0'],
else:
    tests_requires += ['pytest<5.0', 'more_itertools<8.0']
    setup_requires += ['pytest-runner<5.0'],

Adding pytest-runner dependency to pyproject.toml is also not sufficient as gpustat gets installed before it.

As others have said, it can and should be fixed upstream, and it can also be addressed by doing poetry run pip install pytest-runner before poetry install, but that’s far from perfect…

It would be awesome if Poetry, as a workaround, provided a way to specify additional package dependencies within [tool.poetry.dependencies] section, something like the following:

gpustat = { version = "*", depends_on= ["pytest-runner"]}  # actually, to cater for a general case it should be possible to provide a version constraint for those dependencies

@sdispater does it make sense? It’s a workaround, but it would at least solve the problem in the short term…