We shouldn't require a recent version of setuptools to install xarray

@canol reports on our mailing that our setuptools 41.2 (released 21 August 2019) install requirement is making it hard to install recent versions of xarray at his company: https://groups.google.com/g/xarray/c/HS_xcZDEEtA/m/GGmW-3eMCAAJ

Hello, this is just a feedback about an issue we experienced which caused our internal tools stack to stay with xarray 0.15 version instead of a newer versions.

We are a company using xarray in our internal frameworks and at the beginning we didn’t have any restrictions on xarray version in our requirements file, so that new installations of our framework were using the latest version of xarray. But a few months ago we started to hear complaints from users who were having problems with installing our framework and the installation was failing because of xarray’s requirement to use at least setuptools 41.2 which is released on 21th of August last year. So it hasn’t been a year since it got released which might be considered relatively new.

During the installation of our framework, pip was failing to update setuptools by saying that some other process is already using setuptools files so it cannot update setuptools. The people who are using our framework are not software developers so they didn’t know how to solve this problem and it became so overwhelming for us maintainers that we set the xarray requirement to version >=0.15 <0.16. We also share our internal framework with customers of our company so we didn’t want to bother the customers with any potential problems.

You can see some other people having having similar problem when trying to update setuptools here (although not related to xarray): https://stackoverflow.com/questions/49338652/pip-install-u-setuptools-fail-windows-10

It is not a big deal but I just wanted to give this as a feedback. I don’t know how much xarray depends on setuptools’ 41.2 version.

I was surprised to see this in our setup.cfg file, added by @crusaderky in #3628. The version requirement is not documented in our docs.

Given that setuptools may be challenging to upgrade, would it be possible to relax this version requirement?

[crusaderky]

pandas is really unstable and its API breaks every other version. Extending its support window from 1 to 2 years would be extremely expensive and frustrating to maintain.

[shoyer]

I’m not here to argue, but pyproject.toml was introduced in PEP-518, which was accepted over 4 years ago. I know packaging moves slowly but I’m curious how long something has to be around before becoming “established” and ceases to be “novel”. 😉

It looks like pip has supported pyproject.toml since version 10.0.0, on 2018-04-14. That’s more recent than Python 3.6 (which, to be fair, we are about to drop support for).

Consistent with my earlier suggestion about setuptools, I think we should support the oldest packaging tools that were released at the time of our earliest supported Python release. So if we switch to requiring Python 3.7 in our next major release, we could switch to using pyproject.toml, too.