Restore support for newlines in Summary

setuptools version

>= 59

Python version

3.8

OS

linux

Additional environment information

No response

Description

builds failing with

 File "/code/venv/lib/python3.8/site-packages/setuptools/dist.py", line 151, in single_line
   raise ValueError('Newlines are not allowed')

Expected behavior

Warning or nothing, just escape silently \n for oneline metadata fields

Lot of already released eggs wont ever comply to this policy, but we need to install them at the version they are pinned in our builds for the sake of reproducibility.

How to Reproduce

pip install django-hijack==2.1.10

Output

  ERROR: Command errored out with exit status 1:
   command: venv/bugpip/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-10qe4uxx/django-hijack_c7b8d72ff0ad41998610fe76e5822048/setup.py'"'"'; __file__='"'"'/tmp/pip-install-10qe4uxx/django-hijack_c7b8d72ff0ad41998610fe76e5822048/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-awfzhldp
       cwd: /tmp/pip-install-10qe4uxx/django-hijack_c7b8d72ff0ad41998610fe76e5822048/
  Complete output (26 lines):
  running egg_info
  creating /tmp/pip-pip-egg-info-awfzhldp/django_hijack.egg-info
  writing /tmp/pip-pip-egg-info-awfzhldp/django_hijack.egg-info/PKG-INFO
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-10qe4uxx/django-hijack_c7b8d72ff0ad41998610fe76e5822048/setup.py", line 44, in <module>
      setup(name="django-hijack",
    File "venv/bugpip/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
      return distutils.core.setup(**attrs)
    File "/usr/lib/python3.8/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/usr/lib/python3.8/distutils/dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "venv/bugpip/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 292, in run
      writer(self, ep.name, os.path.join(self.egg_info, ep.name))
    File "venv/bugpip/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 656, in write_pkg_info
      metadata.write_pkg_info(cmd.egg_info)
    File "/usr/lib/python3.8/distutils/dist.py", line 1117, in write_pkg_info
      self.write_pkg_file(pkg_info)
    File "venv/bugpip/lib/python3.8/site-packages/setuptools/dist.py", line 167, in write_pkg_file
      write_field('Summary', single_line(self.get_description()))
    File "venv/bugpip/lib/python3.8/site-packages/setuptools/dist.py", line 151, in single_line
      raise ValueError('Newlines are not allowed')
  ValueError: Newlines are not allowed

Code of Conduct

• I agree to follow the PSF Code of Conduct

[jaraco]

In v59.4.0, I’ve reverted the change, restoring support for bad metadata.

[boegel]

I strongly support the request to let setuptools strip out the excess newlines, rather than making this a hard error and essentially breaking the installation of existing Python package releases that happen to have a newline in the description field (see https://github.com/easybuilders/easybuild-easyblocks/pull/2623 for our particular case, where is happened by accident due to fixing code style issues that were being reported by flake8, and we were totally unaware of it).

I fully understand the motivation for trying to get people to not using newlines in packages (cfr. #1390), but frankly I think this is the wrong approach…

Considering looking at this from the point of view of someone who is not very familiar with installing Python packages. All of a sudden, they’re starting to see errors like this:

ValueError: Newlines are not allowed

First of all: the error is very confusing, since it doesn’t mention at all where newlines are not allowed. In the code? In the package name? What’s going on?! In #2870 it was acknowledged that the error message could be better, and it was mentioned that “I expect these errors to be rare.”. I think this lengthy discussion counters that last argument…

The first reaction from someone hitting this error is most likely going to be “I knew packaging was messy in Python, but this is just... wow”. What’s worse is that the end user has no easy way around it. I appreciate the suggested workarounds mentioned in https://github.com/pypa/setuptools/issues/2893#issuecomment-973173978, but I think these are outside of what a person who doesn’t have extensive experience in installing Python packages would consider doing. Sure, most of those suggestions are targeted at package maintainers, but what if the package maintainer is unaware, or not responsive?

Even a simple workaround like using setuptools<59 is probably too much for non-experienced people. They often use whatever setuptools/pip/python comes with their OS, and they wouldn’t even know where to begin to figure out this problem, let alone fix it. I’m very sure that most people just use pip install example. If that fails, they’re likely to just give up quite quickly… (I don’t applaud this, but I do feel it’s a common reaction)

I have seen my share of problems with getting software installed, and I must say it took me a while to get my head around what was happening when I first ran into this problem. I simply couldn’t understand why setuptools wouldn’t just actively strip out the problematic newline characters, print a big fat warning, and then just go ahead and install the package anyway (because why wouldn’t it).

Please consider changing this aggressive behavior of breaking a package installation for what most would consider a very silly problem that is easy to bypass (by just stripping out the newlines, replacing them with a space), before the setuptools versions that make this a hard error are being picked up in various Linux distributions and start causing more trouble around the world. For some environments (like Gentoo Prefix), it is already too late, but they will likely also pick up a new setuptools version, so hopefully the breakage can be short-lived.

It’s now clear that there are dozens of Python packages out there that have a range of versions that are essentially no longer installable with a recent version of setuptools, although there’s no good reason for it (since the workaround in setuptools is trivial).

I’m happy to contribute to help with working out a better approach for dealing with excess newlines in package descriptions, without massively disrupting the user experience.