File encoding issue in _install_wheel (pip 20.2, python 2)

Environment

• pip version: 20.2
• Python version: 2.7
• OS: linux

Description

pip install mr.bob fails with pip 20.2:

Processing /home/sbi-local/.cache/pip/wheels/d3/64/58/cd7b9ff05f450397981fa37a0a279ed1d12cf763af16d8d7d7/mr.bob-0.1.2-py2-none-any.whl
Requirement already satisfied: Jinja2>=2.5.0 in /home/sbi-local/.virtualenvs/tempenv-7d78147760d35/lib/python2.7/site-packages (from mr.bob) (2.11.2)
Requirement already satisfied: six>=1.2.0 in /home/sbi-local/.virtualenvs/tempenv-7d78147760d35/lib/python2.7/site-packages (from mr.bob) (1.15.0)
Requirement already satisfied: setuptools in /home/sbi-local/.virtualenvs/tempenv-7d78147760d35/lib/python2.7/site-packages (from mr.bob) (44.1.1)
Requirement already satisfied: MarkupSafe>=0.23 in /home/sbi-local/.virtualenvs/tempenv-7d78147760d35/lib/python2.7/site-packages (from Jinja2>=2.5.0->mr.bob) (1.1.1)
Installing collected packages: mr.bob
ERROR: Exception:
Traceback (most recent call last):
  File "/home/sbi-local/.virtualenvs/tempenv-7d78147760d35/lib/python2.7/site-packages/pip/_internal/cli/base_command.py", line 216, in _main
    status = self.run(options, args)
  File "/home/sbi-local/.virtualenvs/tempenv-7d78147760d35/lib/python2.7/site-packages/pip/_internal/cli/req_command.py", line 182, in wrapper
    return func(self, options, args)
  File "/home/sbi-local/.virtualenvs/tempenv-7d78147760d35/lib/python2.7/site-packages/pip/_internal/commands/install.py", line 421, in run
    pycompile=options.compile,
  File "/home/sbi-local/.virtualenvs/tempenv-7d78147760d35/lib/python2.7/site-packages/pip/_internal/req/__init__.py", line 90, in install_given_reqs
    pycompile=pycompile,
  File "/home/sbi-local/.virtualenvs/tempenv-7d78147760d35/lib/python2.7/site-packages/pip/_internal/req/req_install.py", line 831, in install
    requested=self.user_supplied,
  File "/home/sbi-local/.virtualenvs/tempenv-7d78147760d35/lib/python2.7/site-packages/pip/_internal/operations/install/wheel.py", line 829, in install_wheel
    requested=requested,
  File "/home/sbi-local/.virtualenvs/tempenv-7d78147760d35/lib/python2.7/site-packages/pip/_internal/operations/install/wheel.py", line 658, in _install_wheel
    file.save()
  File "/home/sbi-local/.virtualenvs/tempenv-7d78147760d35/lib/python2.7/site-packages/pip/_internal/operations/install/wheel.py", line 442, in save
    with self._zip_file.open(self.src_record_path) as f:
  File "/usr/lib/python2.7/zipfile.py", line 984, in open
    zinfo = self.getinfo(name)
  File "/usr/lib/python2.7/zipfile.py", line 932, in getinfo
    'There is no item named %r in the archive' % name)
KeyError: "There is no item named u'mrbob/tests/templates/encodingc\\u030c/mapc\\u030ca/c\\u0301a.bob' in the archive"

I’ve not had time to investigate if that error comes from a bug in that old package or is a bug in the new _install_wheel. It works fine with pip 20.1 though, as well as with pip 20.2 on python 3.

[uranusjr]

This sounds like a hairy topic. Quoting Python 2 documentation on zipfile:

There is no official file name encoding for ZIP files. If you have unicode file names, you must convert them to byte strings in your desired encoding before passing them to write(). WinZip interprets all file names as encoded in CP437, also known as DOS Latin.

So I guess there is no “correct” way here. Python 3 detects UTF-8 support with a header flag. I’m not familiar with the ZIP spec to comment on the approach, and would assume it’s the best we can do.

[uranusjr]

I update #8684 to always assume UTF-8 instead. This should be quite straightforward now.