race condition replacing a setuptools dependency IOError METADATA no such file

This is a strange one. I have narrowed it down to setuptools new version (patch) as the only thing that has changed recently that causes a flow to break. I have reduced the problem to the bare minimum, but the flow that is actually occurring is a bit more complicated, here are the minimum steps needed to reproduce the problem.

I have been installing junit-xml and zope.interface in a test environment using the following command:

python -m pip install --upgrade --force-reinstall junit-xml zope.interface

This is the output:

Collecting junit-xml
  Using cached junit-xml-1.7.tar.gz
Collecting zope.interface
  Using cached zope.interface-4.3.3-cp27-cp27m-win_amd64.whl
Collecting six (from junit-xml)
  Using cached six-1.10.0-py2.py3-none-any.whl
Collecting setuptools (from zope.interface)
  Using cached setuptools-34.1.0-py2.py3-none-any.whl
Collecting appdirs>=1.4.0 (from setuptools->zope.interface)
  Using cached appdirs-1.4.0-py2.py3-none-any.whl
Collecting packaging>=16.8 (from setuptools->zope.interface)
  Using cached packaging-16.8-py2.py3-none-any.whl
Collecting pyparsing (from packaging>=16.8->setuptools->zope.interface)
  Using cached pyparsing-2.1.10-py2.py3-none-any.whl
Installing collected packages: six, junit-xml, appdirs, pyparsing, packaging, setuptools, zope.interface
  Found existing installation: six 1.10.0
    Uninstalling six-1.10.0:
      Successfully uninstalled six-1.10.0
  Rolling back uninstall of six
Exception:
Traceback (most recent call last):
  File "C:\Python27\lib\site-packages\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "C:\Python27\lib\site-packages\pip\commands\install.py", line 342, in run
    prefix=options.prefix_path,
  File "C:\Python27\lib\site-packages\pip\req\req_set.py", line 784, in install
    **kwargs
  File "C:\Python27\lib\site-packages\pip\req\req_install.py", line 851, in install
    self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
  File "C:\Python27\lib\site-packages\pip\req\req_install.py", line 1064, in move_wheel_files
    isolated=self.isolated,
  File "C:\Python27\lib\site-packages\pip\wheel.py", line 247, in move_wheel_files
    prefix=prefix,
  File "C:\Python27\lib\site-packages\pip\locations.py", line 140, in distutils_scheme
    d = Distribution(dist_args)
  File "C:\Python27\lib\site-packages\setuptools\dist.py", line 320, in __init__
    _Distribution.__init__(self, attrs)
  File "C:\Python27\lib\distutils\dist.py", line 287, in __init__
    self.finalize_options()
  File "C:\Python27\lib\site-packages\setuptools\dist.py", line 386, in finalize_options
    ep.require(installer=self.fetch_build_egg)
  File "C:\Python27\lib\site-packages\pkg_resources\__init__.py", line 2318, in require
    items = working_set.resolve(reqs, env, installer, extras=self.extras)
  File "C:\Python27\lib\site-packages\pkg_resources\__init__.py", line 862, in resolve
    new_requirements = dist.requires(req.extras)[::-1]
  File "C:\Python27\lib\site-packages\pkg_resources\__init__.py", line 2562, in requires
    dm = self._dep_map
  File "C:\Python27\lib\site-packages\pkg_resources\__init__.py", line 2809, in _dep_map
    self.__dep_map = self._compute_dependencies()
  File "C:\Python27\lib\site-packages\pkg_resources\__init__.py", line 2818, in _compute_dependencies
    for req in self._parsed_pkg_info.get_all('Requires-Dist') or []:
  File "C:\Python27\lib\site-packages\pkg_resources\__init__.py", line 2800, in _parsed_pkg_info
    metadata = self.get_metadata(self.PKG_INFO)
  File "C:\Python27\lib\site-packages\pkg_resources\__init__.py", line 1468, in get_metadata
    value = self._get(self._fn(self.egg_info, name))
  File "C:\Python27\lib\site-packages\pkg_resources\__init__.py", line 1577, in _get
    with open(path, 'rb') as stream:
IOError: [Errno 2] No such file or directory: 'c:\\python27\\lib\\site-packages\\six-1.10.0.dist-info\\METADATA'

However, the file DOES exist. (the path displayed is not the correct case of the actual path though, which is C:\Python27\Lib\site-packages\six-1.10.0.dist-info\metadata

uninstalling everything and then trying the command again yields the same result. changing the command to remove the upgrade or the force reinstall does not have the same problem.

this is very weird, and the only thing that changed is the new version of setuptools

[ulope]

I’ve been hitting (what appears to be) the same bug but with appdirs instead of six

[afitzgerald]

I also just encountered this as well. Downgrading from setuptools-35.0.2 to setuptools-33.1.1 worked…