Missing setuptools.egg-info crashes pip

This has crashed deploys on at least 9 of our production servers in the last week. I’ve been struggling to create a reproducable use case but can’t, sometimes it’s fine, other times it really isn’t. Reproducable test case provided here. Any insight would be hugely appreciated!

Also worth noting that we’ve seen the same issue when running pip install --upgrade virtualenv so probably not package related?

$ pip install  --upgrade supervisor
Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/pip/basecommand.py", line 232, in main
    status = self.run(options, args)
  File "/usr/local/lib/python2.7/dist-packages/pip/commands/install.py", line 339, in run
    requirement_set.prepare_files(finder)
  File "/usr/local/lib/python2.7/dist-packages/pip/req/req_set.py", line 229, in prepare_files
    req_to_install.check_if_exists()
  File "/usr/local/lib/python2.7/dist-packages/pip/req/req_install.py", line 931, in check_if_exists
    self.satisfied_by = pkg_resources.get_distribution(self.req)
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 461, in get_distribution
    dist = get_provider(dist)
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 341, in get_provider
    return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 870, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 740, in resolve
    env = Environment(self.entries)
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 927, in __init__
    self.scan(search_path)
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 957, in scan
    self.add(dist)
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 973, in add
    if self.can_add(dist) and dist.has_version():
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 2658, in has_version
    self.version
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 2466, in version
    for line in self._get_metadata(self.PKG_INFO):
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 2510, in _get_metadata
    for line in self.get_metadata_lines(name):
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 1927, in get_metadata_lines
    return yield_lines(self.get_metadata(name))
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 1921, in get_metadata
    with open(self.path,'rU') as f:
IOError: [Errno 2] No such file or directory: '/usr/lib/python2.7/dist-packages/setuptools.egg-info'

$ echo $?
2

[mal]

After further investigation we’ve managed to consistently recreate this on a clean ubuntu 12.04 box like so (run as root):

#!/bin/sh

# Installs 0.6rc11 via 0.6.24-1ubuntu1
apt-get install -y python-setuptools

# Here we have some easy_install usage before our deploy logic
# decides if we need to execute the rest of the code below.

wget -O- https://bootstrap.pypa.io/get-pip.py | python
pip install --upgrade setuptools
pip install --upgrade virtualenv

[qwcode]

closing based on https://github.com/pypa/pip/issues/2438#issuecomment-87450508

if anyone wants to double confirm the develop branch fixes this (or doesn’t), please do, and post results.