wonky pth file breaks upgrade

nstalling collected packages: pbr, pip
  Found existing installation: pbr 0.10.0
Cleaning up...
  Removing temporary dir /tmp/pip_build_root...
Cannot remove entries from nonexistent file /home/robertc/work/openstack/oslo.config/easy-install.pth
Exception information:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/usr/local/lib/python2.7/dist-packages/pip/commands/install.py", line 283, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/usr/local/lib/python2.7/dist-packages/pip/req.py", line 1431, in install
    requirement.uninstall(auto_confirm=True)
  File "/usr/local/lib/python2.7/dist-packages/pip/req.py", line 555, in uninstall
    paths_to_remove.add_pth(easy_install_pth, './' + easy_install_egg)
  File "/usr/local/lib/python2.7/dist-packages/pip/req.py", line 1785, in add_pth
    self.pth[pth_file] = UninstallPthEntries(pth_file)
  File "/usr/local/lib/python2.7/dist-packages/pip/req.py", line 1868, in __init__
    raise UninstallationError("Cannot remove entries from nonexistent file %s" % pth_file)
UninstallationError: Cannot remove entries from nonexistent file /home/robertc/work/openstack/oslo.config/easy-install.pth

[stxlvt]

I tried filing this as an anaconda bug, but @ilanschnell closed it as a setuptools/pip issue. I’m including a summary of the discussion here:

Setup

D:\workspace>conda info
Current conda install:

             platform : win-32
        conda version : 3.18.5
  conda-build version : 1.14.1
       python version : 2.7.10.final.0
     requests version : 2.8.1
     root environment : D:\anaconda32  (writable)
  default environment : D:\anaconda32
     envs directories : D:\anaconda32\envs
        package cache : D:\anaconda32\pkgs
         channel URLs : https://repo.continuum.io/pkgs/free/win-32/
                        https://repo.continuum.io/pkgs/free/noarch/
                        https://repo.continuum.io/pkgs/pro/win-32/
                        https://repo.continuum.io/pkgs/pro/noarch/
          config file : None
    is foreign system : False

D:\workspace>conda create --name tst python
Fetching package metadata: ....
Solving package specifications: ............
Package plan for installation in environment D:\anaconda32\envs\tst:

The following NEW packages will be INSTALLED:

    msvc_runtime: 1.0.1-vc9_0   [vc9]
    pip:          7.1.2-py27_0
    python:       2.7.10-4
    setuptools:   18.4-py27_0
    wheel:        0.26.0-py27_1

Proceed ([y]/n)? y

Linking packages ...
[      COMPLETE      ]|##################################################| 100%
#
# To activate this environment, use:
# > activate tst
#

D:\workspace>activate tst
Activating environment "D:\anaconda32\envs\tst"...

The bug

[tst] D:\workspace>pip install -U setuptools
Collecting setuptools
  Downloading setuptools-18.5-py2.py3-none-any.whl (462kB)
    100% |################################| 462kB 787kB/s
Installing collected packages: setuptools
  Found existing installation: setuptools 18.4
Cannot remove entries from nonexistent file d:\anaconda32\envs\tst\lib\site-pack
ages\easy-install.pth

Trying to install -U any package depending on setuptools also fails with the same error.

Known workarounds

Adding the --ignore-installed option will sucessfully upgrade setuptools and the problem will be solved for the lifetime of that particular conda environment. Downloading and running ez_setup.py also fixes the problem for that conda environment. The second workaround will create the missing pth file, while the first will not, but in both cases everything seems to work afterwards.

While using conda to upgrade setuptools is also a workaround, it is not a good solution, since a lot of packages on pypi (1822 in 2013) depend on setuptools and therefore try to upgrade setuptools too, when upgraded:

[tst] D:\workspace>pip install -U zc.buildout
Collecting zc.buildout
  Downloading http://python/root/pypi/+f/739/582d22e3ddd5e/zc.buildout-2.5.0-py2
.py3-none-any.whl (261kB)
    100% |################################| 262kB 6.8MB/s
Collecting setuptools>=8.0 (from zc.buildout)
  Downloading http://python/root/pypi/+f/d40/182384798e286/setuptools-18.8.1-py2
.py3-none-any.whl (463kB)
    100% |################################| 466kB 10.2MB/s
Installing collected packages: setuptools, zc.buildout
  Found existing installation: setuptools 18.5
Cannot remove entries from nonexistent file d:\anaconda32\envs\tst\lib\site-pack
ages\easy-install.pth

A lot of pypi packages are not installable using conda install, and so using conda to upgrade these is not an option.

None of these workarounds are good permanent solutions to this bug because:

1. They will trip a lot of users who do not know that adding -I fixes the problem and will have to spend a considerable amount of time searching for some solution.
2. They break automated testing (e.g. tox)
3. They do not fix the bug permanently on that PC, but just work for the lifetime of the environment, which is no good if the environment is a throwaway environment.

Cause

The error message is correct, there is no easy-install.pth file in site-packages. The error occurs when pip tries to uninstall the old setuptools package, in pip/req/req_uninstall.py:160. The file pip is trying to remove (easy-install.pth) does not exist in a fresh Anaconda/Miniconda installation because no conda packages are able to include this file.

This is a periodically sleeping bug, since it rears its head whenever the conda setuptools version (currently 18.8.1) lags behind the pip setuptools version (currently 19.1.1).

[rbtcollins]

The assumption of easy-install.pth file is guarded by a check for .egg:

        elif dist.location.endswith('.egg'):
            # package installed by easy_install
            # We cannot match on dist.egg_name because it can slightly vary
            # i.e. setuptools-0.6c11-py2.6.egg vs setuptools-0.6rc11-py2.6.egg
            paths_to_remove.add(dist.location)
            easy_install_egg = os.path.split(dist.location)[1]
            easy_install_pth = os.path.join(os.path.dirname(dist.location),
                                            'easy-install.pth')
            paths_to_remove.add_pth(easy_install_pth, './' + easy_install_egg)

So my guess is that your conda package looks like an easy_install egg: this may confuse other tools as well. Perhaps don’t do that?