pip install --upgrade stalls at the "uninstalling" phase

Environment

• pip version: 19.0
• Python version: 2.7.15rc1
• OS: Ubuntu 18.04.1 LTS (Kernel: 4.15.0-1031-aws)

Description

Running pip install --upgrade for any package that needs upgrading will cause pip to stall at the “Uninstalling” stage.

Expected behavior

Packages should be upgraded.

How to Reproduce

1. On a system with cryptography-2.4.2 installed…
2. Run pip install --upgrade cryptography
3. Pip gets as far as Uninstalling cryptography-2.4.2 but doesn’t go any further. -vvv does not provide any extra output.

Output

  Using version 2.5 (newest of versions: 0.1, 0.2, 0.2.1, 0.2.2, 0.3, 0.4, 0.5, 0.5.1, 0.5.2, 0.5.3, 0.5.4, 0.6, 0.6.1, 0.7, 0.7.1, 0.7.2, 0.8, 0.8.1, 0.8.2, 0.9, 0.9.1, 0.9.2, 0.9.3, 1.0, 1.0.1, 1.0.2, 1.1, 1.1.1, 1.1.2, 1.2, 1.2.1, 1.2.2, 1.2.3, 1.3, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.4, 1.5, 1.5.1, 1.5.2, 1.5.3, 1.6, 1.7, 1.7.1, 1.7.2, 1.8, 1.8.1, 1.8.2, 1.9, 2.0, 2.0.1, 2.0.2, 2.0.3, 2.1, 2.1.1, 2.1.2, 2.1.3, 2.1.4, 2.2, 2.2.1, 2.2.2, 2.3, 2.3.1, 2.4, 2.4.1, 2.4.2, 2.5)
  Created temporary directory: /tmp/pip-unpack-RPH06y
  Looking up "https://files.pythonhosted.org/packages/17/fd/4c2c8953a9dfe38fbe0c3adafb6355540bd98cef70cc82734acb0a4c0e2f/cryptography-2.5-cp27-cp27mu-manylinux1_x86_64.whl" in the cache
  Current age based on date: 5543
  Ignoring unknown cache-control directive: immutable
  Freshness lifetime from max-age: 365000000
  The response is "fresh", returning cached response
  365000000 > 5543
  Using cached https://files.pythonhosted.org/packages/17/fd/4c2c8953a9dfe38fbe0c3adafb6355540bd98cef70cc82734acb0a4c0e2f/cryptography-2.5-cp27-cp27mu-manylinux1_x86_64.whl
  Downloading from URL https://files.pythonhosted.org/packages/17/fd/4c2c8953a9dfe38fbe0c3adafb6355540bd98cef70cc82734acb0a4c0e2f/cryptography-2.5-cp27-cp27mu-manylinux1_x86_64.whl#sha256=42fad67d7072216a49e34f923d8cbda9edacbf6633b19a79655e88a1b4857063 (from https://pypi.org/simple/cryptography/) (requires-python:>=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*)
  Added cryptography from https://files.pythonhosted.org/packages/17/fd/4c2c8953a9dfe38fbe0c3adafb6355540bd98cef70cc82734acb0a4c0e2f/cryptography-2.5-cp27-cp27mu-manylinux1_x86_64.whl#sha256=42fad67d7072216a49e34f923d8cbda9edacbf6633b19a79655e88a1b4857063 to build tracker '/tmp/pip-req-tracker-gljwal'
  Removed cryptography from https://files.pythonhosted.org/packages/17/fd/4c2c8953a9dfe38fbe0c3adafb6355540bd98cef70cc82734acb0a4c0e2f/cryptography-2.5-cp27-cp27mu-manylinux1_x86_64.whl#sha256=42fad67d7072216a49e34f923d8cbda9edacbf6633b19a79655e88a1b4857063 from build tracker '/tmp/pip-req-tracker-gljwal'
Requirement already satisfied, skipping upgrade: asn1crypto>=0.21.0 in /usr/lib/python2.7/dist-packages (from cryptography) (0.24.0)
Requirement already satisfied, skipping upgrade: enum34; python_version < "3" in /usr/lib/python2.7/dist-packages (from cryptography) (1.1.6)
Requirement already satisfied, skipping upgrade: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography) (1.0.22)
Requirement already satisfied, skipping upgrade: cffi!=1.11.3,>=1.8 in /usr/local/lib/python2.7/dist-packages (from cryptography) (1.11.5)
Requirement already satisfied, skipping upgrade: six>=1.4.1 in /usr/lib/python2.7/dist-packages (from cryptography) (1.11.0)
Requirement already satisfied, skipping upgrade: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi!=1.11.3,>=1.8->cryptography) (2.19)
Installing collected packages: cryptography
  Found existing installation: cryptography 2.4.2
    Uninstalling cryptography-2.4.2:

There is more output but it’s mostly “Found link” and “Skipping link”. Let me know if you need the full output.

[zooba]

@cjerdonek That sounds like a reasonable fix. I’ll take a quick look

[YannickJadoul]

We’re having the same problem over at the cibuildwheel project when running this on Travis: https://github.com/joerick/cibuildwheel/issues/122

However, I think I’ve figured out what causes this:

It seems https://github.com/pypa/pip/commit/c7ae06c79827c5d4aaa4f4b4656dfc525e5e55bc added a new way of generating temporary directories to copy uninstalled packages so that they can be rolled back if the new package’s installation failed.

Notice the _generate_names method, generating potentially len(LEADING_CHARS)**len(name) (i.e., almost always a ridiculously enormous number) of temporary folder names. When the parent folder cannot be written to (because of permission issues, for example), the creation of these temporary folders will fail and the loop will go on checking all possibilities and never managing to find one.

See https://travis-ci.org/YannickJadoul/cibuildwheel/jobs/485146671 and https://api.travis-ci.org/v3/job/485146671/log.txt for a log of the installation and the permissions: user travis is a member of the admin group, so the group permissions are what you’re looking for.

Not sure what’s the best way to fix this, but I’m quite confident this is the problem.

I see e.g. /Library/Python/2.7/site-packages/-onfigparser-3.5.1.dist-info/, config -> -onfig ?!

This new method of generating temporary folder names also explains the weird - names. Probably leftovers from a failed/interrupted installations/updates? Cfr. #6194, as well.