pipenv clean fails if PyGObject is installed inside the venv

Issue description

pipenv clean fails (throws exception) when executed with PyGObject installed inside the virtual environment.

<del>It is possible that there are other packages on PyPI that can trigger this behavior, but so far I haven’t found any other. I don’t know what’s different about the PyGObject package that causes this, either.</del>

Update: it looks like any package that installs data files into the include subdirectory of the virtual environment could trigger the problem.

Expected result

If the Pipfile doesn’t contain PyGObject, or any other package that depends on it, I expect pipenv clean to uninstall it.

Actual result

$ pipenv clean -v
Traceback (most recent call last):
  File "/home/dsimon/.local/bin/pipenv", line 11, in <module>
    sys.exit(cli())
  File "/home/dsimon/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/dsimon/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/dsimon/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/dsimon/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/dsimon/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/dsimon/.local/lib/python3.6/site-packages/pipenv/vendor/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/home/dsimon/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/dsimon/.local/lib/python3.6/site-packages/pipenv/vendor/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/dsimon/.local/lib/python3.6/site-packages/pipenv/cli/command.py", line 602, in clean
    do_clean(ctx=ctx, three=state.three, python=state.python, dry_run=dry_run)
  File "/home/dsimon/.local/lib/python3.6/site-packages/pipenv/core.py", line 2526, in do_clean
    canonicalize_name(pkg.project_name) for pkg in project.get_installed_packages()
  File "/home/dsimon/.local/lib/python3.6/site-packages/pipenv/project.py", line 320, in get_installed_packages
    packages = [pkg for pkg in workingset if self.dist_is_in_project(pkg)]
  File "/home/dsimon/.local/lib/python3.6/site-packages/pipenv/project.py", line 320, in <listcomp>
    packages = [pkg for pkg in workingset if self.dist_is_in_project(pkg)]
  File "/home/dsimon/.local/lib/python3.6/site-packages/pipenv/project.py", line 311, in dist_is_in_project
    prefix = _normalized(self.env_paths["prefix"])
  File "/home/dsimon/.local/lib/python3.6/site-packages/pipenv/project.py", line 951, in env_paths
    for k, v in sysconfig._INSTALL_SCHEMES[scheme].items()
  File "/home/dsimon/.local/lib/python3.6/site-packages/pipenv/project.py", line 951, in <dictcomp>
    for k, v in sysconfig._INSTALL_SCHEMES[scheme].items()
KeyError: 'py_version_short'

Steps to replicate

Edit: PyGObject’s dependencies must be installed (e.g. with the distro’s package manager) before attempting to install it with pip. On Ubuntu, these are: libcairo-dev libdbus-1-dev libgirepository1.0-dev.

# 1. create a new environment with no packages
mkdir temp && cd temp
pipenv install
# 2. install PyGObject inside the virtual environment
pipenv run pip install pygobject
# 3. try to run clean
pipenv clean

---

$ pipenv --support

Pipenv version: '2018.10.13'

Pipenv location: '/home/dsimon/.local/lib/python3.6/site-packages/pipenv'

Python location: '/usr/bin/python3'

Python installations found:

• 3.7.0: /usr/bin/python3.7m
• 3.7.0: /usr/bin/python3.7
• 3.6.6: /usr/bin/python3.6m
• 3.6.6: /usr/bin/python3.6
• 3.5.6: /usr/bin/python3.5
• 3.5.6: /usr/bin/python3.5m
• 3.4.9: /usr/bin/python3.4m
• 3.4.9: /usr/bin/python3.4
• 3.3.7: /usr/bin/python3.3
• 3.3.7: /usr/bin/python3.3m
• 2.7.15: /usr/bin/python2.7

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.6.6',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '4.18.13-200.fc28.x86_64',
 'platform_system': 'Linux',
 'platform_version': '#1 SMP Wed Oct 10 17:29:59 UTC 2018',
 'python_full_version': '3.6.6',
 'python_version': '3.6',
 'sys_platform': 'linux'}

System environment variables:

• QTINC
• LS_COLORS
• MC_SKIN
• XDG_MENU_PREFIX
• MODULES_RUN_QUARANTINE
• LANG
• GDM_LANG
• HISTCONTROL
• DISPLAY
• GUESTFISH_RESTORE
• HOSTNAME
• QTDIR
• EDITOR
• COLORTERM
• GUESTFISH_INIT
• USERNAME
• XDG_VTNR
• SSH_AUTH_SOCK
• XDG_SESSION_ID
• MODULES_CMD
• USER
• ENV
• GUESTFISH_PS1
• DESKTOP_SESSION
• WAYLAND_DISPLAY
• GNOME_TERMINAL_SCREEN
• PWD
• HOME
• XDG_SESSION_TYPE
• BASH_ENV
• XDG_DATA_DIRS
• XDG_SESSION_DESKTOP
• GJS_DEBUG_OUTPUT
• LOADEDMODULES
• MAIL
• VISUAL
• QTLIB
• TERM
• VTE_VERSION
• SHELL
• QT_IM_MODULE
• XMODIFIERS
• XDG_CURRENT_DESKTOP
• GNOME_TERMINAL_SERVICE
• SHLVL
• XDG_SEAT
• MODULEPATH
• GDMSESSION
• GNOME_DESKTOP_SESSION_ID
• LOGNAME
• DBUS_SESSION_BUS_ADDRESS
• XDG_RUNTIME_DIR
• MODULEPATH_modshare
• PATH
• GUESTFISH_OUTPUT
• MODULESHOME
• HISTSIZE
• GJS_DEBUG_TOPICS
• SESSION_MANAGER
• LESSOPEN
• BASH_FUNC_module%%
• BASH_FUNC_switchml%%
• BASH_FUNC_scl%%
• BASH_FUNC__moduleraw%%
• _
• OLDPWD
• PYTHONDONTWRITEBYTECODE
• PIP_SHIMS_BASE_MODULE
• PIP_PYTHON_PATH

Pipenv–specific environment variables:

Debug–specific environment variables:

• PATH: /opt/totalphase/datacenter/bin:/home/dsimon/bin:/home/dsimon/.local/bin:/opt/totalphase/datacenter/bin:/home/dsimon/bin:/home/dsimon/.local/bin:/usr/lib64/qt-3.3/bin:/usr/share/Modules/bin:/usr/lib64/ccache:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin
• SHELL: /bin/bash
• EDITOR: vim
• LANG: en_US.UTF-8
• PWD: /home/dsimon/temp

---

Contents of Pipfile (‘/home/dsimon/temp/Pipfile’):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]

[dev-packages]

[requires]
python_version = "3.6"

Contents of Pipfile.lock (‘/home/dsimon/temp/Pipfile.lock’):

{
    "_meta": {
        "hash": {
            "sha256": "415dfdcb118dd9bdfef17671cb7dcd78dbd69b6ae7d4f39e8b44e71d60ca72e7"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.6"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {},
    "develop": {}
}

[CognexDaniel]

Looks like what’s triggering the issue is that PyGObject installs pygobject.h as a data file into the include directory. If I create a mock package that does nothing but install something as a data file into include and install that into pipenv’s virtual environment, I can already reproduce the exception in pipenv clean. E.g. setup.py:

from setuptools import setup

setup(
    name='repro',
    version='0.0.1',
    description='Reproducer for exception in pipenv clean',
    data_files=[('include', ['repro.h'])]
)

(You need to create repro.h in the same dir; its contents don’t matter).

[techalchemy]

Ugh, that is super annoying. That will need to go into like 3 different projects 😦 I would definitely support a PR for this, your change looks good to me. Thanks for digging into this @CognexDaniel