Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

pipenv 2020.8.13 failing to install some package dependencies

See original GitHub issue

Issue description

installing packages with pipenv 2020.8.13 is failing to install all package dependencies, or installing them in such a way that I can’t tell they’re installed. ¯\_(ツ)_/¯

specifically I’m seeing it having issues installing six for some strange reason, which may very well have something to do with my invocation of pipenv, so please refer to the “Steps to replicate” section for a detailed reproduction example.

Expected result

I expect all packages and dependencies to be installed.

example output after installing with pipenv 2020.6.2, which works as expected:

$ python -m pip list
Package        Version
-------------- -------
attrs          19.3.0
iniconfig      1.0.1
more-itertools 8.4.0
packaging      20.4
pip            20.2.2
pluggy         0.13.1
py             1.9.0
pyparsing      2.4.7
pytest         6.0.1
setuptools     49.3.1
six            1.15.0
toml           0.10.1
wheel          0.34.2

Actual result

six is clearly missing and understandably fails when I attempt to import it.

example output after installing with pipenv 2020.8.13, which show six doesn’t seem to be installed:

$ python -m pip list
Package        Version
-------------- -------
attrs          19.3.0
iniconfig      1.0.1
more-itertools 8.4.0
packaging      20.4
pip            20.2.2
pluggy         0.13.1
py             1.9.0
pyparsing      2.4.7
pytest         6.0.1
setuptools     49.3.1
toml           0.10.1
wheel          0.34.2

Steps to replicate

given the Pipfile and Pipfile.lock from the pipenv --support details below and the following Dockerfile:

FROM python:3.8-alpine as build-old

ENV PYTHONUSERBASE /pyvenv_old
WORKDIR /build

COPY Pipfile Pipfile.lock ./

RUN set -x && \
    python -m pip install pipenv==2020.6.2 && \
    PIP_USER=1 \
    PIP_IGNORE_INSTALLED=1 \
    pipenv install --system --deploy


FROM python:3.8-alpine as build-new

ENV PYTHONUSERBASE /pyvenv_new
WORKDIR /build

COPY Pipfile Pipfile.lock ./

RUN set -x && \
    python -m pip install pipenv==2020.8.13 && \
    PIP_USER=1 \
    PIP_IGNORE_INSTALLED=1 \
    pipenv install --system --deploy


FROM python:3.8-alpine

COPY --from=build-old /pyvenv_old /pyvenv_old
COPY --from=build-new /pyvenv_new /pyvenv_new

RUN echo "# 2020.6.2 packages" > pipenv_installed_packages && \
    PYTHONUSERBASE=/pyvenv_old python -m pip list >> pipenv_installed_packages && \
    echo -e "\n# 2020.8.13 packages" >> pipenv_installed_packages && \
    PYTHONUSERBASE=/pyvenv_new python -m pip list >> pipenv_installed_packages

CMD ["cat", "pipenv_installed_packages"]

if you place all three files in the same dir you can verify what I’m seeing with these commands:

$ docker build -t pipenv_six_bug .
<snip build output>

$ docker run --rm pipenv_six_bug
# 2020.6.2 packages
Package        Version
-------------- -------
attrs          19.3.0
iniconfig      1.0.1
more-itertools 8.4.0
packaging      20.4
pip            20.2.2
pluggy         0.13.1
py             1.9.0
pyparsing      2.4.7
pytest         6.0.1
setuptools     49.3.1
six            1.15.0
toml           0.10.1
wheel          0.34.2

# 2020.8.13 packages
Package        Version
-------------- -------
attrs          19.3.0
iniconfig      1.0.1
more-itertools 8.4.0
packaging      20.4
pip            20.2.2
pluggy         0.13.1
py             1.9.0
pyparsing      2.4.7
pytest         6.0.1
setuptools     49.3.1
toml           0.10.1
wheel          0.34.2

obviously simplified for repro, this is a common pattern we use at my company to pare down the final image and only use pipenv for package installation.

$ pipenv --support for 2020.6.2

Pipenv version: '2020.6.2'

Pipenv location: '/usr/local/lib/python3.8/site-packages/pipenv'

Python location: '/usr/local/bin/python'

Python installations found:

  • 3.8.5: /usr/local/bin/python3
  • 3.8.5: /usr/local/bin/python3.8

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.8.5',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '5.4.0-42-generic',
 'platform_system': 'Linux',
 'platform_version': '#46~18.04.1-Ubuntu SMP Fri Jul 10 07:21:24 UTC 2020',
 'python_full_version': '3.8.5',
 'python_version': '3.8',
 'sys_platform': 'linux'}

System environment variables:

  • HOSTNAME
  • PYTHON_PIP_VERSION
  • SHLVL
  • HOME
  • GPG_KEY
  • PYTHON_GET_PIP_URL
  • PATH
  • LANG
  • PYTHONUSERBASE
  • PYTHON_VERSION
  • PWD
  • PYTHON_GET_PIP_SHA256
  • PIP_DISABLE_PIP_VERSION_CHECK
  • PYTHONDONTWRITEBYTECODE
  • PIP_SHIMS_BASE_MODULE
  • PIP_PYTHON_PATH
  • PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

Debug–specific environment variables:

  • PATH: /usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
  • LANG: C.UTF-8
  • PWD: /build

Contents of Pipfile (‘/build/Pipfile’):

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[packages]
pytest = "==6.0.1"

[requires]
python_version = "3.8"

Contents of Pipfile.lock (‘/build/Pipfile.lock’):

{
    "_meta": {
        "hash": {
            "sha256": "224c1f1c10cc0c59f97d80c3a38b2d19dffd6763a9da042cec329a0abda08b99"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.8"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "attrs": {
            "hashes": [
                "sha256:08a96c641c3a74e44eb59afb61a24f2cb9f4d7188748e76ba4bb5edfa3cb7d1c",
                "sha256:f7b7ce16570fe9965acd6d30101a28f62fb4a7f9e926b3bbc9b61f8b04247e72"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
            "version": "==19.3.0"
        },
        "iniconfig": {
            "hashes": [
                "sha256:80cf40c597eb564e86346103f609d74efce0f6b4d4f30ec8ce9e2c26411ba437",
                "sha256:e5f92f89355a67de0595932a6c6c02ab4afddc6fcdc0bfc5becd0d60884d3f69"
            ],
            "version": "==1.0.1"
        },
        "more-itertools": {
            "hashes": [
                "sha256:68c70cc7167bdf5c7c9d8f6954a7837089c6a36bf565383919bb595efb8a17e5",
                "sha256:b78134b2063dd214000685165d81c154522c3ee0a1c0d4d113c80361c234c5a2"
            ],
            "markers": "python_version >= '3.5'",
            "version": "==8.4.0"
        },
        "packaging": {
            "hashes": [
                "sha256:4357f74f47b9c12db93624a82154e9b120fa8293699949152b22065d556079f8",
                "sha256:998416ba6962ae7fbd6596850b80e17859a5753ba17c32284f67bfff33784181"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
            "version": "==20.4"
        },
        "pluggy": {
            "hashes": [
                "sha256:15b2acde666561e1298d71b523007ed7364de07029219b604cf808bfa1c765b0",
                "sha256:966c145cd83c96502c3c3868f50408687b38434af77734af1e9ca461a4081d2d"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
            "version": "==0.13.1"
        },
        "py": {
            "hashes": [
                "sha256:366389d1db726cd2fcfc79732e75410e5fe4d31db13692115529d34069a043c2",
                "sha256:9ca6883ce56b4e8da7e79ac18787889fa5206c79dcc67fb065376cd2fe03f342"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
            "version": "==1.9.0"
        },
        "pyparsing": {
            "hashes": [
                "sha256:c203ec8783bf771a155b207279b9bccb8dea02d8f0c9e5f8ead507bc3246ecc1",
                "sha256:ef9d7589ef3c200abe66653d3f1ab1033c3c419ae9b9bdb1240a85b024efc88b"
            ],
            "markers": "python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3'",
            "version": "==2.4.7"
        },
        "pytest": {
            "hashes": [
                "sha256:85228d75db9f45e06e57ef9bf4429267f81ac7c0d742cc9ed63d09886a9fe6f4",
                "sha256:8b6007800c53fdacd5a5c192203f4e531eb2a1540ad9c752e052ec0f7143dbad"
            ],
            "index": "pypi",
            "version": "==6.0.1"
        },
        "six": {
            "hashes": [
                "sha256:30639c035cdb23534cd4aa2dd52c3bf48f06e5f4a941509c8bafd8ce11080259",
                "sha256:8b74bedcbbbaca38ff6d7491d76f2b06b3592611af620f8426e82dddb04a5ced"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
            "version": "==1.15.0"
        },
        "toml": {
            "hashes": [
                "sha256:926b612be1e5ce0634a2ca03470f95169cf16f939018233a670519cb4ac58b0f",
                "sha256:bda89d5935c2eac546d648028b9901107a595863cb36bae0c73ac804a9b4ce88"
            ],
            "version": "==0.10.1"
        }
    },
    "develop": {}
}
$ pipenv --support for 2020.8.13

Pipenv version: '2020.8.13'

Pipenv location: '/usr/local/lib/python3.8/site-packages/pipenv'

Python location: '/usr/local/bin/python'

Python installations found:

  • 3.8.5: /usr/local/bin/python3
  • 3.8.5: /usr/local/bin/python3.8

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.8.5',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '5.4.0-42-generic',
 'platform_system': 'Linux',
 'platform_version': '#46~18.04.1-Ubuntu SMP Fri Jul 10 07:21:24 UTC 2020',
 'python_full_version': '3.8.5',
 'python_version': '3.8',
 'sys_platform': 'linux'}

System environment variables:

  • HOSTNAME
  • PYTHON_PIP_VERSION
  • SHLVL
  • HOME
  • GPG_KEY
  • PYTHON_GET_PIP_URL
  • PATH
  • LANG
  • PYTHONUSERBASE
  • PYTHON_VERSION
  • PWD
  • PYTHON_GET_PIP_SHA256
  • PIP_DISABLE_PIP_VERSION_CHECK
  • PYTHONDONTWRITEBYTECODE
  • PIP_SHIMS_BASE_MODULE
  • PIP_PYTHON_PATH
  • PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

Debug–specific environment variables:

  • PATH: /usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
  • LANG: C.UTF-8
  • PWD: /build

Contents of Pipfile (‘/build/Pipfile’):

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[packages]
pytest = "==6.0.1"

[requires]
python_version = "3.8"

Contents of Pipfile.lock (‘/build/Pipfile.lock’):

{
    "_meta": {
        "hash": {
            "sha256": "224c1f1c10cc0c59f97d80c3a38b2d19dffd6763a9da042cec329a0abda08b99"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.8"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "attrs": {
            "hashes": [
                "sha256:08a96c641c3a74e44eb59afb61a24f2cb9f4d7188748e76ba4bb5edfa3cb7d1c",
                "sha256:f7b7ce16570fe9965acd6d30101a28f62fb4a7f9e926b3bbc9b61f8b04247e72"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
            "version": "==19.3.0"
        },
        "iniconfig": {
            "hashes": [
                "sha256:80cf40c597eb564e86346103f609d74efce0f6b4d4f30ec8ce9e2c26411ba437",
                "sha256:e5f92f89355a67de0595932a6c6c02ab4afddc6fcdc0bfc5becd0d60884d3f69"
            ],
            "version": "==1.0.1"
        },
        "more-itertools": {
            "hashes": [
                "sha256:68c70cc7167bdf5c7c9d8f6954a7837089c6a36bf565383919bb595efb8a17e5",
                "sha256:b78134b2063dd214000685165d81c154522c3ee0a1c0d4d113c80361c234c5a2"
            ],
            "markers": "python_version >= '3.5'",
            "version": "==8.4.0"
        },
        "packaging": {
            "hashes": [
                "sha256:4357f74f47b9c12db93624a82154e9b120fa8293699949152b22065d556079f8",
                "sha256:998416ba6962ae7fbd6596850b80e17859a5753ba17c32284f67bfff33784181"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
            "version": "==20.4"
        },
        "pluggy": {
            "hashes": [
                "sha256:15b2acde666561e1298d71b523007ed7364de07029219b604cf808bfa1c765b0",
                "sha256:966c145cd83c96502c3c3868f50408687b38434af77734af1e9ca461a4081d2d"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
            "version": "==0.13.1"
        },
        "py": {
            "hashes": [
                "sha256:366389d1db726cd2fcfc79732e75410e5fe4d31db13692115529d34069a043c2",
                "sha256:9ca6883ce56b4e8da7e79ac18787889fa5206c79dcc67fb065376cd2fe03f342"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
            "version": "==1.9.0"
        },
        "pyparsing": {
            "hashes": [
                "sha256:c203ec8783bf771a155b207279b9bccb8dea02d8f0c9e5f8ead507bc3246ecc1",
                "sha256:ef9d7589ef3c200abe66653d3f1ab1033c3c419ae9b9bdb1240a85b024efc88b"
            ],
            "markers": "python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3'",
            "version": "==2.4.7"
        },
        "pytest": {
            "hashes": [
                "sha256:85228d75db9f45e06e57ef9bf4429267f81ac7c0d742cc9ed63d09886a9fe6f4",
                "sha256:8b6007800c53fdacd5a5c192203f4e531eb2a1540ad9c752e052ec0f7143dbad"
            ],
            "index": "pypi",
            "version": "==6.0.1"
        },
        "six": {
            "hashes": [
                "sha256:30639c035cdb23534cd4aa2dd52c3bf48f06e5f4a941509c8bafd8ce11080259",
                "sha256:8b74bedcbbbaca38ff6d7491d76f2b06b3592611af620f8426e82dddb04a5ced"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
            "version": "==1.15.0"
        },
        "toml": {
            "hashes": [
                "sha256:926b612be1e5ce0634a2ca03470f95169cf16f939018233a670519cb4ac58b0f",
                "sha256:bda89d5935c2eac546d648028b9901107a595863cb36bae0c73ac804a9b4ce88"
            ],
            "version": "==0.10.1"
        }
    },
    "develop": {}
}

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:7 (1 by maintainers)

github_iconTop GitHub Comments

2reactions
youngbobcommented, Aug 20, 2020

Here’s my solution using the venv method for a multi-stage build:

FROM python:3.8-alpine as build

WORKDIR /build

COPY Pipfile Pipfile.lock ./

RUN set -x && \
    python -m pip install pipenv==2020.8.13 && \
    PIPENV_VENV_IN_PROJECT=1 \
    pipenv install --deploy


FROM python:3.8-alpine

COPY --from=build /build /build
ENV PATH /build/.venv/bin:$PATH

# ...

Using PIPENV_VENV_IN_PROJECT=1 puts the venv in a known/stable path that you can copy to the second stage, and adding .venv/bin to your PATH makes any later commands running python use the venv.

2reactions
joshsleepercommented, Aug 19, 2020

@frostming I can understand where you’re coming from, although I guess that leaves me in a weird place for the moment.

how about this: if we wanted to use pipenv to handle the install with intent to copy that single folder into a second docker build stage, do you have any suggestions or thoughts for how that could be done?

since pipenv isn’t used for the shipped application, it seems like there should be a reasonable way to use it only for the install portion of the build without including it in the final image.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Why is pipenv failing to install dependencies in production ...
I have tried to run it outside the virtualenv but can't seem to get the correct version of pipenv I have version 2021.5.29...
Read more >
Advanced Usage of Pipenv - Python Packaging Authority
This will fail a build if the Pipfile.lock is out–of–date, instead of generating a new one. Or you can install packages exactly as...
Read more >
Release and Version History — pipenv 2020.8.13.dev0 文档
#4274; Fixed a bug which caused Pipenv to fail to install packages in a virtual ... and pipenv sync will no longer attempt...
Read more >
pipenv Documentation - Read the Docs
Will create a virtual env and install dependencies (if it does not exist already) ... Fix for pipenv lock fails for packages with...
Read more >
30371 (Use pipenv to create venv for development) - Sage Trac
Moreover, the installation of some packages might fail (depending on your system ... I'm using pipenv from homebrew, which is version 2020.8.13 ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

Generating requirements.txt using invoke produces a trailing escape character

Next page

"Your Pipfile requires python_version 3.8, but you are using 2.7.16" But VM is made with 3.8.3