Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Environment markers in dependencies should be merged, not overriden by each other

See original GitHub issue

Issue description

Pipenv.lock file might contain different markers after pipenv lock.

This issue is quite annoying when you deal with a Pipfile that contains a lot of requirements especially when most of them are outdated.

Expected result

Markers remain unchanged after pipenv lock if nothing has been changed in Pipfile and on the PyPI.

Actual result

Markers change unpredictably.

Steps to replicate

You can use this script to reproduce the bug. Please see the asciicast for the details.

Environment

I experience this issue

on latest Gentoo with latest pipenv from PyPI and from master of this repo
on Ubuntu 16.04 with latest pipenv from PyPI

$ pipenv --support # Gentoo

Pipenv version: '2018.7.1.dev0'

Pipenv location: '/home/mim/src/pipenv/pipenv'

Python location: '/usr/bin/python3.7'

Other Python installations in PATH:

2.7: /usr/bin/python2.7
2.7: /usr/bin/python2.7
3.6: /usr/bin/python3.6m
3.6: /usr/bin/python3.6
3.7: /usr/bin/python3.7m
3.7: /usr/bin/python3.7
3.7.0: /usr/bin/python
2.7.15: /usr/bin/python2
3.7.0: /usr/bin/python3

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.7.0',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '4.17.10-gentoo',
 'platform_system': 'Linux',
 'platform_version': '#1 SMP Wed Jul 25 23:37:33 +03 2018',
 'python_full_version': '3.7.0',
 'python_version': '3.7',
 'sys_platform': 'linux'}

System environment variables:

LC_ALL
LS_COLORS
STY
ANDROID_HOME
XDG_MENU_PREFIX
LANG
LESS
DISPLAY
OPENGL_PROFILE
CONFIG_PROTECT_MASK
EDITOR
GPG_TTY
COLORTERM
ASCIINEMA_REC
JAVA_HOME
GCC_SPECS
SSH_AUTH_SOCK
GLADE_CATALOG_PATH
VBOX_APP_HOME
HUSHLOGIN
USER
GLADE_MODULE_PATH
PAGER
DESKTOP_SESSION
PWD
HOME
MANPAGER
WINDOW
SSH_AGENT_PID
GSETTINGS_BACKEND
XDG_DATA_DIRS
JDK_HOME
GLADE_PIXMAP_PATH
GTK_MODULES
MAIL
WINDOWPATH
CONFIG_PROTECT
SHELL
TERM
VTE_VERSION
JAVAC
XDG_CURRENT_DESKTOP
OPENCL_PROFILE
ANDROID_SWT
MOZ_GMP_PATH
SHLVL
LANGUAGE
MANPATH
WINDOWID
LOGNAME
DBUS_SESSION_BUS_ADDRESS
XSESSION
XAUTHORITY
ANT_HOME
XDG_CONFIG_DIRS
PATH
INFOPATH
TERMCAP
SESSION_MANAGER
LESSOPEN
_
PYTHONDONTWRITEBYTECODE
PIP_PYTHON_PATH

Pipenv–specific environment variables:

Debug–specific environment variables:

PATH: /home/mim/.local/bin:/home/mim/.local/bin:/usr/x86_64-pc-linux-gnu/gcc-bin/8.1.0:/usr/lib/llvm/6/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/opt/android-sdk-update-manager/tools:/opt/android-sdk-update-manager/platform-tools:/home/mim/.npm-global/bin:/home/mim/.npm-global/bin:
SHELL: /bin/bash
EDITOR: /bin/nano
LANG: ru_RU.utf8
PWD: /home/mim

$ pipenv --support # Ubuntu 16.04

Pipenv version: '2018.7.1'

Pipenv location: '/home/mim/.local/lib/python3.6/site-packages/pipenv'

Python location: '/usr/bin/python3.6'

Other Python installations in PATH:

2.7: /usr/bin/python2.7
2.7: /usr/bin/python2.7
3.5: /usr/bin/python3.5m
3.5: /usr/bin/python3.5
3.6: /usr/bin/python3.6m
3.6: /usr/bin/python3.6
2.7.12: /usr/bin/python
2.7.12: /usr/bin/python2
3.5.2: /usr/bin/python3

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.6.6',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '4.13.0-45-generic',
 'platform_system': 'Linux',
 'platform_version': '#50~16.04.1-Ubuntu SMP Wed May 30 11:18:27 UTC 2018',
 'python_full_version': '3.6.6',
 'python_version': '3.6',
 'sys_platform': 'linux'}

System environment variables:

XDG_VTNR
LC_PAPER
XDG_SESSION_ID
LC_ADDRESS
XDG_GREETER_DATA_DIR
LC_MONETARY
CLUTTER_IM_MODULE
SESSION
GPG_AGENT_INFO
XDG_MENU_PREFIX
TERM
SHELL
QT_LINUX_ACCESSIBILITY_ALWAYS_ON
WINDOWID
LC_NUMERIC
UPSTART_SESSION
GNOME_KEYRING_CONTROL
GTK_MODULES
USER
LS_COLORS
QT_ACCESSIBILITY
LC_TELEPHONE
XDG_SESSION_PATH
UNITY_HAS_3D_SUPPORT
XDG_SEAT_PATH
SSH_AUTH_SOCK
SESSION_MANAGER
DEFAULTS_PATH
XDG_CONFIG_DIRS
UNITY_DEFAULT_PROFILE
PATH
DESKTOP_SESSION
QT_QPA_PLATFORMTHEME
QT_IM_MODULE
LC_IDENTIFICATION
__fish_sysconfdir
XDG_SESSION_TYPE
PWD
JOB
XMODIFIERS
__fish_datadir
LANG
GNOME_KEYRING_PID
MANDATORY_PATH
GDM_LANG
LC_MEASUREMENT
IM_CONFIG_PHASE
COMPIZ_CONFIG_PROFILE
PAPERSIZE
GDMSESSION
SESSIONTYPE
GTK2_MODULES
XDG_SEAT
SHLVL
HOME
LANGUAGE
GNOME_DESKTOP_SESSION_ID
UPSTART_INSTANCE
__fish_help_dir
XDG_SESSION_DESKTOP
UPSTART_EVENTS
LOGNAME
COMPIZ_BIN_PATH
XDG_DATA_DIRS
QT4_IM_MODULE
DBUS_SESSION_BUS_ADDRESS
LESSOPEN
UPSTART_JOB
INSTANCE
XDG_RUNTIME_DIR
DISPLAY
XDG_CURRENT_DESKTOP
GTK_IM_MODULE
LESSCLOSE
LC_TIME
__fish_bin_dir
XAUTHORITY
LC_NAME
COLORTERM
_
PYTHONDONTWRITEBYTECODE
PIP_PYTHON_PATH

Pipenv–specific environment variables:

Debug–specific environment variables:

PATH: /home/mim/bin:/home/mim/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/mim/.npm-global/bin
SHELL: /bin/bash
LANG: en_US.UTF-8
PWD: /home/mim

Issue Analytics

State:
Created 5 years ago
Reactions:2
Comments:13 (9 by maintainers)

Top GitHub Comments

1reaction

nickmyattcommented, Apr 5, 2019

I am experiencing the same issue (fluctuating python_version marker) when installing backports.functools-lru-cache.

Does a given Pipfile.lock actually support mutliple python_versions anyway? Quoting https://github.com/pypa/pipenv/issues/2683#issuecomment-409330037

Hi, this has been proposed multiple times (the most informational thread is #1050, I believe), and the resolution is to refrain from doing it, since Pipenv wants to focus on the application aspect of Python development. In an application context, and it is wildly considered best practice to support only one Python minor version in a Python application.

We schedule pipenv update to scan for new dependencies, and generate merge requests if Pipfile.lock has changed. Nondeterministic environment markers is obviously a problem for such a workflow as it generates spurious MRs. My current solution to the problem is

sed -i '/"markers": "python_version/d' Pipfile.lock

before committing Pipfile.lock. Couldn’t pipenv just strip the python_version markers out whenever

[requires]
python_version = "..."

is present?

1reaction

proofit404commented, Sep 22, 2018

For some reason, pipenv can add "markers": null, to the lock file.

Heroku fails to deploy a project before I clean up all the markers from the lock file manually.

Top Results From Across the Web

Markers - Packaging

This class abstracts handling markers for dependencies of a project. It can be passed a single marker or multiple markers that are ANDed...

Python Poetry: how to specify platform-specific dependency ...

I've tested the wheel file and it works fine. The problem: how do I tell Poetry to use the wheel on Windows and...

Manage manifest files - Android Developers

A merge rule marker is an XML attribute you can use to express your preference about how to resolve merge conflicts or remove...

PEP 508 – Dependency specification for Python Software ...

There is one caveat - the URL reference form, specified in PEP 440 is not ... Finally the dependency can be made conditional...

pex_binaries - Pants build

This target generator does not work well to generate pex_binary targets where the entry point is for a third-party dependency. Dependency inference will...