Packages in dev-packages cannot set editable flag if overriding from packages
See original GitHub issueIssue description
I expect to be able to override elements in [packages]
by reproducing them in [dev-packages]
to declare development-time differences.
For example, the following pipfile:
[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true
[dev-packages]
funniest = {editable = true,path = "."}
[packages]
funniest = {path = "."}
[requires]
python_version = "3.7"
with setup.py:
from setuptools import setup
setup(name='funniest',
version='0.1',
description='The funniest joke in the world',
url='http://github.com/storborg/funniest',
author='Flying Circus',
author_email='flyingcircus@example.com',
license='MIT',
packages=['funniest'],
zip_safe=False)
(example taken from the python packaging docs)
and two empty directories, funniest
and foo
. The reason for the foo directory is so we can be in a working directory where funniest
is not importable from the cwd, as this masks the error.
Expected result
I expect that if I run pipenv install
the funniest
package will be installed into the virtual environment. Changes to the current working directory are not applied until reinstallation.
If I run pipenv install --dev
the funniest
package will be installed as editable, so the current working directory is.
That is (paths and output simplified):
example$ cd foo
example/foo$ pipenv --rm
...
example/foo$ pipenv install
...
example/foo$ pipenv run python -c 'import pkg_resources; print(pkg_resources.require("funniest")[0].location)'
~/.local/share/virtualenvs/example-BaUoFXTC/lib/python3.7/site-packages
and
example/foo$ pipenv --rm
...
example/foo$ pipenv install --dev
...
example/foo$ pipenv run python -c 'import pkg_resources; print(pkg_resources.require("funniest")[0].location)'
example/
The generated lockfile looks like:
Actual result
The editable option in [dev-packages]
is masked by the version in [packages]
.
example$ cd foo
example/foo$ pipenv --rm
...
example/foo$ pipenv install
...
example/foo$ pipenv run python -c 'import pkg_resources; print(pkg_resources.require("funniest")[0].location)'
~/.local/share/virtualenvs/example-BaUoFXTC/lib/python3.7/site-packages
and
example/foo$ pipenv --rm
...
example/foo$ pipenv install --dev
...
example/foo$ pipenv run python -c 'import pkg_resources; print(pkg_resources.require("funniest")[0].location)'
~/.local/share/virtualenvs/example-BaUoFXTC/lib/python3.7/site-packages
The generated lockfile is:
{
"_meta": {
"hash": {
"sha256": "35c958102b69fd89f6d04c31de2f9fb49cef92dbcaec95c68f0fd314274fa3c9"
},
"pipfile-spec": 6,
"requires": {
"python_version": "3.7"
},
"sources": [
{
"name": "pypi",
"url": "https://pypi.org/simple",
"verify_ssl": true
}
]
},
"default": {
"funniest": {
"path": "."
}
},
"develop": {
"funniest": {
"path": "."
}
}
}
Note the lack of editable lines in the develop entry.
Steps to replicate
Install a local package in both --dev
and normal modes, differing by their editable flag. The difference is dropped during the locking stage.
$ pipenv --support
Pipenv version: '2018.11.26'
Pipenv location: '/home/matthewwilkes/.local/lib/python3.6/site-packages/pipenv'
Python location: '/usr/bin/python3'
Python installations found:
3.7.3
:/usr/bin/python3.7
3.7.3
:/usr/bin/python3.7m
3.6.7
:/usr/bin/python3.6
3.6.7
:/usr/bin/python3.6m
3.5.6
:/usr/bin/python3.5m
3.5.6
:/usr/bin/python3.5
3.5.3
:/snap/bin/pypy3
3.4.9
:/usr/bin/python3.4
3.4.9
:/usr/bin/python3.4m
2.7.15rc1
:/usr/bin/python2
2.7.13
:/usr/bin/pypy
PEP 508 Information:
{'implementation_name': 'cpython',
'implementation_version': '3.6.7',
'os_name': 'posix',
'platform_machine': 'x86_64',
'platform_python_implementation': 'CPython',
'platform_release': '4.15.0-46-generic',
'platform_system': 'Linux',
'platform_version': '#49-Ubuntu SMP Wed Feb 6 09:33:07 UTC 2019',
'python_full_version': '3.6.7',
'python_version': '3.6',
'sys_platform': 'linux'}
System environment variables:
SESSION_MANAGER
XDG_CONFIG_DIRS
QT_ACCESSIBILITY
LANG
DISPLAY
SHLVL
LOGNAME
LANGUAGE
MANDATORY_PATH
XDG_VTNR
USER
J2REDIR
J2SDKDIR
XAUTHORITY
LG_WEBOS_TV_SDK_HOME
GTK_IM_MODULE
COLORTERM
XDG_SESSION_ID
JAVA_HOME
GNOME_TERMINAL_SCREEN
IM_CONFIG_PHASE
XDG_SESSION_DESKTOP
GDMSESSION
TEXTDOMAINDIR
DEFAULTS_PATH
WINDOWPATH
USERNAME
TEXTDOMAIN
GTK2_MODULES
DERBY_HOME
PWD
DBUS_SESSION_BUS_ADDRESS
GNOME_DESKTOP_SESSION_ID
VTE_VERSION
DESKTOP_SESSION
GJS_DEBUG_TOPICS
QT4_IM_MODULE
CLUTTER_IM_MODULE
GNOME_TERMINAL_SERVICE
XDG_MENU_PREFIX
XDG_DATA_DIRS
QT_IM_MODULE
MOZ_USE_XINPUT2
S_COLORS
XDG_SESSION_TYPE
GJS_DEBUG_OUTPUT
TERM
SHELL
GTK_MODULES
SSH_AGENT_PID
GNOME_SHELL_SESSION_MODE
WEBOS_CLI_TV
PATH
XDG_CURRENT_DESKTOP
XMODIFIERS
HOME
XDG_RUNTIME_DIR
SSH_AUTH_SOCK
XDG_SEAT
GPG_AGENT_INFO
OLDPWD
LS_COLORS
_
PIP_DISABLE_PIP_VERSION_CHECK
PYTHONDONTWRITEBYTECODE
PIP_SHIMS_BASE_MODULE
PIP_PYTHON_PATH
PYTHONFINDER_IGNORE_UNSUPPORTED
Pipenv–specific environment variables:
Debug–specific environment variables:
PATH
:/home/matthewwilkes/bin:/home/matthewwilkes/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/lib/jvm/java-8-oracle/bin:/usr/lib/jvm/java-8-oracle/db/bin:/usr/lib/jvm/java-8-oracle/jre/bin:/home/matthewwilkes/.local/bin
SHELL
:/usr/bin/zsh
LANG
:en_GB.UTF-8
PWD
:/secure/coding/pipenvtest
Contents of Pipfile
(‘/secure/coding/pipenvtest/Pipfile’):
[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true
[dev-packages]
funniest = {editable = true,path = "."}
[packages]
funniest = {path = "."}
[requires]
python_version = "3.7"
Contents of Pipfile.lock
(‘/secure/coding/pipenvtest/Pipfile.lock’):
{
"_meta": {
"hash": {
"sha256": "35c958102b69fd89f6d04c31de2f9fb49cef92dbcaec95c68f0fd314274fa3c9"
},
"pipfile-spec": 6,
"requires": {
"python_version": "3.7"
},
"sources": [
{
"name": "pypi",
"url": "https://pypi.org/simple",
"verify_ssl": true
}
]
},
"default": {
"funniest": {
"path": "."
}
},
"develop": {
"funniest": {
"path": "."
}
}
}
Issue Analytics
- State:
- Created 4 years ago
- Reactions:2
- Comments:7 (1 by maintainers)
Whilst the reasoning above is solid (dev packages should never override prod packages), I agree with @sc250024 that there is a vital workflow that is going unsupported here. The equivalent in node would be
npm link
.@techalchemy is there something we can do to support this? IMO i think it would work very cleanly if
pipenv install --dev
gave preference to packages under[dev-packages]
when a collision occurs. That way it would be very clear how the collision is resolved, where right now its an implicit preference to[packages]
at the expense of what is (for me at least) a super common workflow.This seems to be a valid use case that @MatthewWilkes brought up, no? I’m having a similar issue. The reason why I want
editable
in development is so that I can work locally, but when I’m building my project as a Docker image for distribution, I want it to be a package to make things easier for multi-stage Docker builds.Currently this does not work if I have
my-app = {editable = true,path = "."}
in[packages]
:And if the container runs, I get the following error:
The reason is because the
my-app
egg gets placed into the/build
directory in the first container because it’seditable
, and doesn’t get placed inPYROOT
like all of the other packages do.What’s the recommended solution around this then?