running inside tox uses unvendorized dependencies

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Debian testing

• Poetry version: 0.12.11

Issue

I’m trying to use poetry inside tox following instructions in #222.

I think that when installing the local package inside virtualenv created by tox a new poetry is used using dependencies from surrounding virtualenv.

If the project has dependency conflicting with poetry dependencies the installation fails.

My conflicting dependency is jsonschema, jsonschema ^2.6 used by project is not compatible with jsonschema 3 used by poetry.

pyproject.toml

[tool.poetry]
name = "foo"
version = "0.1.0"
authors = "me"
[tool.poetry.dependencies]
python = "^2.7"
jsonschema = "^2.6"

[tool.poetry.dev-dependencies]
tox = "^3.0"

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

tox.ini

[tox]
envlist = py27
skipdist = True

[testenv]
whitelist_externals =
    poetry
skip_install = true
commands =
    poetry install -vvv

$ poetry run tox -r 
py27 recreate: /tmp/foo/.tox/py27
py27 installed: DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.,foo==0.1.0
py27 run-test-pre: PYTHONHASHSEED='2696339194'
py27 runtests: commands[0] | poetry install -vvv
Using virtualenv: /tmp/foo/.tox/py27
Installing dependencies from lock file


Package operations: 9 installs, 0 updates, 0 removals

  - Installing filelock (3.0.10)
  - Installing functools32 (3.2.3-2)
  - Installing pluggy (0.8.1)
  - Installing py (1.7.0)
  - Installing six (1.12.0)
  - Installing toml (0.10.0)
  - Installing virtualenv (16.3.0)
  - Installing jsonschema (2.6.0)
  - Installing tox (3.7.0)
  - Installing foo (0.1.0)
                                                                                                                                       
[EnvCommandError]                                                                                                       
Command ['/tmp/foo/.tox/py27/bin/pip', 'install', '-e', '/tmp/foo', '--no-deps'] errored with the following output:                       
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be main  
tained after that date. A future version of pip will drop support for Python 2.7.                                                    
Obtaining file:///tmp/foo                                                                                                            
  Installing build dependencies: started                                                                                             
  Installing build dependencies: finished with status 'done'                                                                         
  Getting requirements to build wheel: started                                                                                       
  Getting requirements to build wheel: finished with status 'error'                                                                  
  Complete output from command /tmp/foo/.tox/py27/bin/python /tmp/foo/.tox/py27/local/lib/python2.7/site-packages/pip/_vendor/pep51  
7/_in_process.py get_requires_for_build_wheel /tmp/tmpJlWvBQ:                                                                        
  Traceback (most recent call last):                                                                                                 
    File "/tmp/foo/.tox/py27/local/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py", line 207, in <module>             
      main()                                                                                                                         
    File "/tmp/foo/.tox/py27/local/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py", line 197, in main                 
      json_out['return_val'] = hook(**hook_input['kwargs'])                                                                          
    File "/tmp/foo/.tox/py27/local/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py", line 54, in get_requires_for_bui  
ld_wheel                                                                                                                             
      return hook(config_settings)                                                                                                   
    File "/tmp/pip-build-env-p66lJi/overlay/lib/python2.7/site-packages/poetry/masonry/api.py", line 23, in get_requires_for_build_  
wheel                                                                                                                                
      poetry = Poetry.create(".")                                                                                                    
    File "/tmp/pip-build-env-p66lJi/overlay/lib/python2.7/site-packages/poetry/poetry.py", line 106, in create                       
      cls.check(local_config)                                                                                                        
    File "/tmp/pip-build-env-p66lJi/overlay/lib/python2.7/site-packages/poetry/poetry.py", line 226, in check                        
      validation_errors = validate_object(config, "poetry-schema")                                                                   
    File "/tmp/pip-build-env-p66lJi/overlay/lib/python2.7/site-packages/poetry/json/__init__.py", line 25, in validate_object        
      validator = jsonschema.Draft7Validator(schema)                                                                                 
  AttributeError: 'module' object has no attribute 'Draft7Validator'                                                                 
                                                                                                                                     
  ----------------------------------------                                                                                           
Command "/tmp/foo/.tox/py27/bin/python /tmp/foo/.tox/py27/local/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py get_r  
equires_for_build_wheel /tmp/tmpJlWvBQ" failed with error code 1 in /tmp/foo                                                         
                                                                                                                                       
Exception trace:
 /home/u/.poetry/lib/poetry/_vendor/py2.7/cleo/application.py in run() at line 94
   status_code = self.do_run(input_, output_)
 /home/u/.poetry/lib/poetry/console/application.py in do_run() at line 88
   return super(Application, self).do_run(i, o)
 /home/u/.poetry/lib/poetry/_vendor/py2.7/cleo/application.py in do_run() at line 197
   status_code = command.run(input_, output_)
 /home/u/.poetry/lib/poetry/console/commands/command.py in run() at line 77
   return super(BaseCommand, self).run(i, o)
 /home/u/.poetry/lib/poetry/_vendor/py2.7/cleo/commands/base_command.py in run() at line 146
   status_code = self.execute(input_, output_)
 /home/u/.poetry/lib/poetry/_vendor/py2.7/cleo/commands/command.py in execute() at line 107
   return self.handle()
 /home/u/.poetry/lib/poetry/console/commands/install.py in handle() at line 89
   self.env.run("pip", "install", "-e", str(setup.parent), "--no-deps")
 /home/u/.poetry/lib/poetry/utils/env.py in run() at line 492
   return super(VirtualEnv, self).run(bin, *args, **kwargs)
 /home/u/.poetry/lib/poetry/utils/env.py in run() at line 354
   raise EnvCommandError(e)

install [--no-dev] [--dry-run] [-E|--extras EXTRAS] [--develop DEVELOP]

ERROR: InvocationError for command '/home/u/.poetry/bin/poetry install -vvv' (exited with code 1)
_______________________________________________________________ summary _______________________________________________________________
ERROR:   py27: commands failed

[schinckel]

I believe that the suggested way of installing using poetry in tox does not install to an isolated environment: I’m not sure if this is what is causing this issue, but I think it exposes some issues about the way poetry works, that mean it cannot work with tox to ensure a fully isolated test run.

Instead of installing packages to the tox environment that is currently active, it installs to the same environment that is used in development. Whilst this works, it does not feel like it is the correct approach - the point of tox is that it is an isolated test environment, and thus should be an isolated virtual environment too.

It is possible to indicate to poetry that it should not create a virtual environment: but this does not appear to change if poetry will use a virtual environment - in my tox test runs it still installs to the same location (or, more specifically, indicates that it doesn’t need to install).

Likewise, there is a setting that indicates that it should use an “in-project” virtualenv. However, this appears to not work either: it still installs to my “development” virtualenv.

But, setting config values from within a tox environment test run is not a satisfactory pathway when using poetry anyway: all poetry config ... commands write to the user’s global poetry settings: there does not appear to be a way to have configuration that only applies to this current execution of poetry.

Pipenv avoids this problem by automatically installing to the current virtualenv if it detects one: but I think an alternative solution could be to allow setting which virtualenv should be used to install to.

[sinoroc]

Issues #1745 and #1941 might be related.

From my point of view having poetry install and poetry run in the tox commands does not make much sense, since tox already takes care of installing the project (via the sdist built thanks to isolated_build = true) and its dependencies in a virtual environment. If somehow tox had a a way to read and install the dev-dependencies instead of the run-time dependencies then the integration would be much smoother. Maybe the development dependencies should simply be listed in an extra instead.