Environment variable substitution should be attempted after each action

I’m using tox-docker which exports some environment variables for me to use to connect to a container’s exposed port. I tried using setenv to construct a DSN from those environment variables so that the test suite will be able to know which container to connect to:

# ...
[setenv]
    redis: TEST_BROKER=redis://{env:REDIS_HOST}:{env:REDIS_6379_PORT}/0
    redis: TEST_BACKEND=redis://{env:REDIS_HOST}:{env:REDIS_6379_PORT}/1

This unfortunately fails in the following fashion:

tox -e 3.7-integration-redis -r
3.7-integration-redis create: /home/omer/Documents/Projects/celery/.tox/3.7-integration-redis
_________________________________________________________________________________________________ summary __________________________________________________________________________________________________
  3.7-integration-redis: commands succeeded
  congratulations :)
Traceback (most recent call last):
  File "/home/omer/.local/share/virtualenvs/celery/lib/python3.7/site-packages/tox/config/__init__.py", line 348, in get
    return self.resolved[name]
KeyError: 'TEST_BROKER'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/omer/.local/share/virtualenvs/celery/bin/tox", line 10, in <module>
    sys.exit(cmdline())
  File "/home/omer/.local/share/virtualenvs/celery/lib/python3.7/site-packages/tox/session/__init__.py", line 42, in cmdline
    main(args)
  File "/home/omer/.local/share/virtualenvs/celery/lib/python3.7/site-packages/tox/session/__init__.py", line 66, in main
    exit_code = session.runcommand()
  File "/home/omer/.local/share/virtualenvs/celery/lib/python3.7/site-packages/tox/session/__init__.py", line 190, in runcommand
    return self.subcommand_test()
  File "/home/omer/.local/share/virtualenvs/celery/lib/python3.7/site-packages/tox/session/__init__.py", line 218, in subcommand_test
    run_sequential(self.config, self.venv_dict)
  File "/home/omer/.local/share/virtualenvs/celery/lib/python3.7/site-packages/tox/session/commands/run/sequential.py", line 9, in run_sequential
    if venv.setupenv():
  File "/home/omer/.local/share/virtualenvs/celery/lib/python3.7/site-packages/tox/venv.py", line 584, in setupenv
    status = self.update(action=action)
  File "/home/omer/.local/share/virtualenvs/celery/lib/python3.7/site-packages/tox/venv.py", line 242, in update
    self.hook.tox_testenv_create(action=action, venv=self)
  File "/home/omer/.local/share/virtualenvs/celery/lib/python3.7/site-packages/pluggy/hooks.py", line 289, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/home/omer/.local/share/virtualenvs/celery/lib/python3.7/site-packages/pluggy/manager.py", line 68, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/home/omer/.local/share/virtualenvs/celery/lib/python3.7/site-packages/pluggy/manager.py", line 62, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/home/omer/.local/share/virtualenvs/celery/lib/python3.7/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/home/omer/.local/share/virtualenvs/celery/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/omer/.local/share/virtualenvs/celery/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/home/omer/.local/share/virtualenvs/celery/lib/python3.7/site-packages/tox/venv.py", line 677, in tox_testenv_create
    redirect=reporter.verbosity() < reporter.Verbosity.DEBUG,
  File "/home/omer/.local/share/virtualenvs/celery/lib/python3.7/site-packages/tox/venv.py", line 543, in _pcall
    env = self._get_os_environ(is_test_command=is_test_command)
  File "/home/omer/.local/share/virtualenvs/celery/lib/python3.7/site-packages/tox/venv.py", line 462, in _get_os_environ
    env.update(self.envconfig.setenv)
  File "/home/omer/.local/share/virtualenvs/celery/lib/python3.7/site-packages/tox/config/__init__.py", line 364, in __getitem__
    x = self.get(name, self._DUMMY)
  File "/home/omer/.local/share/virtualenvs/celery/lib/python3.7/site-packages/tox/config/__init__.py", line 358, in get
    self.resolved[name] = res = self.reader._replace(val)
  File "/home/omer/.local/share/virtualenvs/celery/lib/python3.7/site-packages/tox/config/__init__.py", line 1517, in _replace
    replaced = Replacer(self, crossonly=crossonly).do_replace(value)
  File "/home/omer/.local/share/virtualenvs/celery/lib/python3.7/site-packages/tox/config/__init__.py", line 1553, in do_replace
    expanded = substitute_once(value)
  File "/home/omer/.local/share/virtualenvs/celery/lib/python3.7/site-packages/tox/config/__init__.py", line 1551, in substitute_once
    return self.RE_ITEM_REF.sub(self._replace_match, x)
  File "/home/omer/.local/share/virtualenvs/celery/lib/python3.7/site-packages/tox/config/__init__.py", line 1589, in _replace_match
    return self._replace_env(match)
  File "/home/omer/.local/share/virtualenvs/celery/lib/python3.7/site-packages/tox/config/__init__.py", line 1610, in _replace_env
    raise tox.exception.MissingSubstitution(key)
tox.exception.MissingSubstitution: MissingSubstitution: REDIS_HOST

I’d like tox to defer variable substitution if it fails and retry it until the test commands are executed. If it failed before that, tox should fail as well.

This gives plugins the opportunity to export environment variables which we can use in latter stages.

[gaborbernat]

In the meantime, as a workaround, I would propose you creating a plugin that does this 👍

[gaborbernat]

This will be duplicate of #1191 then.