Unable to use imports with stages (see components example)

After running the Dockerized flask server in the “components example” (https://github.com/taverntesting/tavern/tree/master/example/components) , I then run the py.test tests in the same directory. The test fails with the below, which is a similar failure I’m experiencing in my own code project (I’m new to Tavern). If I put variables in a common file, I can import and use those variables. However, if I have a common stages section I want run (ie login and save the JWT token), it appears successive tests can’t make use of such an import. Such is the case in the example.

/Users/hamx0r/PyCharmProjects/tavern/venv/bin/python /Applications/PyCharm.app/Contents/plugins/python/helpers/pycharm/_jb_pytest_runner.py --path /Users/hamx0r/PyCharmProjects/tavern/example/components
Testing started at 4:31 PM ...
Launching pytest with arguments /Users/hamx0r/PyCharmProjects/tavern/example/components --no-header --no-summary -q in /Users/hamx0r/PyCharmProjects/tavern/example/components

============================= test session starts ==============================
collecting ... collected 2 items

test_hello.tavern.yaml::Test authenticated /hello FAILED                 [ 50%]Error running prepared request
Traceback (most recent call last):
  File "/Users/hamx0r/PyCharmProjects/tavern/venv/lib/python3.6/site-packages/urllib3/connection.py", line 175, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "/Users/hamx0r/PyCharmProjects/tavern/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 96, in create_connection
    raise err
  File "/Users/hamx0r/PyCharmProjects/tavern/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 86, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 61] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/hamx0r/PyCharmProjects/tavern/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 706, in urlopen
    chunked=chunked,
  File "/Users/hamx0r/PyCharmProjects/tavern/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 394, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/Users/hamx0r/PyCharmProjects/tavern/venv/lib/python3.6/site-packages/urllib3/connection.py", line 239, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/Users/hamx0r/.pyenv/versions/3.6.14/lib/python3.6/http/client.py", line 1291, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Users/hamx0r/.pyenv/versions/3.6.14/lib/python3.6/http/client.py", line 1337, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Users/hamx0r/.pyenv/versions/3.6.14/lib/python3.6/http/client.py", line 1286, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Users/hamx0r/.pyenv/versions/3.6.14/lib/python3.6/http/client.py", line 1046, in _send_output
    self.send(msg)
  File "/Users/hamx0r/.pyenv/versions/3.6.14/lib/python3.6/http/client.py", line 984, in send
    self.connect()
  File "/Users/hamx0r/PyCharmProjects/tavern/venv/lib/python3.6/site-packages/urllib3/connection.py", line 205, in connect
    conn = self._new_conn()
  File "/Users/hamx0r/PyCharmProjects/tavern/venv/lib/python3.6/site-packages/urllib3/connection.py", line 187, in _new_conn
    self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x1111e0780>: Failed to establish a new connection: [Errno 61] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/hamx0r/PyCharmProjects/tavern/venv/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/Users/hamx0r/PyCharmProjects/tavern/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 756, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/Users/hamx0r/PyCharmProjects/tavern/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 574, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5009): Max retries exceeded with url: /hello/Jim (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x1111e0780>: Failed to establish a new connection: [Errno 61] Connection refused',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/hamx0r/PyCharmProjects/tavern/tavern/_plugins/rest/request.py", line 483, in run
    return self._prepared()
  File "/Users/hamx0r/PyCharmProjects/tavern/tavern/_plugins/rest/request.py", line 463, in prepared_request
    return session.request(**self._request_args)
  File "/Users/hamx0r/PyCharmProjects/tavern/venv/lib/python3.6/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/hamx0r/PyCharmProjects/tavern/venv/lib/python3.6/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/Users/hamx0r/PyCharmProjects/tavern/venv/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5009): Max retries exceeded with url: /hello/Jim (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x1111e0780>: Failed to establish a new connection: [Errno 61] Connection refused',))

example/components/test_hello.tavern.yaml:0 (/Users/hamx0r/PyCharmProjects/tavern/example/components/test_hello.tavern.yaml::Test authenticated /hello)
Format variables:
  service:s = 'http://localhost:5009'

Source test stage (line 9):
  - name: Unauthenticated /hello
    request:
      url: "{service:s}/hello/Jim"
      method: GET
    response:
      status_code: 401

Formatted stage:
  name: Unauthenticated /hello
  request:
    method: GET
    url: 'http://localhost:5009/hello/Jim'
  response:
    status_code: 401

Errors:
E   tavern.util.exceptions.RestRequestException




FAILED                   [100%]Error running prepared request
Traceback (most recent call last):
  File "/Users/hamx0r/PyCharmProjects/tavern/venv/lib/python3.6/site-packages/urllib3/connection.py", line 175, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "/Users/hamx0r/PyCharmProjects/tavern/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 96, in create_connection
    raise err
  File "/Users/hamx0r/PyCharmProjects/tavern/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 86, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 61] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/hamx0r/PyCharmProjects/tavern/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 706, in urlopen
    chunked=chunked,
  File "/Users/hamx0r/PyCharmProjects/tavern/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 394, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/Users/hamx0r/PyCharmProjects/tavern/venv/lib/python3.6/site-packages/urllib3/connection.py", line 239, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/Users/hamx0r/.pyenv/versions/3.6.14/lib/python3.6/http/client.py", line 1291, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Users/hamx0r/.pyenv/versions/3.6.14/lib/python3.6/http/client.py", line 1337, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Users/hamx0r/.pyenv/versions/3.6.14/lib/python3.6/http/client.py", line 1286, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Users/hamx0r/.pyenv/versions/3.6.14/lib/python3.6/http/client.py", line 1046, in _send_output
    self.send(msg)
  File "/Users/hamx0r/.pyenv/versions/3.6.14/lib/python3.6/http/client.py", line 984, in send
    self.connect()
  File "/Users/hamx0r/PyCharmProjects/tavern/venv/lib/python3.6/site-packages/urllib3/connection.py", line 205, in connect
    conn = self._new_conn()
  File "/Users/hamx0r/PyCharmProjects/tavern/venv/lib/python3.6/site-packages/urllib3/connection.py", line 187, in _new_conn
    self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x110d95160>: Failed to establish a new connection: [Errno 61] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/hamx0r/PyCharmProjects/tavern/venv/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/Users/hamx0r/PyCharmProjects/tavern/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 756, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/Users/hamx0r/PyCharmProjects/tavern/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 574, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5009): Max retries exceeded with url: /ping (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x110d95160>: Failed to establish a new connection: [Errno 61] Connection refused',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/hamx0r/PyCharmProjects/tavern/tavern/_plugins/rest/request.py", line 483, in run
    return self._prepared()
  File "/Users/hamx0r/PyCharmProjects/tavern/tavern/_plugins/rest/request.py", line 463, in prepared_request
    return session.request(**self._request_args)
  File "/Users/hamx0r/PyCharmProjects/tavern/venv/lib/python3.6/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/hamx0r/PyCharmProjects/tavern/venv/lib/python3.6/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/Users/hamx0r/PyCharmProjects/tavern/venv/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5009): Max retries exceeded with url: /ping (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x110d95160>: Failed to establish a new connection: [Errno 61] Connection refused',))

example/components/test_ping.tavern.yaml:0 (/Users/hamx0r/PyCharmProjects/tavern/example/components/test_ping.tavern.yaml::Test authenticated /ping)
Format variables:
  service:s = 'http://localhost:5009'

Source test stage (line 9):
  - name: Unauthenticated /ping
    request:
      url: "{service:s}/ping"
      method: GET
    response:
      status_code: 401

Formatted stage:
  name: Unauthenticated /ping
  request:
    method: GET
    url: 'http://localhost:5009/ping'
  response:
    status_code: 401

Errors:
E   tavern.util.exceptions.RestRequestException








test_ping.tavern.yaml::Test authenticated /ping 

============================== 2 failed in 1.00s ===============================

Process finished with exit code 1

[dongfangtianyu]

On other topics:

The Tavern tests are not run “in order”

Clearly, The Tavern tests are run “in order”

In test_hello.tavern.yaml, there are three steps:

1. Unauthenticated /hello (in test_hello.tavern.yaml)
2. Login and acquire token (in auth_parse.yaml)
3. Authenticated /hello (in test_hello.tavern.yaml)

auth_parse.yaml is included and executed at the specified location https://github.com/taverntesting/tavern/blob/master/example/components/test_hello.tavern.yaml#L16

---

I expected a single authentication request to get a token, and then that token to be used for all tests.

Currently, the test files are isolated from each other, so the result of a request will not be shared among all tests. If you really need to do this, you can consider using a hook to directly share global variables in all requests, but it may make the test more complicated:

# conftest.py

test_login_token = ""


def pytest_tavern_beta_after_every_response(expected, response):
    global test_login_token
    if "token" in response.json():
        test_login_token = response.json()["token"]


def pytest_tavern_beta_before_every_test_run(test_dict, variables):
    variables["test_login_token"] = test_login_token

[dongfangtianyu]

great, @hamx0r ! You have found the key to the problem!

After testing, specifying the version of the third-party library in Dockerfile can avoid this problem.