Mocking is not completely undone when decorated tests are nested

First of all, I’d like to point out that I only noticed this problem in a project where HTTP requests are mocked with multiple libraries (I know, bad style).

Environment

• Ubuntu 20.04
• Python 3.X, X in (6, 8)
• responses 0.17.0
• httpretty 1.1.4

Steps to Reproduce

1. As mentioned, I ended up with a testsuite that uses two libraries for mocking: responses and httpretty.
2. In an effort to reduce code duplication in the test suite, I wanted to re-use a test method decorated with responses.activate within another test method, that is also decorated with responses.activate.
3. Another test method decorated with httpretty.activate will fail, if it is executed after the test including the nesting is executed.

To illustrate this, I have created a gist.

Expected Result

Mocks introduced my the responses package are completely rolled back, even if there is a nesting of mocks.

Actual Result

To illustrate this, I have created a gist:

• In it’s current state (no nesting of tests decorated with responses.activate), all tests in the gist will pass.
• If the decorator in line 29 is commented in, one test will fail:

======================================================================
ERROR: test_c_httpretty (scratch_4.TestRequest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/andi/virtualenvs/smash/lib/python3.8/site-packages/httpretty/core.py", line 2075, in wrapper
    return test(*args, **kw)
  File "/home/andi/.config/JetBrains/PyCharm2021.2/scratches/scratch_4.py", line 29, in test_c_httpretty
    self.assertEqual(b"Hello 3", requests.get("http://example.com/3").content)
  File "/home/andi/virtualenvs/smash/lib/python3.8/site-packages/requests/api.py", line 75, in get
    return request('get', url, params=params, **kwargs)
  File "/home/andi/virtualenvs/smash/lib/python3.8/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/andi/virtualenvs/smash/lib/python3.8/site-packages/requests/sessions.py", line 529, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/andi/virtualenvs/smash/lib/python3.8/site-packages/requests/sessions.py", line 645, in send
    r = adapter.send(request, **kwargs)
  File "/home/andi/virtualenvs/smash/lib/python3.8/site-packages/responses/__init__.py", line 842, in unbound_on_send
    return self._on_request(adapter, request, *a, **kwargs)
  File "/home/andi/virtualenvs/smash/lib/python3.8/site-packages/responses/__init__.py", line 821, in _on_request
    raise response
requests.exceptions.ConnectionError: Connection refused by Responses - the call doesn't match any registered mock.

Request: 
- GET http://example.com/3

Available matches:

[crazyscientist]

or better switch to responses

That’s the plan, but replacing the mock definitions in over 700 tests takes time.

just split two libraries into two different classes

That has no effect, even if the two use cases occur in different modules. As far as I can tell, it affects all discovered httpretty tests executed after the nested one.

why do you call one test from another, this is an architectural mistake.

That is true, but I was lazy and just wanted to run an already existing test with some additional constraint. So, I just defined the new test case with decorator and called the existing one.

I think applying these two design patterns will eliminate the issue

Yes, there are a few more workarounds I can think of, but I wanted to share my findings nevertheless. Because someone might also consider this unexpected behavior.

For what it’s worth, it just occurred to me, that this problem is not limited to the use of two mocking libraries. Even if one wants to perform real requests, those will fail. I have extended the gist and get this new exception:

ERROR: test_d (scratch_4.TestRequest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/andi/.config/JetBrains/PyCharm2021.2/scratches/scratch_4.py", line 32, in test_d
    response = requests.head("https://www.google.com")
  File "/home/andi/virtualenvs/smash/lib/python3.8/site-packages/requests/api.py", line 102, in head
    return request('head', url, **kwargs)
  File "/home/andi/virtualenvs/smash/lib/python3.8/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/andi/virtualenvs/smash/lib/python3.8/site-packages/requests/sessions.py", line 529, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/andi/virtualenvs/smash/lib/python3.8/site-packages/requests/sessions.py", line 645, in send
    r = adapter.send(request, **kwargs)
  File "/home/andi/virtualenvs/smash/lib/python3.8/site-packages/responses/__init__.py", line 842, in unbound_on_send
    return self._on_request(adapter, request, *a, **kwargs)
  File "/home/andi/virtualenvs/smash/lib/python3.8/site-packages/responses/__init__.py", line 821, in _on_request
    raise response
requests.exceptions.ConnectionError: Connection refused by Responses - the call doesn't match any registered mock.

Request: 
- HEAD https://www.google.com/

Available matches:

I do appreciate your lack of enthusiasm for “fixing” this. My primary goal was to document this.

[beliaev-maksim]

copy gist here to track the history:

import unittest

import requests
import responses


class TestRequest(unittest.TestCase):
    def test_a(self):
        self.assertEqual(200, requests.get("https://www.google.com").status_code)

    @responses.activate
    def test_b_response_1(self):
        responses.add(responses.GET, "http://example.com/1", body="Hello 1")
        self.assertEqual(b"Hello 1", requests.get("http://example.com/1").content)

    @responses.activate
    def test_b_response_2(self):
        self.test_b_response_1()

    def test_c(self):
        response = requests.head("https://www.google.com")
        self.assertEqual(response.status_code, 200)