Advice request: monkeypatch vs mock.patch

I’m working on a codebase where tests use a mixture of mock.patch and pytest’s monkeypatch, seemingly based on authors’ personal preferences.

The official docs for the latter, https://docs.pytest.org/en/latest/monkeypatch.html, refer to a blog post that’s nearing its 10th anniversary; meanwhile the earlier made it into Python proper.

I wonder if there’s official advice, like “use X”, or perhaps “if you need feature Y, use Z” to choose between the two.

[asottile]

It does seem to come down to personal preference as far as I’ve seen so far. There’s really three options that work well for pytest (imo) so I’ll outline them here. Note these are my opinions and not necessarily representative of others involved with the project or any sort of “official” stance. It’s possible we should put something together in the documentation since it is a pretty common subject 👍

1. monkeypatch
   • PRO: comes with pytest, no extra dependencies in python2 / python 3
   • PRO (or CON depending on your attitude here, MagicMock is some crazy shenanigans): is dead simple, no MagicMock, no call tracking, just normal attribute setting
   • CON: as it’s a fixture, the scope is often more broad than expected instead of “just part of the function” or “just the function”, it can often lead to patches “leaking” into other fixtures / etc. It’s also difficult to control the ordering in some cases
     • ok this isn’t strictly fair, there is a context manager version, it’s just not the “default style”
   • CON: potentially less battle tested than mock (for example, #4536)
2. mock
   • CON: for python2.x you need a dependency on the mock backport
   • PRO: if you’re python3.x only, it comes with the standard library (unittest.mock)
   • PRO: many more features than monkeypatch (call tracking, MagicMock, assertion framework (though it has a really bad api (out of date article before mock 2.x made this ~slightly better))
   • PRO: tight control over mocked context via context managers / decorators (if you want fixtures, you can make those too with yield
   • CON: MagicMock is some crazy magic
3. pytest-mock: adds a mocker fixture which uses mock under the hood but with a surface area / api similar to monkeypatch
   • Basically all of the features of mock, but with the api of monkeypatch. I don’t like using it due to the same reasons I mentioned about scoping of patches in monkeypatch

In code that I write, I tend to stick to mock (if it’s even necessary at all)

I also wonder if pytest should gut monkeypatch when dropping python2.x and replace the internals with unittest.mock 🤔

[bluetech]

I’m not @RonnyPfannschmidt but here is my opinion on why mock.patch/monkeypatch is usually better avoided.

Monkeypatching, by definition, breaks the abstraction barrier. Some code reaches into some other code and changes it bowls. The code calls some_function(), but what actually runs is patched_in_function(). The trouble with relying on internal details is that it is brittle. If the code is refactored to call some_other_function() instead, the test breaks, even if the behavior is exactly the same.

A better alternative is to “formalize” the relationship between the test and the code. The conventional way to do it is give the test explicit control over the particular thing it wants to patch, usually using dependency injection.

For an example I’ll use the post linked by @asottile. In it, they have this code

def restart_server(server):
    ...

def restart_servers_in_datacenter(servers, datacenter):
    for server in servers:
        if server.endswith("-%s" % datacenter):
            restart_server(server)

and they want to write a test for restart_servers_in_datacenter, but without it actually going to restart actual servers. What they did was to patch the restart_server function, and they explain some problems they ran into and how they fixed them.

The alternative to patching is do something like this:

class ServerManager:
    # Takes some dependencies itself (for example).
    def __init__(self, http_client: HttpClient, auth_token: str) -> None:
        self.http_client = http_client
        self.auth_token = auth_token

     def restart_server(self, server: Server) -> None:
          self.http_client.post("restart server", auth_token=self.auth_token, {server: server.id})

def restart_servers_in_datacenter(server_manager: ServerManager, servers, datacenter):
    for server in servers:
        if server.endswith("-%s" % datacenter):
            server_manager.restart_server(server)

Now the test doesn’t need to patch. It can do this:

fake_server_manager = mock.create_autospec(ServerManager)
restart_servers(fake_server_manager, servers, datacenter)
assert len(fake_server_manager.restart_server.calls) == 1

Now the test cannot make mistakes, it most provide its own implementation of the dependency. It is not possible for the real code to run accidentally. And there is no abstraction being broken, no peace is disturbed, just regular argument passing.

You can decide to fake at a deeper level, if you want to increase the coverage:

fake_http_client = mock.create_autospec(HttpClient)
server_manager = ServerManager(fake_http_client, 'fake token')
restart_servers(server_manager, servers, datacenter)
assert len(fake_server_manager.restart_server.calls) == 1

Sometimes it’s beneficial to go “full Java” and do this:

class ServerManager(meta=ABCMeta):
      @abstractmethod
     def restart_server(self, server: Server) -> None: pass

# As before.
class RealServerManager(ServerManager): ...

# Tests can use this.
class FakeServerManager(ServerManager):
     def __init__(self) -> None:
          self.restarted_servers = {}  # type: Set[Server]

     def restart_server(self, server: Server) -> None:
          self.restarted_servers.add(server)

depending on the case.

This intersects with general OO good practices for testable code, see here for example. This style of programming is also enforced in the object-capability security model, which I (personally) hope will gain more prominence in the future.

As a disclaimer, I should say that sometimes monkeypatching in tests is necessary, for example when dealing with external code you have no control over. And sometimes it is just easiest and putting more effort is not worth it. And sometimes you intentionally want to test some internal detail. I don’t mean to be dogmatic.