Sentry has ~8% overhead in Django test suites

Apologies for a somewhat vague report, I’m happy to expand this once it’s decided what the appropriate course of action is.

We’ve found that disabling Sentry (not calling init) in tests saves around 8% of test time on a large Django codebase. This result has been replicated by an engineer at another company with a different Django codebase.

It could be that this is just the expected overhead, in which case I think documenting this would be great. Some advice or comment in that documentation around whether this is worth it would be great as well – is it worth 8% to ensure that Sentry doesn’t interact badly with the rest of the codebase, or is Sentry reliable and isolated enough that it’s unlikely to catch any issues and the 8% time saving is more important.

I wouldn’t be surprised if the overhead is not expected in typical production use, and that tests are a weird case (they throw a lot of handled exceptions for example).

Alternatively, it could be that this overhead is not expected and that this is a performance issue that Sentry would like to address. If so I’m happy to provide data from our test suite if you can point me towards what would be useful for you.

[sl0thentr0py]

Updating this old thread. We now have a benchmark repository where we compare the overhead from a normal django app, one instrumented with Sentry and one instrumented with OpenTelemetry. There is also a related video explaining the setup and some numbers.

There is no official collection of numbers yet but I can update this thread again when we do.

[untitaker]

Thanks @ChillarAnand that confirms the suspicion. I hope we can free resources internally to work on this.

Also, is there a way to estimate the approximate overhead of sentry profiling in production projects?

this generally depends on the kind of web framework and what kind of extensions you are using – 8% is definetly on the upper end, also because we hook into a lot of Django. Integrations for AIOHTTP, Sanic and Flask capture much less data and as such the overhead will be lower. However, if you install a lot of Flask extensions you may get close to the same overhead as Django.