Incompatible with pytest-xdist / thread safety

Describe the bug

I’m using pytest-xdist to run my tests in parallel (mostly a single test parameterized for a vast amount of different test data). Since I wanted to try snapshot-based testing, I’ve added syrupy to the mix. After initial creation of __snapshots__/mytest.ambr (using the --snapshot-update), it disappears from disk at some point during the test run. Without pytest-xdist (i.e. without using -n auto option) this doesn’t happen.

To reproduce

I don’t have a clean reproduction, only an observation. My setup is a single test for a bit of processing, parameterized with a couple of hundreds different input files. While the .ambr file appear initially, after about 80% of tests done, the file disappears.

Expected behavior

Snapshot files get updated with correct data without disappearing, even when used with pytest-xdist or pytest-parallel.

Environment:

• OS: Ubuntu 20.04
• Syrupy Version: 1.4.0
• Python Version: 3.8

Additional context

I assume the problem happens because each test execution writes to the file individually and thus a race condition happens. Not sure what can be done about it, maybe buffer the snapshot file somewhere and write it only at the end of the test run? There are surely better ideas. I assume that e.g. the JUnit reporter works correctly with pytest-xdist, maybe check what it does?

Either way, I wanted to bring this to your attention, would be happy if there was a short- to middle-term solution. If not, I’ll run the suite without parallelization, it’s not that big of a deal at the moment, at least for me 😃

[noahnu]

Taking a stab at this over the weekend! ⚒️

@mcataford you’ll want to work off of the next branch. I was running into performance issues which will need to be tackled first. We’re running benchmarks against the main branch, so you can use those benchmarks as a reference.

[noahnu]

Will try take a deeper look this weekend but what needs to be clarified:

• How do we know when xdist is done and wrapping up? Is there a pytest hook?
• How can we communicate the snapshots we want to write to a single worker/controller? Is there a built in communication mechanism?

If I can’t solve it immediately, I’ll do some refactoring in syrupy to hopefully make it easier to reason about. I’ve been meaning to do some serious refactoring to the syrupy internals for a while and have been planning it for a v2 release since it’ll affect plugin development.