Allow deterministic snapshot testing

🚀 Feature Request

Add a function that uses arbitraries to deterministically create output.

Motivation

When working with legacy code it is often hard to define exact properties. The main goal usually is to “keep the existing behavior”. Nonetheless, property-based testing can help here by generating random inputs for a fixed seed and returning input-output pairs. These can then be saved and compared with help of snapshot testing tools, e. g. Jest Snapshot testing.

Example

This could be implemented as

interface IOutput<T1, R> {
  inputs: [T1],
  output: R
}

function generateResults<T1, R>(
        arb1: Arbitrary<T1>,
        resultGetter: (t1:T1) => R): IOutput<T1, R>[];

and then used in jest as

function add(a, b) {
  return a+b
}

...

it('returns the same results as before', () => {
  const results = fc.generateResults(
    fc.float(),
    fc.float(),
    add
  )
  expect(results).toMatchSnapshot()
})

Open questions

• How to handle seeding? The output needs to be deterministic (otherwise the snapshot will fail all the time). My preferred solution would be to just add a global option for a seed that by default is 0, but I am not sure how fast-check behaves with a fixed seed 0.
• I would be willing to try myself at a PR, if there is interest in this, but haven’t worked with the internals of the library before, so I might need some pointers where to start.

[dubzzz]

@dbartholomae

For your information:

The next minor version of fast-check - 1.22.0 - will certainly change the way fast-check generates random values. Up to 1.21.0 random instances generated by fast-check were too close - see https://github.com/dubzzz/fast-check/issues/524. I had to change the way fast-check uses random generators to fix this issue.

You might need to update your snapshots.

[dbartholomae]

D’oh, I should have seen that xD Well, I’ll stick to sample then for now 😃 Is there any other feature you would like me help with? Just tag me in the relevant issue, I have some spare time between the years.