Manually add concrete examples to test

Consider the following scenario: Through usage, a bug is discovered in a function for some concrete input. In order to test the bug and potentially shrink the input a property based test is created. But, it turns out that none of the random values that the property is tried with encounters the bug. For instance, let’s say that a function fails when given a prime number but none of the random numbers generated happens to be prime? What does one do in such a case?

It seems to me that it would be nice if it was possible to specify some concrete examples for assert that it would then always try. Something like this:

fc.assert(
  fc.property(fc.tuple(fc.nat(), fc.string()), ...),
  {
    examples: [
      [ 28374, "foobarbaz" ]
    ]
  }
);

The above is just a way to combine the random generated input with some hardcoded input that is known to trigger or have triggered an error.

What do you think about the above? I just ran in to a problem similar to the above. As of right now, I’ve just duplicated the test with the values I know doesn’t work hardcoded into it.

[dubzzz]

The feature of concrete examples has been released with fast-check@1.4.0.

Looking forward for your feedbacks on it.

I will work on improving it by adding shrinker to it later. This feature is really a good idea 😃

[dubzzz]

@paldepind Next minor release will include shrinking capabilities on examples 🥳

See https://github.com/dubzzz/fast-check/pull/2391