Manually add concrete examples to test
See original GitHub issueConsider the following scenario: Through usage, a bug is discovered in a function for some concrete input. In order to test the bug and potentially shrink the input a property based test is created. But, it turns out that none of the random values that the property is tried with encounters the bug. For instance, let’s say that a function fails when given a prime number but none of the random numbers generated happens to be prime? What does one do in such a case?
It seems to me that it would be nice if it was possible to specify some concrete examples for assert
that it would then always try. Something like this:
fc.assert(
fc.property(fc.tuple(fc.nat(), fc.string()), ...),
{
examples: [
[ 28374, "foobarbaz" ]
]
}
);
The above is just a way to combine the random generated input with some hardcoded input that is known to trigger or have triggered an error.
What do you think about the above? I just ran in to a problem similar to the above. As of right now, I’ve just duplicated the test with the values I know doesn’t work hardcoded into it.
Issue Analytics
- State:
- Created 5 years ago
- Reactions:1
- Comments:11 (11 by maintainers)
The feature of concrete examples has been released with fast-check@1.4.0.
Looking forward for your feedbacks on it.
I will work on improving it by adding shrinker to it later. This feature is really a good idea 😃
@paldepind Next minor release will include shrinking capabilities on examples 🥳
See https://github.com/dubzzz/fast-check/pull/2391