Use golang fuzzing-like api for seeds instead of requiring duplicating prop tests

💡 Idea

Currently if you want to check a prop test seed in, you need to duplicate the prop test to keep the unit test plus the prop test.

It’d be great if there was an api like go’s F.add() api that allowed you to provide seed test cases that are tested in addition to properties being tested.

see https://tip.golang.org/doc/tutorial/fuzz#fuzz_test

func FuzzReverse(f *testing.F) {
    testcases := []string{"Hello, world", " ", "!12345"}
    for _, tc := range testcases {
        f.Add(tc)  // Use f.Add to provide a seed corpus
    }
    f.Fuzz(func(t *testing.T, orig string) {
        rev := Reverse(orig)
        doubleRev := Reverse(rev)
        if orig != doubleRev {
            t.Errorf("Before: %q, after: %q", orig, doubleRev)
        }
        if utf8.ValidString(orig) && !utf8.ValidString(rev) {
            t.Errorf("Reverse produced invalid UTF-8 string %q", rev)
        }
    })
}

Motivation

this would allow minimizing LOC for a property test while allowing for both property testing and checking in old found bugs

Example

A prototype api might be:

it("should add properties successfully", async () => {
    expect(
      await overrideSvgAttributes("<SVG></SVG>", {
        height: null,
        width: null,
        "aria-hidden": true,
        "aria-label": null,
      })
    ).toBe('<svg aria-hidden="true"></svg>');

    const svgArbitrary = fc
      .record({
        start: fc.mixedCase(fc.constantFrom("<svg>")),
        middle: fc.fullUnicodeString(),
        end: fc.mixedCase(fc.constantFrom("</svg>")),
      })
      .map(({ start, middle, end }) => `${start}${middle}${end}`);

    await fc.assert(
      fc.asyncProperty(
        svgArbitrary,
        fc.record<SVGAttributes>({
          height: fc.option(fc.integer()),
          width: fc.option(fc.integer()),
          "aria-hidden": fc.option(fc.boolean()),
          "aria-label": fc.option(fc.string()),
        }),
        fc.context(),
        async (svgSource, overrides, ctx) => {
          ctx.log(svgSource);
          const transformedSource = await overrideSvgAttributes(
            svgSource,
            overrides
          );
          ctx.log(transformedSource);
          expect(transformedSource).toBeTruthy();
          // every truthy override should exist in the transformed source
          Object.entries(overrides)
            .filter(([, value]) => {
              ctx.log(`${value}, ${!!value}`);
              return !!value;
            })
            .forEach(([override]) => {
              expect(transformedSource).toContain(override);
            });
        }
      ),
	{
	seeds: ["<svG></svg>", {}]
    );
  });

more concisely:

fc.assert(fc.property(testCase()), { seeds: [...list of seeds] })

I don’t think that’s a great api, but something like it would be amazing

[jasikpark]

thx for working on all this again btw!

[jasikpark]

partial examples would be really funny, true