Model Based Testing - Restore State on Shrinking

💬 Question and Help

First time with a question here 😄 I would like to thank you for the library, it is a great tool that have help a lot. I have search though the docs and other issues, but I couldn’t find exactly what I’m looking for. So here is my question.

Is there a way to restore state before every shrink?

Or said in a different way, Is there a way to execute a command/function before each shrink?

When doing Model Based Testing for me it’s fundamental to restore the state of the system after every test. This is mandatory when there is an state mutation and shared resources. There is an easy way to do it:

fc.asyncProperty(
  fc.commands(commands),
  async (cmds) => {
    await restoreState(); // For example, cleaning a db
    const s = () => ({ model: { ... }, real: new DbClient() });
    fc.asyncModelRun(s, cmds);
  }
);

But when an error is found, I didn’t found a way to restore the state between every shrink. I have to enable endOnFailure, which produces non minimal counterexamples.

I think that this library is powerful enough to find very useful counterexamples in these cases. Is it possible to restore the state before each shrink?

Please, if you want me to elaborate more on the question or provide more examples, I’ll include them.

It there is no solution or alternative, I’ll happy trying to implement this if considered.

Others workaraunds that I tried

I tried to modify the command generator to always include an sepecific first command in the sequence, but couldn’t apply .map(...) to the generator in order to append that command.

[vvp]

Fwiw, the initial state provider is called on each shrinking run and it can be async too, so we used something like this to restore a snapshot state:

asyncProperty(commands(...), async (cmds) => {
  const initialState = async () => {
    let stateSnapshot = await restoreState(...)
    console.log("Restored state snapshot")
    return {
      model: await initializeModel(stateSnapshot),
      real: stateSnapshot,
    }
  }
  await asyncModelRun(initialState, cmds)
})

[ignaciobll]

I think it’s probably because when looking for the methods of the AsyncProperty I ended up on the signature page or looking at IAsyncProperty. I didn’t notice IAsyncPropertyWithHooks.

I think it could be really useful a little section on the tips. Maybe a name like Restore Initial State with Model Based Testing could be enough. A little snippet like this could be helpful:

fc.asyncProperty(
  fc.commands(commands),
  async (cmds) => {
    fc.asyncModelRun(() => ({ model: { ... }, real: new Client() }), cmds);
  }
).beforeEach(async () => await restoreInitialState());

Oh! And one of my main doubts. I couldn’t found explicitly said that this hooks are called also when shrinking.