RFC: Data driven testing

The one feature I miss the most from other test frameworks is a nice, clean way of testing multiple inputs and multiple outputs. I first came over this pattern when using Spock, where they call it Data driven testing. It looks basically like this (example taken from link):

def "maximum of two numbers"(int a, int b, int c) {
    expect:
    Math.max(a, b) == c

    where:
    a | b | c
    1 | 3 | 3
    7 | 4 | 7
    0 | 0 | 0
}

This provides a very concise and readable test, perfect for testing pure functions without resorting to looping through values manually.

In Jest, I think the best way would be to implement it similar to how jest-each works today:

jest.each([
  [1, 1, 2],
  [1, 2, 3],
  [2, 1, 3],
])([a, b, c]) => {
  expect(add(a, b)).toBe(expected);
});

Potentially by parsing a template string, so it’s closer to Spock’s implementation:

jest.each`
  a | b | expected
  0 | 1 | 1
  1 | 1 | 2
`(({a, b, expected}) => {
  expect(a + b).toEqual(expected);
});

(notice the destructuring matches the names in the headers)

jest-each allows us to define custom test names, which I think is a good idea. If not, we basically have this:

[
  [1, 1, 2],
  [1, 2, 3],
  [2, 1, 3],
].forEach(([a, b, expected]) => {
  test(`${a} and ${b} is ${expected}` => {
    expect(a + b).toBe(expected);
  });
})

I think the big benefit with supporting it out of the box in Jest is that we can provide static analysis of it, and maybe better messages. We might even be able to hook it into fuzzy testing or other improvements.

It might just boil down to simple sugar around [].forEach and a template string, but encouraging the pattern by having built-in support might improve developer’s test suite.

---

There is also babel-plugin-gwt, but I don’t think introducing custom syntax is something we want to do in Jest core. If we do implement this feature though, that plugin can target whatever mechanism we come up with here, for a potentially better interop.

/cc @mattphillips

[mattphillips]

I second this! Data driven testing is something I’ve wanted in JS (especially in Jest) for a while.

As you can see I’ve tried to make it possible with jest-each and babel-plugin-gwt, it’s hard to find the right balance between being intuitive, not verbose and expressive enough to make dynamic, but unique test titles.

I really like the suggestion of using tagged template literals to try to mimic Spocks data tables API - without the need to do it at a compiler level like in babel-plugin-gwt.

My only concern with using tagged template literals is everything will be a string and the underlying library (possibly Jest) will have to convert them into actual types.

I’ve just had a play with jest-each (over here) to change its API to use tagged template strings, with one requirement - all data must be passed in as interpolations. This way the library doesn’t care what the data is. Here is an example:

describe('.add', () => {
  each`
    a    | b    | expected
    ${0} | ${0} | ${0}
    ${0} | ${1} | ${1}
    ${1} | ${1} | ${2}
  `.test('returns $expected when given $a and $b', ({ a, b, expected }) => {
    expect(a + b).toBe(expected);
  });
});
↓↓↓↓↓↓↓↓↓↓

I’d be more than happy to build this as part of Jest’s core, if approved and once an API is agreed 😃

[SimenB]

#6102