RFC: Suite Context

So I’ve been thinking about adding “context” or “state” to suites…

In any test runner, the motive behind before, after, before.each, and after.each hooks (and their equivalents) is to setup or tear down individual tests or entire suites. Of course, uvu allows this pattern, but I think we can make it even better.

When setting up or tearing down environment values, you often make the thing(s) you just set up accessible to your tests. For example, this may be a DB table record, a HTTP client, etc – it doesn’t matter what the thing is, but with uvu – at least currently – you’re forced to share/write into a scope-hoisted variable(s):

Before

const User = suite('User');

let client, user;

User.before(async () => {
  client = await DB.connect();
});

User.before.each(async () => {
  user = await client.insert('insert into users ... returning *');
});

User.after.each(async () => {
  await client.destroy(`delete from users where id = ${user.id}`);
  user = undefined;
});

User.after(async () => {
  client = await client.end();
});

User('should not have Teams initially', async () => {
  const teams = await client.select(`
    select id from users_teams 
    where user_id = ${user.id};
  `);

  assert.is(teams.length, 0);
});

// ...

User.run();

This is fine – and it obviously works; however, it can be messy once you have multiple suites in the same file, or if you’ve abstracted suite hooks and want to use them across multiple locations.

Instead, I’m thinking that hooks should affect a state or context value (name TBD) that any Suite hooks can affect/mutate. Then the final state value is passed into each of the Suite’s tests.

This pattern alone will allow hooks to operate independently and clean up top-level variables. My only concerns are that:

• it may be too easy to mutate state too freely, which may unintentionally affect later tests
• it may be less obvious that your side effects need to be cleaned up

After

const User = suite('User');

User.before(async context => {
  context.client = await DB.connect();
});

User.before.each(async context => {
  context.user = await context.client.insert('insert into users ... returning *');
});

User.after.each(async context => {
  await context.client.destroy(`delete from users where id = ${user.id}`);
  context.user = undefined;
});

User.after(async context => {
  context.client = await context.client.end();
});

User('should not have Teams initially', async context => {
  const { client, user } = context;

  const teams = await client.select(`
    select id from users_teams 
    where user_id = ${user.id};
  `);

  assert.is(teams.length, 0);
});

// ...

User.run();

Again, we may want to reuse our DB.connect and DB.destroy helpers – so now we can do that:

import * as $ from './helpers';

const User = suite('User');

User.before($.DB.connect);
User.after($.DB.destroy);

// Still keep User-specific before.each/after.each hooks here

// ...

The last component of this is that suite() itself could accept an initial state/context value. TypeScript users will also be able to declare the expected Context/State type information this way:

interface Context {
  client?: DB.Client;
}

interface UserContext extends Context {
  user?: IUser;
}

const User = suite<UserContext>('User', {
  client: undefined,
  user: undefined,
});

// ...

// The `context` would be typed
User.before.each(async context => {
  context.user = await context.client.insert('insert into users ... returning *');
});

// ...

// The `context` would also be typed
User('should not have Teams initially', async context => {
  const { client, user } = context;

  const teams = await client.select(`
    select id from users_teams 
    where user_id = ${user.id};
  `);

  assert.is(teams.length, 0);
});

---

I already have a local branch that fully supports these examples. AKA, this is done & working. I’m just looking for feedback before committing it.

---

If you made it this far, thank you! 😅 🙌

Please let me know:

1. Is this a good idea?
2. Do we call this “state” or “context”?
3. With regard to test modifications (accidentally) leaking into subsequent tests, how do we resolve that? Does it suffice to say “that’s your fault” or is there something better that can be done?

Thanks 🙇

[lukeed]

Thank you all for feedback. I’ve accepted this feature and will merge implementation tomorrow for a 0.3.0 release 🎉

I definitely echo @pngwn’s thoughts re: suite.every – thank you for the suggestion @giuseppeg but I’ll table this for now. Keep the ideas coming though 😃

Even during my examples, I found myself leaning towards Context. Given that there was no strong push for State here, I’ll continue with Context.

I’ll move ahead with a Proxy wrapper for context as it’s passed into each test. The Proxy will log a warning if a test tries to set a value. I won’t make warnings hide-able since turning them off just means you get a silent no-op. That would suck. This Proxy work isn’t done yet, but it’s pretty straightforward.

For the future parallelism, I’m planning on doing suite-level parallelism only. At least, that’s what I’ll be promising publicly. I think it is possible to have individually-parallelized tests, but:

1. I’m not sure if there’s really a gain to be had with that
2. this can be unexpected; eg, relied on test execution order
3. synchronizing test output would become a significant part of the uvu code base (ouch)
4. I don’t want to have multi “levels” of parallelism configuration/options to control this flow — eg; suite.serial vs suite.parallel and --parallel vs parallel-tests… etc. I really don’t want test files to change/refactor in order to change how they’re executed. That feels counter-intuitive.

[pngwn]

I’ll create an RFC.