feat(@angular/cdk/testing): Component Harness Feedback

This is an open discussion aiming to regroup Component Harness feedback and improvement ideas.

Most of the items below are focused on simplifying the API. The current APIs are a bit too complex; this can have a negative impact on TestHarness adoption.

For test authors

1. Easier access to harness

HarnessLoader is a nice abstraction but it can make harness instantiation cumbersome.

Actual approach

let fixture: ComponentFixture<MyDialogButton>;
let loader: HarnessLoader;
let rootLoader: HarnessLoader;

beforeEach(() => {
  fixture = TestBed.createComponent(MyDialogButton);
  loader = TestbedHarnessEnvironment.loader(fixture);
  rootLoader = TestbedHarnessEnvironment.documentRootLoader(fixture);
});

it('loads harnesses', async () => {
  const dialogButtonHarness = await TestbedHarnessEnvironment.harnessForFixture(fixture, MyDialogButtonHarness);

  const buttonHarness = await loader.getHarness(MyButtonHarness);
});

It would be nice to have faster harness access like:

Suggestion 1.A

let fixture: ComponentFixture<MyDialogButton>;

beforeEach(() => {
  fixture = TestBed.createComponent(MyDialogButton);
});

it('loads harnesses', async () => {
  const dialogButtonHarness = await TestbedHarnessEnvironment.getHarness(MyDialogButtonHarness, {fixture});

  const buttonHarness = await TestbedHarnessEnvironment.getHarness(MyButtonHarness, {fixture});
});

or even global functions like getHarness and getProtractorHarness functions could make the tests even more readable:

Suggestion 1.B

let fixture: ComponentFixture<MyDialogButton>;

beforeEach(() => {
  fixture = TestBed.createComponent(MyDialogButton);
});

it('loads harnesses', async () => {
  const dialogButtonHarness = await getHarness(MyDialogButtonHarness, {fixture});
  const buttonHarness = await getHarness(MyButtonHarness, {fixture});
});

For harness authors

2. LocatorFactory abstraction

The LocatorFactory approach (e.g. locatorFor() method returns a function that takes no parameters) can be confusing and cumbersome.

Actual approach

class MyPopupHarness extends ComponentHarness {
  static hostSelector = 'my-popup';

  protected getTriggerElement = this.locatorFor('button');

  async toggle() {
    const trigger = await this.getTriggerElement();
    return trigger.click();
  }

}

Suggestion 2.A

Simple accessor methods like get() or getOptional() seem easier to use and more intuitive.

class MyPopupHarness extends ComponentHarness {
  static hostSelector = 'my-popup';

  async toggle() {
    const trigger = await this.get('button');
    return trigger.click();
  }
}

We can let developers factorize the way they want:

getTriggerElement() {
  return this.get('button');
}

3. async / await vs chaining

I am personally not a big fan of chaining (a.k.a. builder pattern) but in cases like this one where we end up with lots of awaits, this can simplify the interface:

Actual approach

async isDisabled() {
  const el = await this.getMessageElement();
  const text = await el.text();
  return text === 'Disabled';
}

Suggestion 3.A

async isDisabled() {
  return (await this.getMessageElement().text()) === 'Disabled';
}

4. Trigger any event

TestElement should have a triggerEvent function that allows harness authors to trigger any event.

Suggestion 4.A

el.triggerEvent('dragenter', {})

Common

5. Provide synchronous functions

Some environments can query the DOM synchronously (e.g. TestbedHarnessEnvironment) or through some under the hood chaining (e.g. Cypress) (Cf. https://docs.cypress.io/guides/core-concepts/introduction-to-cypress.html#Chains-of-Commands). Harness authors might want to focus on these environments. In that case, they will want to use synchronous functions and keep tests and harnesses easier to read & write.

Current approach

class ItemListHarness extends ComponentHarness {
  static hostSelector = 'app-item-list';

  getItems = this.locatorForAll('li');

  async getItemNames() {
    const items = await this.getItems();
    const itemNames = await Promise.all(items.map(item => item.text()));
    return itemNames;
  }
}

it('loads harnesses', async () => {
  const itemListHarness = await loader.getHarness(ItemListHarness);
  expect(await itemListHarness.getItemNames()).toEqual(['🍔', '🍟']);
});

Suggestion 5.A

Providing synchronous alternatives to accessors.

class ItemListHarness extends ComponentHarness {
  static hostSelector = 'app-item-list';

  getItemNamesSync() {
    return this.getAllSync('li').map(item => item.textSync());
  }
}

it('loads harnesses', () => {
  const itemListHarness = loader.getHarnessSync(ItemListHarness);
  expect(itemListHarness.getItemNamesSync()).toEqual(['🍔', '🍟']);
});

6. TestbedHarnessEnvironment vs. TestBedHarnessEnvironment

TestbedHarnessEnvironment could be renamed to TestBedHarnessEnvironment to stay consistent with TestBed 😉

7. Merge TestBed and TestbedHarnessEnvironment

In some future, wouldn’t it be nice to merge TestbedHarnessEnvironment with TestBed which means moving test harness to the angular repo?

8. Cypress support

An external library could provide a CypressHarnessEnvironment but as presented in the 5th item, Cypress is based on an abstract chain of commands. TestElement doesn’t seem to be the right abstraction for this use case especially for getters like text(), getProperty() etc…

This is the last item on the list but probably the most important one. One of the key features of harnesses is the test environment abstraction and harness reuse through environments (TestBed, Protractor etc…) but if I am using TestBed and Cypress and if I can’t reuse my harnesses with Cypress then it somewhat defeats the purpose of harnesses.

[yjaaidi]

We (jscutlery) just came up with this library @jscutlery/cypress-harness to support harnesses on Cypress. This solves items 5 & 8.

The only remaining issue is item 3 which turned into https://github.com/angular/components/issues/21183 as 1 & 2 can be solved with adapters & helpers.

[mmalerba]

Yeah changing the instanceof sounds reasonable to me. Just be sure to include a comment explaining why we’re avoiding it.

That’s a pretty cool TestElement prototype. I’ll file an FR to consider adding it. Will need to discuss with the team before deciding if we want to do it, but it does seem like it could help a lot with readability of test code.