E2E tests ignore javascript errors

If a javascript error occurs during execution of an E2E test, it’s ignored and not surfaced to the console or logs. If it causes an assertion to fail (due to missing content, etc) the test will “fail” but you will have no way to see that the JS error caused it. If the assertions pass, everything looks good which is a really bad outcome.

I think chromedriver probably has a way to get logged errors, but if not the E2E tests should register an unhandled error handler on the window so they can propagate this out somehow (maybe into a hidden dom element that the test runner checks at the end).

An easy way to reproduce this is to shove a setTimeout that throws an error somewhere in the blazor code (this sample code is from a blazor bug I’m troubleshooting), like:

  private insertText(batch: RenderBatch, parent: LogicalElement, childIndex: number, textFrame: RenderTreeFrame) {
    let textContent = batch.frameReader.textContent(textFrame)!;
    if (typeof (textContent) !== "string") {
      textContent = "";
      window.setTimeout(() => {
        throw new Error(`Found non-string in batch text content with parent ${parent}`);
      }, 1);
    }
    const newTextNode = document.createTextNode(textContent);
    insertLogicalChild(newTextNode, parent, childIndex);
  }

This will produce an error in the browser console, but the test will not fail or dump the error to stdout/stderr, even if you sleep before your assertions to give time for the error to fire. If you turn the setTimeout into a regular throw, the test will fail.

[javiercn]

@kg we pipe this to the ITestOutput (or whatever is called) from xUnit. At the very least it shows up in Visual Studio.

[TanayParikh]

we normally don’t check for errors on the console to drive the outcome of tests

and in the rare cases that we do need the console output, it’s done as per below:

https://github.com/dotnet/aspnetcore/blob/7259991b5d2ecef36979fd55e48884de8d69f114/src/Components/test/E2ETest/ServerExecutionTests/ServerReconnectionTest.cs#L81-L82