Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[Feature] Pause test (`page.pause()`) on failure

See original GitHub issue

Sometimes I ran Playwright test, the the test failed. The browser is closed before I can inspect it. The trace may not provide enough information (e.g. when debugging websockets or the test failed while waiting for a response from server).

When I run Playwright again with the inspector, I can no longer reproduce the problem.

Since E2E tests can be more prone to heisenbugs, I’d like to suggest having an option to pause the test when a test is failed — before the browser context is closed, so developers will have a chance to investigate what happened.

I have tried this:

test.afterEach(async ({ page }, testInfo) => {
  if (testInfo.status !== testInfo.expectedStatus) {
    await page.pause()
  }
})

However,

The inspector closes itself because 30 seconds has passed (due to test timeout).
Stack trace is not shown; there is no indication of what failed.

After a while I came up with this workaround…

// When running outside CI, pause Playwright when a test failed.
// See: https://github.com/microsoft/playwright/issues/10132
test.afterEach(async ({ page }, testInfo) => {
  if (!process.env.CI && testInfo.status !== testInfo.expectedStatus) {
    process.stderr.write(
      `❌ ❌ PLAYWRIGHT TEST FAILURE ❌ ❌\n${
        testInfo.error?.stack || testInfo.error
      }\n`,
    )
    testInfo.setTimeout(0)
    await page.pause()
  }
})

I wish this is available as a CLI flag.

Issue Analytics

State:
Created 2 years ago
Reactions:17
Comments:14 (4 by maintainers)

Top GitHub Comments

3reactions

s-h-a-d-o-wcommented, Jun 13, 2022

For me, overriding the timeout using testInfo.setTimeout(0) doesn’t work, playwright still exits. Is it just me?

If not, how does everybody deal with the fact that playwright will either freeze forever if timeout is 0 and something can’t be found or it will exit, preventing debugging.

Check out Playwright Test for VS Code plugin, it’ll stop on failure when debugging!

There are three problems with this:

Not everyone uses VS Code
The error isn’t output anywhere (While one can inspect the state, if you have an assertion that contains function calls, it’s… problematic)
No capability to specify environment variables (Again somewhat minor but… can be useful to e.g. use different test setups)

2reactions

dtinthcommented, Jan 6, 2022

@ddaypunk The trace viewer is very helpful in most cases. However, not everything is recorded in traces, e.g. WebSocket messages.

@dgozman Sorry for late reply.

Did you try running with inspector and clicking “resume” button so that the test goes to the end? Does this make the problem disappear so that you are unable to debug it?

Here’s a small repro.

import { test, expect } from '@playwright/test';

test('basic test', async ({ page }) => {
  // Let’s assume that in a real app, when clicking #button,
  // the #message would change to "Loading" and later "Completed".
  // However, in a rare case, the message would get stuck at "Loading".
  const html = `
    <!DOCTYPE html>
    <button id="button" onclick="message.textContent='Loading'">Click me</button>
    <p id="message">Message</p>
    `
  await page.goto('data:text/html,' + encodeURIComponent(html));
  await page.click('#button');
  await expect(page.locator('#message')).toHaveText('Completed');
});

I run this with:

env PWDEBUG=1 npx playwright test

Then clicked the “Resume” button.

I think we already pause at the end of a test if you run with inspector.

Apparently, when the test is run to the end, the browser is gone, regardless of whether the test failed or passed. Coming from Cypress, when the test finished running, you can still interact with the web page, and having that facility helped us in debugging failed test tremendously.

Why do you want to see the stack trace? Don’t you get a failed action in the inspector? I am not sure there is a good place to show this error, before the test finishes and test reporter picks it up.

If a Playwright action failed, it does show in the inspector:

However, a test can also fail due to other reasons. For example, I changed this:

-  await expect(page.locator('#message')).toHaveText('Completed');
+  throw new Error('Test failed because something else');

In this case, nothing would show up on the inspector.

Top Results From Across the Web

Protractor Pause On Failure - selenium webdriver

Is there a way to pause automatically on failure? I don't know in advance which test is going to fail, so now I'm...

Playwright Inspector and page.pause() (#214) - YouTube

Specifically, we take a look at how ` pause ()` and Playwright Inspector can help you to understand what's happening in your tests...

HTML DOM Audio pause() Method - W3Schools

Definition and Usage. The pause() method halts (pauses) the currently playing audio. Tip: This method is often used together with the play() method....

await - JavaScript - MDN Web Docs

await is usually used to unwrap promises by passing a Promise as the expression . Using await pauses the execution of its surrounding...

Pause test suite execution on test case fail - Katalon Community

If it paused, then I could troubleshoot at the point of failure, but because it ... We also need the Pause test suite...