[BUG] Calling waitForNavigation twice doesn't work

Context:

• Playwright Version: 0.11.1
• Operating System: macOS
• Extra: Node 12.14.1

Code Snippet Here’s a minimal repro of the behavior I’m seeing.

const { chromium, webkit, firefox } = require('playwright');

(async () => {
  const browser = await chromium.launch({ headless: false });
  const page = await browser.newPage();

  await page.goto('https://www.behance.net/search');
  // await page.waitForNavigation({ waitUntil: 'networkidle0' });

  // temporary - to ensure JS bindings are installed so that the history API will always be used for the second navigation
  await page.waitFor(2000);

  await Promise.all([
    page.click('[data-ut="moodboards-tab"]'),
    page.waitForNavigation({ waitUntil: 'networkidle0' }),
  ]);

  await page.screenshot({path: 'repro.png'});

  await browser.close();
})();

Describe the bug

This is a follow-up to https://github.com/microsoft/playwright/issues/996.

It seems that when I call waitForNavigation twice in the same context, the second call doesn’t work. In this case the second call is a history API navigation event, so that could be part of the reason too.

When I go to a page without any extra waiting, then click a link to navigate and wait for that navigation, everything works fine. But when I also try to wait for the first navigation, the second wait resolves immediately without the page being fully loaded. I’ve tried a couple of variations of this, including passing waitUntil: 'networkidle0' } into page.goto instead of calling waitForNavigation, and using a Promise.all for the first navigation too. All of these produce the same result.

Perhaps there’s something I’m misunderstanding about how waitForNavigation works? The docs seem to indicate that history API changes are considered relevant navigation events too.

When I don’t waitForNavigation as part of the first page.goto, the wait for the second navigation works correctly:

When I do waitForNavigation as part of the first page.goto, the wait for the second navigation doesn’t work:

Thanks for taking a look!

[dgozman]

We decided that current behavior is fine. The right way to use “network idle” in this case is to ensure that all requests started after the action has been finished. There needs to be some wrapper that listens to request, requestfinished and requestfailed events during the action. Something along these lines:

async function wrapWithNetworkIdle(page, action) {
  let networkSettledCallback;
  const networkSettledPromise = new Promise(f => networkSettledCallback = f);

  let requestCounter = 0;
  const onRequest = () => ++requestCounter;
  const onRequestDone =() => {
    // Let the page handle responses asynchronously (via setTimeout(0)).
    const evaluate = page.evaluate(() => new Promise(f => setTimeout(f, 0)));
    evaluate.catch(e => null).then(() => {
      if (!--requestCounter)
        networkSettledCallback();
    });
  };

  page.on('request', onRequest);
  page.on('requestfinished', onRequestDone);
  page.on('requestfailed', onRequestDone);

  const result = await action();
  await networkSettledPromise;
  
  page.removeListener('request', onRequest);
  page.removeListener('requestfinished', onRequestDone);
  page.removeListener('requestfailed', onRequestDone);

  return result;
}

We’ll experiment with this later.

[dgozman]

I think the issue here is that pushState does not reset networkidle detection, and so the second waitForNavigation call just resolves immediately after the pushState. That sounds like a bug.