AssertionError when intercepting requests on same url

Steps to reproduce

Tell us about your environment:

• Puppeteer version: 1.4.0
• Platform / OS version: MacOS High Sierra
• URLs (if applicable): any
• Node.js version: 8.9.4

What steps will reproduce the problem? I have a pool of puppeteer browsers running (a minimum of 2). I recently added a request interceptor to filter out ad urls. When performing multiple requests to the same url, usually happens on the 3rd time, an AssertionError is generated for all filtered requests: AssertionError [ERR_ASSERTION]: Request is already handled!. The strack trace points to the interceptedRequest.continue() line of my page.on('request', (interceptedRequest) => {}) function in the api route below. Further on in the strack trace points to node_modules/puppeteer/lib/NetworkManager.js:220:10 and node_modules/puppeteer/lib/NetworkManager.js:169:12 while also showing an unhandled promise rejection error

Please include code that reproduces the issue.

// api route using the pool

const browserPagePool = require("../utils/browserPagePool");

router.get("/api/test", async (req, res, next) => {
  try {

    const page = await browserPagePool.acquire();

    await page.setRequestInterception(true);
    page.on('request', (interceptedRequest) => {
      if ( isAdUrl(interceptedRequest.url()) ) {
        interceptedRequest.abort();
      } else {
        interceptedRequest.continue();
      }
    });

    await page.goto(url, { waitUntil: "networkidle0", timeout: 10000 });
    const content = await page.content()
    await browserPagePool.release(page); 

    return res.status(200).json(content);

  } catch (err) {
    console.error(err);
  }
});

// browserPagePool.js

const genericPool = require("generic-pool");
const puppeteer = require("puppeteer");
const browserPromise = async () => await puppeteer.launch({ headless: true });

const factory = {
  create: async function() {
    const browser = await browserPromise();
    return browser.newPage();
  },
  destroy: function(puppeteer) {
    return puppeteer.close();
  }
};

const browserPagePool = genericPool.createPool(factory, {
  max: 10,
  min: 2,
  maxWaitingClients: 50
});

module.exports = browserPagePool;

Should I somehow be clearing out previous requests from an already used puppeteer pool, or checking if a request is already completed in the interceptedRequest function, or something else? I’m not sure how to gracefully handle this.

[aslushnikov]

@lsbyerley if you’re pooling browser pages, you should un-subscribe your current event listeners from the page object before releasing the page back to the pool; otherwise, you re-subscribe multiple times to the event.

A quick and a little dirty fix would be to add removeAllListeners('request') call:

  page.removeAllListeners('request'); // remove listeners we were using for this task.
  await browserPagePool.release(page); 

Hope this helps.

[amitksingh1490]

you can check interceptedRequest._interceptionHandled to be false in order to avoid this error