[Bug]: Protocol error when calling puppeteer.connect()

Bug description

I am using the basic approach as set out in this post to connect from a client docker container to any one of a number of chrome docker containers (in a docker swarm/service, potentially across several servers behind nginx, deployed using CapRover).

In each chrome container I maintain a pool (just a simple array) of browser objects, and direct incoming requests to an appropriate browser as follows (very similar to the linked post):

import http from 'node:http';       // https://nodejs.org/api/http.html
import httpProxy from 'http-proxy'; // https://www.npmjs.com/package/http-proxy

const proxy = new httpProxy.createProxyServer({ ws: true });

// an array (pool) of pre-launched and managed browser objects...
const browsers = [ ... ];

http
  .createServer()
  .on('upgrade', (req, socket, head) => {
	  const browser = browsers[Math.floor(Math.random() * browsers.length)]; // in reality I don't just pick a browser at random
	  const target = browser.wsEndpoint();
	  proxy.ws(req, socket, head, { target });
  })
  .listen(3222);

The above is listening at ws://srv-captain--chrome:3222 (communication is “internal” over the docker network between containers).

Then, in my client container, I connect to the common endpoint ws://srv-captain--chrome:3222 as follows:

import puppeteer from 'puppeteer'; // https://www.npmjs.com/package/puppeteer (using version 17.1.3 at time of posting this)
try {
	const browser = await puppeteer.connect({ browserWSEndpoint: 'ws://srv-captain--chrome:3222' });
} catch (err) {
	console.error('error connecting to browser', err);
}

This works really well, except that I am getting occasional/inconsistent errors like these when calling puppeteer.connect() in the client container above:

Protocol error (Emulation.setDeviceMetricsOverride): Session closed. Most likely the page has been closed.
Protocol error (Performance.enable): Target closed.

Almost always, if I simply try to connect again, the connection is made without further error, and at the first attempt.

I have no idea why the error is complaining that the page has been closed or Target closed since, at this point in the process, I’m not attempting to interact with any page, and I know from listening for browser.on('disconnected'...), and also monitoring the chromium processes themselves, that each browser in the array is still working fine… none has crashed.

Any idea what’s going on here?

---

UPDATE after further testing

Of course, in the client container we don’t connect to a browser just for the sake of it, like in the above snippet, but to open a page and do some stuff with the page. In practice, in the client container it’s more like the following test snippet:

const doIteration = function (i) {
	return new Promise(async (resolve, reject) => {
		// mimic incoming requests coming in at random times over a short period by introducing a random initial delay...
		await new Promise(resolve => setTimeout(resolve, Math.random() * 5000));
		// now actually connect...
		let browser;
		try {
			browser = await puppeteer.connect({ browserWSEndpoint: `ws://srv-captain--chrome:3222?queryParam=loop_${i}` });
		} catch (err) {
			reject(err);
			return;
		}
		// now that we have a browser, open a new page...
		const page = await browser.newPage();
		// do something useful with the page (not shown here) and then close it..
		await page.close();
		// now disconnect (but don't close) the browser...
		browser.disconnect();
		resolve();
	});            
};

const promises = [];
for (let i = 0; i < 15; i++) {
	promises.push( doIteration(i) );
}

try {
	await Promise.all(promises);
} catch (err) {
	console.error(`error doing stuff`, err);
}

Each iteration above is being performed multiple times concurrently… I am using Promise.all() on an array of iteration promises to mimic multiple concurrent incoming requests in my production code. The above is enough to reproduce the problem… the error doesn’t happen on calling puppeteer.connect() with every iteration, just some.

So there seems to be some sort of interplay between opening/closing a page in one iteration, and calling puppeteer.connect() in another, despite closing the page and disconnecting the browser properly in each iteration? This probably also explains the Most likely the page has been closed error message when calling puppeteer.connect() if there is some hangover relating to a page closed in another iteration… though for some reason this error occurs when calling puppeteer.connect()?

---

With the use of a pool of browser objects in the browsers array, and a docker swarm having multiple containers on multiple servers, each upgrade message could be received at a different container (which could even be on a different server) and could be routed to a different browser in the browsers array. But I now think that this is a red herring, because in the further testing I narrowed the problem down by routing all requests to browsers[0] and also scaling the service down to just one container… so that the upgrade messages are always handled by the same container on the same server and routed to the same browser… and the problem still occurs.

---

Full stacktrace for the above-mentioned error:

Error: Protocol error (Emulation.setDeviceMetricsOverride): Session closed. Most likely the page has been closed.
	at CDPSession.send (file:///root/workspace/myclientapp/node_modules/puppeteer/lib/esm/puppeteer/common/Connection.js:281:35)
	at EmulationManager.emulateViewport (file:///root/workspace/myclientapp/node_modules/puppeteer/lib/esm/puppeteer/common/EmulationManager.js:33:73)
	at Page.setViewport (file:///root/workspace/myclientapp/node_modules/puppeteer/lib/esm/puppeteer/common/Page.js:1776:93)
	at Function._create (file:///root/workspace/myclientapp/node_modules/puppeteer/lib/esm/puppeteer/common/Page.js:242:24)
	at runMicrotasks (<anonymous>)
	at processTicksAndRejections (node:internal/process/task_queues:96:5)
	at async Target.page (file:///root/workspace/myclientapp/node_modules/puppeteer/lib/esm/puppeteer/common/Target.js:123:23)
	at async Promise.all (index 0)
	at async BrowserContext.pages (file:///root/workspace/myclientapp/node_modules/puppeteer/lib/esm/puppeteer/common/Browser.js:577:23)
	at async Promise.all (index 0)

Puppeteer version

17.1.3

Node.js version

16.17.0

npm version

8.15.0

What operating system are you seeing the problem on?

Linux

Relevant log output

No response

[OrKoN]

It does look like a bug, I will try to reproduce it, thanks for the script!

[OrKoN]

@drmrbrewer calling browser.pages() should not fail anymore but the number of pages returned will depend on how many pages are actually open in the browser at the moment. I don’t think there is anything to be done about it except for using the browserContext.pages() and creating a new browsing context for a concurrent connection.