[REGRESSION]: chromium.connect does not work with vanilla CDP servers anymore

Hi there, apologies if this has been asked before but I didn’t find anything.

Up until v1.3.0 it was possible to .connect to a vanilla browser websocket URL (either created through launching Chrome with --remote-debugging) or using Puppeteer).

v1.4.0 introduced the new “New Client / Server Wire protocol”, since then it’s not possible to use .connect with other original CDP websocket servers anymore (it will hang here). I also noticed through logging that the new wire protocol looks much different from the regular CDP chatter.

My question would be if there’s still a way to use chromium.connect with the traditional CDP websocket URLs or if there are pointers on how to best write a translation layer to accomplish that.

My use-case is to have a single websocket url which exposes a fleet of chromium browsers by proxying the websocket connections (similar to browserless.io) and both puppeteer & playwright being able to connect to it and control the browsers.

The following worked prior to v1.4.0:

const puppeteer = require("puppeteer")

puppeteer
  .launch({
    headless: true,
    defaultViewport: null,
    args: [
      "--remote-debugging-port=9222",
      "--remote-debugging-address=0.0.0.0",
    ],
  })
  .then(async (browser) => {
    console.log(browser._connection.url())
  })
// => ws://0.0.0.0:9222/devtools/browser/490a2b32-ce62-4b6b-a530-d4931fdeb046

const pw = require("playwright")

;(async () => {
  const browser = await pw.chromium.connect({
    wsEndpoint: "ws://0.0.0.0:9222/devtools/browser/490a2b32-ce62-4b6b-a530-d4931fdeb046",
  })

  const context = await browser.newContext()
  const page = await context.newPage()

  await page.goto("https://www.example.com/")
  await page.screenshot({ path: "example.png" })

  await browser.close()
})()

[Niek]

@pavelfeldman Not OP, but I want to highlight that there are various reasons to connect to existing Chrome CDP websockets that are not launched by Playwright. For example, if you have a scalable Chrome (e.g. k8s) cluster that already works with Puppeteer/Playwright. Or if you use Playwright with a third party infra solution like Browserless (see comment above of @joelgriffith) or HeadlessTesting.

It would be awesome if there would be a way to call launchServer() with an existing CDP websocket for these usecases.

[joelgriffith]

@pavelfeldman thanks for going over this. I think the things you’ve outlined make a lot of sense, in particular the network traffic (an almost 10x drop in network chatter is huge). This is a big one for us.

For us, the problem is maintenance burden as we support playwright, puppeteer, selenium as well as our own REST APIs. Adding the new playwright server was quick, but over the long term the cost does add up, so it’s not a trivial change. As much as the change was a bit of a shock, I think it’s worth the burden given what you’ve said.

A few things that would be nice:

• Support for user-data-dir in browser-server. Not sure what the complications are there since it’s not part of the current API.
• Allow for a newContext event so we can inject network interceptors for ad-blocking in our stack: https://github.com/browserless/chrome/blob/master/src/chrome-helper.ts#L139. There’s a few other things as well here, but it sounds like playwright takes care of them mostly, so just this ad-blocker would be nice.