Support for custom selector engines (Querying nested shadow roots)

What is this? Other tools offer the ability to provide a custom engine selecting elements in the DOM.

(example from playwright)

  await playwright.selectors.register(selectorEngine, { name: 'shadow' })
  await page.waitForSelector('shadow=#no-downloads span', {timeout: 3000}

I’ve seen https://github.com/puppeteer/puppeteer/issues/382 but i’m not sure this offers such an easy mechanism thats really simple for end users. I’m happy to be wrong on this, i couldn’t find any examples.

Playwright offers this. https://github.com/microsoft/playwright/blob/master/docs/api.md#selectorsregisterenginefunction-args

In Selenium world its things like this: https://chercher.tech/java/custom-locators-selenium-webdriver

Why is this useful? Traditionally these custom locators would be used to provide the ability to select elements via XPATH or JQuery selectors.

Why do i want this? I maintain: https://github.com/Georgegriff/query-selector-shadow-dom which allows users to write css selectors that automatically pierce web component shadow roots and it was trivial to add support in Playwright to use my library as a selector engine. Like so:

const { selectorEngine } = require("query-selector-shadow-dom/plugins/playwright");
const playwright = require('playwright')
  await playwright.selectors.register(selectorEngine, { name: 'shadow' })

  const browser = await playwright.chromium.launch({ headless: false})
  const context = await browser.newContext({ viewport: null })
  const page = await context.newPage()

  await page.goto('chrome://downloads')

  await page.waitForSelector('shadow=#no-downloads span', {timeout: 3000})
  await new Promise(resolve => setTimeout(resolve, 3000))   

  await page.close()
  await context.close()
  await browser.close()

Registering this engine allows users to use click waitForSelector and thing that accepts a selector to use my library to automatically pierce shadow roots.

How is my engine implemented in playwright?

Playwright defines this interface: https://github.com/microsoft/playwright/blob/master/docs/api.md#selectorsregisterenginefunction-args which accepts a Function/String They will take your function and pass into into the browser context and handle the rest for you so you can use the engine for click etc.

My library implements this interface: https://github.com/Georgegriff/query-selector-shadow-dom/blob/master/plugins/playwright/index.js (It does this a little strangely using string because i need to inject my library into the function scope)

[paullewis]

Hey, so we now have an experimental API that lets you do this (on master). Roughly it looks like this:

// Custom query handler.
const doesNotHaveClass = 
    (element, className) => element.querySelectorAll(`:not(.${className})`);

// Register it.
puppeteer.__experimental_registerCustomQueryHandler('doesNotHaveClass', 
    doesNotHaveClass);

// Prepend queries with the name of the handler.
const elements = await page.$$('doesNotHaveClass/foo');

We have the following APIs:

__experimental_registerCustomQueryHandler(name: string, queryHandler: QueryHandler): void;
__experimental_unregisterCustomQueryHandler(name: string): void;
__experimental_customQueryHandlers(): Map<string, QueryHandler>;
__experimental_clearQueryHandlers(): void;

Where QueryHandler is a relatively generic term for a function of the form:

(element, selector) => Element | Element[] | NodeListOf<Element>

Other points of note:

• It’s experimental (so it might change!)
• You can only register one function for a given name, and names can only contain [a-zA-Z]
• You can register or invoke a function that doesn’t follow the expectations of $, $$, $$eval, or waitFor{Selector}, and you will either get unexpected outcomes or an error. In short we don’t check that the query handler you invoke is going to do what you expect 😃

[Georgegriff]

Been experimenting with the updated API in 5.2.0 its working great! Was able to implement the following with ease: https://github.com/Georgegriff/query-selector-shadow-dom/pull/36/files

QueryHandler implementation: https://github.com/Georgegriff/query-selector-shadow-dom/pull/36/files#diff-1297c36120ceed6b61d83df8075cc959