Will this "just work" at some point, without having to say which browser to use?

All I care about is that my code works in all browsers, so I’d expected a cross-browser testing setup to offer a clean “run this code, and every one of the supported and installed browsers will do so, yielding separate screenshots and logs for each” despite using a single test script.

Is that on the roadmap? If not, can that please be on the roadmap? Because otherwise all this automates is the installation of the headless browsers themselves. The actual hassle, namely writing separate files and running separate tasks just to test the same thing, is still right there.

Rather than three scripts, the usefulness comes from being able to write this, instead:

const pw = require('playwright');

// zero-conf with all supported browsers enabled: updating pw simply
// gets you more targets as they get added, for free, with zero work.
// The ideal setup for 99% of folks who need to write tests.
const instance = pw.createInstance({ 
  ...
  // but if you're in the 1% of folks who need to target fewer than "all browsers",
  // you get to configure things yourself. And having a "browsers" conf option
  // means that rather than all-enabled, things should be all-disabled, only enabling
  // those browsers that have a truthy conf value, because now you need to make
  // sure new engines *don't* kick in for free:
  browsers: {
    chrome: true, // true/falsey or object with more specific props - keep it simple until it needs to be complicated
    firefox: true
  },
  ...
});

(async () => {
  // plural because we want to target all browsers, not "write lots
  // of code for each separate browser":
  const browsers = await instance.startBrowsers();
  // start testing stuff.
  const page = await browsers.newPage('https://maps.google.com');
  // start performing tasks and waiting for those tasks to complete
  // in _all_ browsers before moving on.
  await page.click('text="Your location"');
  // take as many screenshots as there are browsers running
  await page.screenshot({ path: 'initial-click.png' });
  await browsers.close();
})();

[jperl]

@pomax those are really good points. Build the create-react-app of browser testing is exactly what I am trying to do with qawolf. It is an opinionated way to get up and running with browser tests without configuration. It sets up Puppeteer (working on switching to Playwright) and Jest with one command based on your browser actions, and sets up CI with one command.

One idea based on your thoughts, is that we could make a flag that allows you to run tests on all browsers. npx qawolf test --all-browsers.

I don’t want to hijack this thread, so I reached out to your listed github email because I would love to get your thoughts on how to make it easier to start testing.

[jperl]

@pomax One possible solution is that you can create a launch helper in your codebase which takes the browser as an option and pass the browser as an environment variable. So you could get similar behavior just by calling npm test multiple times.

Ex.

in code

browser = await launch({ browser: process.env.BROWSER || "chrome" });

in CI

BROWSER=chrome npm test
BROWSER=firefox npm test
BROWSER=webkit npm test

FWIW, that is how I plan to handle it in qawolf.