[🐛 Bug]: Selenium runs the same one browser I requested once for each different browser I have installed

What happened?

1. I ran the JavaScript version of the “Putting everything together” script from the tutorial with selenium-webdriver version 4.4.0
2. When I ask it for chrome it runs the tests in Chrome twice, claiming that the second time is being run in Firefox.
3. When I ask it for firefox, it runs the tests in Firefox twice, claiming that the first time is being run in Chrome.
4. I can’t see any indication in the docs about how to either make it use both browsers or make it only run them once.

How can we reproduce the issue?

I just ran the example.

const { By, Builder } = require('selenium-webdriver');
const { suite } = require('selenium-webdriver/testing');
const assert = require("assert");

suite(function(env) {
    describe('First script', function() {
        let driver;

        before(async function() {
            driver = await new Builder().forBrowser('chrome').build();
        });

        after(async () => await driver.quit());

        it('First Selenium script', async function() {
            await driver.get('https://www.selenium.dev/selenium/web/web-form.html');

            let title = await driver.getTitle();
            assert.equal("Web form", title);

            await driver.manage().setTimeouts({ implicit: 500 });

            let textBox = await driver.findElement(By.name('my-text'));
            let submitButton = await driver.findElement(By.css('button'));

            await textBox.sendKeys('Selenium');
            await submitButton.click();

            let message = await driver.findElement(By.id('message'));
            let value = await message.getText();
            assert.equal("Received!", value);
        });

    });
});

I do have a slightly odd setup where I have chromedriver and geckodriver run inside the flatpak sandboxes the browsers are distributed through via wrapper scripts, but, in the process of discovering that I’d forgotten to run rehash in my Zsh to update the PATH cache, I also tried a tarballed firefox release and using manually started drivers with SELENIUM_BROWSER and SELENIUM_REMOTE_URL and I don’t remember there being any observable difference.

Relevant log output

% npm test

> test
> mocha

[INFO] Searching for WebDriver executables installed on the current system...
[INFO] ... located chrome
[INFO] ... located firefox
[INFO] Running tests against [chrome, firefox]


  [chrome]
    First script
      ✔ First Selenium script (1560ms)

  [firefox]
    First script
      ✔ First Selenium script (1028ms)


  2 passing (13s)

Operating System

Kubuntu Linux 20.04 LTS with the Flathub PPA installed

Selenium version

JavaScript 4.4.0

What are the browser(s) and version(s) where you see this issue?

Mozilla Firefox 105.0.1 through the org.mozilla.firefox Flatpak and Chromium 105.0.5195.125 through the com.github.Eloston.UngoogledChromium Flatpak

What are the browser driver(s) and version(s) where you see this issue?

geckodriver 0.31.0 freshly installed via cargo install geckodriver and chromedriver 105.0.5195.52, freshly downloaded from the site

Are you using Selenium Grid?

No response

[ssokolow]

The bug is that, without SELENIUM_BROWSER it reports finding two browsers and then runs the same browser twice, claiming to be running two different browsers in its terminal output.

[INFO] Searching for WebDriver executables installed on the current system...
[INFO] ... located chrome
[INFO] ... located firefox
[INFO] Running tests against [chrome, firefox]


  [chrome]
    First script
      ✔ First Selenium script (1631ms)

  [firefox]
    First script
      ✔ First Selenium script (1470ms)


  2 passing (13s)

That output results from Firefox popping up twice.

…so, in effect, there are two sub-bugs:

1. The terminal output doesn’t match the browsers it’s actually running.
2. It needlessly runs the test suite multiple times under the same browser because it thinks it’s running once per distinct browser.

EDIT: also, SELENIUM_BROWSER=chrome,firefox errors out with Error: Do not know how to build driver: chrome,firefox; did you forget to call usingServer(url)?

The only way I can get the desired behaviour is for X in chrome firefox; do SELENIUM_BROWSER=$X npm test; done and then manually looking at the two independent sets of terminal output.

(Mocha isn’t installed globally, so having npm test set to mocha test.js is more concise than manually running ./node_modules/mocha/bin/_mocha test.js.)

[ssokolow]

if chromedriver/geckodriver is set in path or download via npm etc, it should be able to find them!

It finds them just fine with SELENIUM_BROWSER=chrome or SELENIUM_BROWSER=firefox, so my wild guess is that it’s some kind of parsing issue where it thinks chrome,firefox is a single browser name rather than a comma-delimited list.

That would also make sense in the context of the error message asking if I forgot to manually set an explicit server URL.