[🐛 Bug]: Performance regression of Selenium 4.4

What happened?

Hi, I upgraded Selenium server to 4.4, and now I constantly get 5x or even over 20x performance decrease in comparison to 4.3.

This happens on our Github Actions build, and is even more significant on my local Linux machine (without Docker).

This is a typical result for set of ~15 tests ( = 15 new session requests followed by few simple commands) on GH Actions:

Locally, the difference much more significant: from 5 seconds with 4.3.0 to 2 minutes with 4.4.0 (24x slower).

The test run is: https://github.com/OndraM/php-webdriver/actions/runs/2943219537

From what I noted from the Selenium log, with version 4.4, there is always a big delay before line Session request received by the Distributor, for example

13:35:01.350 DEBUG [DefaultChannelPool$IdleChannelDetector.run] - Closed 1 connections out of 1 in 0 ms
13:35:15.392 INFO [LocalDistributor.newSession] - Session request received by the Distributor: 

note the 15 seconds of no logged operation. While with 4.3.0 the events follow almost instantaneously, for example:

13:34:21.646 DEBUG [RequestConverter.channelRead0] - Closing input pipe.
13:34:21.661 INFO [LocalDistributor.newSession] - Session request received by the Distributor: 

For the tests, PHP language bindings for WebDriver are used - but the handshake and everything should be W3C WebDriver compatible, and was working OK with Selenium 4.3.0.

Please let me know if I can provide any more information or somehow help tracing the issue.

How can we reproduce the issue?

Run Github actions on branch https://github.com/OndraM/php-webdriver/tree/test/selenium-4-performance

Or run locally:

1. git clone -b test/selenium-4-performance [e-mail]:OndraM/php-webdriver.git
2. cd php-webdriver/
3. composer install
4. (in new terminal) java -jar selenium-server-4.4.0.jar standalone
5. (in new terminal) php -S 127.0.0.1:8000 -t tests/functional/web/
6. vendor/bin/phpunit --filter WebDriverByTest -v --debug

Relevant log output

I enabled the finest log level, here are the test runs:

• output for build with Selenium 4.3
• output for build with Selenium 4.4

(Selenium server output is dumped at the end, in the “Prints log” part - search for cat ./logs/selenium-server.log)

Operating System

Ubuntu (Github Actions), Arch (locally)

Selenium version

4.4.0 (jar server), php-webdriver 1.12.1

What are the browser(s) and version(s) where you see this issue?

Chrome 104 (headless)

What are the browser driver(s) and version(s) where you see this issue?

Chromedriver 104.0.5112.79

Are you using Selenium Grid?

4.4.0 in standalone mode

[AB-xdev]

So while comparing the changes between 4.3.0 and 4.4.0 I noticed that the DEFAULT_RETRY_INTERVAL was changed from 0 to 15 (was done in this commit)

This exactly matches the delay observed in the issue which is always around 15s

So just for fun I set the --session-retry-interval to 0 (as it was before) however the problem was still present. When however setting it to 1 I was able to get a Server/Grid response within 2-3s instead of 15s.

So for now using --session-retry-interval 1 looks like a workaround.

[cbonte]

@brondsem you can provide it with the SE_OPTS environment variable