[🐛 Bug]: Performance of Selenium 4 is significantly slower than 3.141.59

What happened?

As discussed in https://github.com/SeleniumHQ/docker-selenium/issues/1444, the time to init a session is significantly slower in Selenium 4.X than it is in version 3.141.59.

In my instance, with 4.1.1 it is anywhere between 4 and 10 times slower than 3.141.59 to initialise a new session.

When resetting sessions between tests, this adds up to a significant amount and radically slows down test runs. In our case w e have around 12,000 tests. When

How can we reproduce the issue?

I've created a repository to demonstrate the performance to just initialise the session: https://github.com/andrewnicols/selenium-1444

Relevant log output

➜  selenium-1444 git:(main) node index.js
Starting runs for chrome 3.141.59 against http://localhost:5444/wd/hub
==================================================
	Completed run for chrome 3.141.59 against http://localhost:5444/wd/hub
	Average run time for 50 runs: 599.647109117508
Starting runs for chrome 4.1.1 against http://localhost:4444/
==================================================
	Completed run for chrome 4.1.1 against http://localhost:4444/
	Average run time for 50 runs: 4900.971164139509
Starting runs for firefox 3.141.59 against http://localhost:5444/wd/hub
==================================================
	Completed run for firefox 3.141.59 against http://localhost:5444/wd/hub
	Average run time for 50 runs: 1001.9559491628409
Starting runs for firefox 4.1.1 against http://localhost:4444/
==================================================
	Completed run for firefox 4.1.1 against http://localhost:4444/
	Average run time for 50 runs: 4622.045899059176

Operating System

MacOS Monterey 12.1

Selenium version

4.1.1

What are the browser(s) and version(s) where you see this issue?

Chrome 97.0.4692.71, Firefox 95.0.2

What are the browser driver(s) and version(s) where you see this issue?

ChromeDriver 97.0.4692.71 (adefa7837d02a07a604c1e6eff0b3a09422ab88d-refs/branch-heads/4692@{#1247}), geckodriver 0.30.0 (d372710b98a6 2021-09-16 10:29 +0300)

Are you using Selenium Grid?

No, but I believe this is also experienced with Grid (See linked issue)

[diemol]

Thanks @andrewnicols for providing a way to validate changes!

We have adjusted the default value of this parameter to zero, which was the value used in Grid 3. More reasoning can be seen in the commit message related to this issue.

After running a few sessions using the sample repo, here are some results:

> repeat 2 { node index.js ; sleep 5 }                                                                                                                                                          

Starting runs for chrome 3.141.59 against http://localhost:5444/wd/hub
==================================================
	Completed run for chrome 3.141.59 against http://localhost:5444/wd/hub
	Average run time for 50 runs: 590.4847569346427
Starting runs for chrome 4.1.1 against http://localhost:4444
==================================================
	Completed run for chrome 4.1.1 against http://localhost:4444
	Average run time for 50 runs: 582.5350178790093
Starting runs for firefox 3.141.59 against http://localhost:5444/wd/hub
==================================================
	Completed run for firefox 3.141.59 against http://localhost:5444/wd/hub
	Average run time for 50 runs: 1578.969312787056
Starting runs for firefox 4.1.1 against http://localhost:4444
==================================================
	Completed run for firefox 4.1.1 against http://localhost:4444
	Average run time for 50 runs: 1547.3766356134415


Starting runs for chrome 3.141.59 against http://localhost:5444/wd/hub
==================================================
	Completed run for chrome 3.141.59 against http://localhost:5444/wd/hub
	Average run time for 50 runs: 579.2586772346497
Starting runs for chrome 4.1.1 against http://localhost:4444
==================================================
	Completed run for chrome 4.1.1 against http://localhost:4444
	Average run time for 50 runs: 584.8253180217743
Starting runs for firefox 3.141.59 against http://localhost:5444/wd/hub
==================================================
	Completed run for firefox 3.141.59 against http://localhost:5444/wd/hub
	Average run time for 50 runs: 1567.552846300602
Starting runs for firefox 4.1.1 against http://localhost:4444
==================================================
	Completed run for firefox 4.1.1 against http://localhost:4444
	Average run time for 50 runs: 1577.1089442491532

I’ve executed this 50 times and now the values are roughly similar. We will check a few more code areas where other improvements can be done before closing this.

[diemol]

After the initial change was made we saw that CPU usage was going up, so we made a couple of changes more to address that situation.

I will close this for now and we will probably do a 4.1.2 release by the end of the week or latest at the beginning of next week (given that we are also fixing other couple of bugs).