Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Headless Chrome slows tests by 10x

See original GitHub issue

What are you trying to achieve?

I am trying to run my tests in headless Chrome using the --headless argument. I expect the tests to run in approximately the same amount of time as the usual runs, as running a clean webdriverio script runs faster headless than in a normal browser window.

What do you get instead?

The tests run about ten times as slow, using 44 seconds instead of 4 seconds.

$ time $(npm bin)/codeceptjs run --verbose > headless-output.txt

real	0m44.973s
user	0m0.992s
sys	0m0.068s

$ nvim codecept.json # removing --headless option

developer at developer-XPS-13-9360 in ~/dev/dx-nuflo/tests/acceptance (acceptance-testing)
$ time $(npm bin)/codeceptjs run --verbose > output.txt

real	0m4.879s
user	0m0.768s
sys	0m0.088s
verbose output from headless run 

CodeceptJS v0.6.3
Using test root "/home/developer/dev/dx-nuflo/tests/acceptance"

Top level-routes --
   [1] Starting recording promises
   Emitted | suite.before ([object Object])
   [1] Queued | hook WebDriverIO._beforeSuite()
 a page to add a new encounter should be rendered
 > [Session] Starting singleton browser session
   [2] Starting recording promises
   Emitted | test.before
   [2] Queued | hook WebDriverIO._before()
   Emitted | test.start ([object Object])
   Emitted | step.before (I am on page "/encounter/new")
   [2] Queued | amOnPage: "/encounter/new"
   Emitted | step.after (I am on page "/encounter/new")
   [2] Queued | return step result
   Emitted | step.before (I see "LOGG INN")
   [2] Queued | see: "LOGG INN"
   Emitted | step.after (I see "LOGG INN")
   [2] Queued | return step result
   [2] Queued | fire test.passed
   [2] Queued | finish test
   Emitted | step.start (I am on page "/encounter/new")
 • I am on page "/encounter/new"
   Emitted | step.start (I see "LOGG INN")
 • I see "LOGG INN"
   Emitted | test.passed ([object Object])
 ✓ OK in 22317ms

   Emitted | test.after
   [2] Queued | hook WebDriverIO._after()
 > [Session] cleaning cookies and localStorage
 fooScen
   [3] Starting recording promises
   Emitted | test.before
   [3] Queued | hook WebDriverIO._before()
   Emitted | test.start ([object Object])
   Emitted | step.before (I am on page "/encounter/new")
   [3] Queued | amOnPage: "/encounter/new"
   Emitted | step.after (I am on page "/encounter/new")
   [3] Queued | return step result
   Emitted | step.before (I see "FOOLOGG INN")
   [3] Queued | see: "FOOLOGG INN"
   Emitted | step.after (I see "FOOLOGG INN")
   [3] Queued | return step result
   [3] Queued | fire test.passed
   [3] Queued | finish test
   Emitted | step.start (I am on page "/encounter/new")
 • I am on page "/encounter/new"
   Emitted | step.start (I see "FOOLOGG INN")
 • I see "FOOLOGG INN"
   [3] Error | Error
   [3] Starting <teardown> session
   Emitted | test.failed ([object Object])
   [3] <teardown> Queued | hook WebDriverIO._failed()
   [3] <teardown> Queued | () => done(err)
   [3] <teardown> Stopping recording promises
 > Screenshot has been saved to /home/developer/dev/dx-nuflo/tests/acceptance/output/fooScen.failed.png
 ✖ FAILED in 11599ms

   Emitted | test.after
   [3] <teardown> Queued | hook WebDriverIO._after()
 > [Session] cleaning cookies and localStorage
   [4] Starting recording promises
   Emitted | suite.after ([object Object])
   [4] Queued | hook WebDriverIO._afterSuite()

-- FAILURES:

  1) Top level-routes: fooScen:

      expected web page to include "FOOLOGG INN"
      + expected - actual

      -Innlogging
      -Brukernavn
      -Passord
      -LOGG INN
      +FOOLOGG INN
      
  
  Scenario Steps:
  
  - I.see("FOOLOGG INN") at Test.Scenario (top-level-routes_test.js:18:5)
  - I.amOnPage("/encounter/new") at Test.Scenario (top-level-routes_test.js:14:5)
  
  Error
      at Object.<anonymous> (/home/developer/dev/dx-nuflo/node_modules/codeceptjs/lib/helper/WebDriverIO.js:1129:51)
      at Object.exec (/home/developer/dev/dx-nuflo/node_modules/webdriverio/build/lib/helpers/safeExecute.js:28:24)
      at Object.resolve (/home/developer/dev/dx-nuflo/node_modules/webdriverio/build/lib/webdriverio.js:189:29)
      at /home/developer/dev/dx-nuflo/node_modules/webdriverio/build/lib/webdriverio.js:538:32
      at _fulfilled (/home/developer/dev/dx-nuflo/node_modules/q/q.js:854:54)
      at self.promiseDispatch.done (/home/developer/dev/dx-nuflo/node_modules/q/q.js:883:30)


  FAIL  | 1 passed, 1 failed   // 34s
   Emitted | global.result ([object Object])


real	0m44.973s
user	0m0.992s
sys	0m0.068s
verbose output from normal (non-headless) run 
CodeceptJS v0.6.3
Using test root "/home/developer/dev/dx-nuflo/tests/acceptance"

Top level-routes --
   [1] Starting recording promises
   Emitted | suite.before ([object Object])
   [1] Queued | hook WebDriverIO._beforeSuite()
 a page to add a new encounter should be rendered
 > [Session] Starting singleton browser session
   [2] Starting recording promises
   Emitted | test.before
   [2] Queued | hook WebDriverIO._before()
   Emitted | test.start ([object Object])
   Emitted | step.before (I am on page "/encounter/new")
   [2] Queued | amOnPage: "/encounter/new"
   Emitted | step.after (I am on page "/encounter/new")
   [2] Queued | return step result
   Emitted | step.before (I see "LOGG INN")
   [2] Queued | see: "LOGG INN"
   Emitted | step.after (I see "LOGG INN")
   [2] Queued | return step result
   [2] Queued | fire test.passed
   [2] Queued | finish test
   Emitted | step.start (I am on page "/encounter/new")
 • I am on page "/encounter/new"
   Emitted | step.start (I see "LOGG INN")
 • I see "LOGG INN"
   Emitted | test.passed ([object Object])
 ✓ OK in 2748ms

   Emitted | test.after
   [2] Queued | hook WebDriverIO._after()
 > [Session] cleaning cookies and localStorage
 fooScen
   [3] Starting recording promises
   Emitted | test.before
   [3] Queued | hook WebDriverIO._before()
   Emitted | test.start ([object Object])
   Emitted | step.before (I am on page "/encounter/new")
   [3] Queued | amOnPage: "/encounter/new"
   Emitted | step.after (I am on page "/encounter/new")
   [3] Queued | return step result
   Emitted | step.before (I see "FOOLOGG INN")
   [3] Queued | see: "FOOLOGG INN"
   Emitted | step.after (I see "FOOLOGG INN")
   [3] Queued | return step result
   [3] Queued | fire test.passed
   [3] Queued | finish test
   Emitted | step.start (I am on page "/encounter/new")
 • I am on page "/encounter/new"
   Emitted | step.start (I see "FOOLOGG INN")
 • I see "FOOLOGG INN"
   [3] Error | Error
   [3] Starting <teardown> session
   Emitted | test.failed ([object Object])
   [3] <teardown> Queued | hook WebDriverIO._failed()
   [3] <teardown> Queued | () => done(err)
   [3] <teardown> Stopping recording promises
 > Screenshot has been saved to /home/developer/dev/dx-nuflo/tests/acceptance/output/fooScen.failed.png
 ✖ FAILED in 1299ms

   Emitted | test.after
   [3] <teardown> Queued | hook WebDriverIO._after()
 > [Session] cleaning cookies and localStorage
   [4] Starting recording promises
   Emitted | suite.after ([object Object])
   [4] Queued | hook WebDriverIO._afterSuite()

-- FAILURES:

  1) Top level-routes: fooScen:

      expected web page to include "FOOLOGG INN"
      + expected - actual

      -Innlogging
      -Brukernavn
      -Passord
      -LOGG INN
      +FOOLOGG INN
      
  
  Scenario Steps:
  
  - I.see("FOOLOGG INN") at Test.Scenario (top-level-routes_test.js:18:5)
  - I.amOnPage("/encounter/new") at Test.Scenario (top-level-routes_test.js:14:5)
  
  Error
      at Object.<anonymous> (/home/developer/dev/dx-nuflo/node_modules/codeceptjs/lib/helper/WebDriverIO.js:1129:51)
      at Object.exec (/home/developer/dev/dx-nuflo/node_modules/webdriverio/build/lib/helpers/safeExecute.js:28:24)
      at Object.resolve (/home/developer/dev/dx-nuflo/node_modules/webdriverio/build/lib/webdriverio.js:189:29)
      at /home/developer/dev/dx-nuflo/node_modules/webdriverio/build/lib/webdriverio.js:538:32
      at _fulfilled (/home/developer/dev/dx-nuflo/node_modules/q/q.js:854:54)
      at self.promiseDispatch.done (/home/developer/dev/dx-nuflo/node_modules/q/q.js:883:30)


  FAIL  | 1 passed, 1 failed   // 4s
   Emitted | global.result ([object Object])


real	0m4.879s
user	0m0.768s
sys	0m0.088s

Test source code


Feature('Top level-routes');

Scenario('a page to add a new encounter should be rendered', (I) => {
  I.amOnPage('/encounter/new');
  I.see('LOGG INN');
});

// just to see how a failing test looks
Scenario('fooScen', (I) => {
  I.amOnPage('/encounter/new');
  I.see('FOOLOGG INN');
});

Details

  • CodeceptJS version: 0.6.3
  • NodeJS Version: 8.1.2
  • Operating System: Ubuntu 16.04 LTS
  • WebDriverIO: 4.8.0
  • Configuration file:
This is my configuration: 
{
  "tests": "./*_test.js",
  "timeout": 5000,
  "output": "./output",
  "helpers": {
    "WebDriverIO" : {
      "browser": "chrome",
      "desiredCapabilities": {
          "browserName": "chrome",
          "chromeOptions": {
              "args": [
                  "headless",
                  "disable-gpu"
              ]
          }
      },
      "windowSize": "320x568",
      "restart": false,
      "timeouts": {
        "script": 10000,
        "page load": 3000,
        "implicit" : 5000
      },
      "url": "http://localhost:23450"
    }
  },
  "include": {
    "I": "./steps_file.js"
  },
  "bootstrap": false,
  "mocha": {},
  "name": "acceptance tests",
  "windowSize": "320x568"
}

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Reactions:7
  • Comments:27 (3 by maintainers)

github_iconTop GitHub Comments

73reactions
oligee80commented, Mar 16, 2018

Try these two ChromeOptions:

options.addArguments("--proxy-server='direct://'");
options.addArguments("--proxy-bypass-list=*");

I dunno in wich forum it was but this did the trick for me

2reactions
GeorgeDubuquecommented, Jul 21, 2019

I am having a similar issue on this one still. I tried with the following arguments

options.add_argument("--proxy-server='direct://'")
options.add_argument("--proxy-bypass-list=*")
options.add_argument("--proxy-server=")

I am still not seeing any speed up. 40 seconds to run in headless and 4 seconds to run in windowed mode. I am running ChromeDriver 75.0.3770.140.

Please let me know if anyone solves this one! Could really help me out. All I am doing is calling driver.get("https://www.google.com")

Ok so after looking into it a bit more it seems that avoiding using headless chrome is the solution for me. Instead I use the pyvirtualdisplay library and set its visibilty to false. This sped up my performance 10x. Here is the code:

from selenium import webdriver
from pyvirtualdisplay import Display
import time

start = time.time()
display = Display(visible=0, size=(800, 600))
display.start()
driver = webdriver.Chrome(
        "/home/george/Desktop/chromedriver")
driver.get("https://www.google.com")
driver.quit()
display.stop()
end = time.time()
print(end-start)

with output:

5.253071069717407

instead of:

42.1179986000061
Read more comments on GitHub >

github_iconTop Results From Across the Web

Improving Browser Performance 10x - Universe Engineering
It is possible to use a single headless browser process and run multiple requests in separate tabs. However, using multiple tabs decreases the ......
Read more >
Re: [chromium-dev] Chromium headless - isolation question
Tests run in parallel, with headless were at least 10x slower, whats more the timings we see indicate some sort of queueing by...
Read more >
What is Headless Browser Testing, When (and Why) to Use It
Headless browser testing improves both the effectiveness and efficiency of your testing process while integrating quality assurance with ...
Read more >
Python Headless Chrome Slows Tests by 20x - Stack Overflow
I am trying to run my tests in headless Chrome using the --headless argument. While it works fine in a normal browser window,...
Read more >
Changelog - Cypress Documentation
The default slow test threshold was changed from 75ms (Mocha's default) to ... When running the --headless Chrome browser via cypress run ,...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

extreme slowness: uses 45 seconds to fill one form

Next page

I.amOnPage is not a function