Sitespeed and Quantum(?)

So I downloaded Firefox Quantum beta in anticipation of next week’s release. I wanted to do a sitespeed comparison of pages in Chrome and Firefox. When I ran in Firefox, I got the below error. I’ll admit that I don’t typically run performance tests in Firefox with Sitespeed so I’m not sure if I’d get this error with another version. So I thought I would post this in case it’s something you are not aware of.

Running version sitespeed.io 6.0.0-alpha1; Firefox 57.0b14 (64-bit)

[2017-11-08 09:06:43] ERROR: WebDriverError
    at Object.throwDecodedError (c:\sitespeed\sitespeed.io\node_modules\selenium-webdriver\lib\error.js:497:15)
    at parseHttpResponse (c:\sitespeed\sitespeed.io\node_modules\selenium-webdriver\lib\http.js:519:13)
    at doSend.then.response (c:\sitespeed\sitespeed.io\node_modules\selenium-webdriver\lib\http.js:441:30)
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:188:7)
    at Driver.schedule (c:\sitespeed\sitespeed.io\node_modules\selenium-webdriver\lib\webdriver.js:807:17)
    at Navigation.to (c:\sitespeed\sitespeed.io\node_modules\selenium-webdriver\lib\webdriver.js:1131:25)
    at Driver.get (c:\sitespeed\sitespeed.io\node_modules\selenium-webdriver\lib\webdriver.js:988:28)
    at context.runWithDriver (c:\sitespeed\sitespeed.io\bin\prescript.js:5:21)
    at SeleniumRunner.runWithDriver (c:\sitespeed\sitespeed.io\node_modules\browsertime\lib\core\seleniumRunner.js:217:12)
    at Object.runWithDriver (c:\sitespeed\sitespeed.io\node_modules\browsertime\lib\core\engine.js:324:31)
    at Object.run (c:\sitespeed\sitespeed.io\bin\prescript.js:4:20)
    at Promise.mapSeries.preScript (c:\sitespeed\sitespeed.io\node_modules\browsertime\lib\core\engine.js:315:23)
From previous event:
    at Promise.resolve.tap.tap (c:\sitespeed\sitespeed.io\node_modules\browsertime\lib\core\engine.js:314:19)
From previous event:
    at runIteration (c:\sitespeed\sitespeed.io\node_modules\browsertime\lib\core\engine.js:313:10)
    at Promise.reduce (c:\sitespeed\sitespeed.io\node_modules\browsertime\lib\core\engine.js:409:27)
From previous event:
    at Promise.resolve.tap.tap.tap.tap.result (c:\sitespeed\sitespeed.io\node_modules\browsertime\lib\core\engine.js:406:17)
    at runCallback (timers.js:781:20)
    at tryOnImmediate (timers.js:743:5)
    at processImmediate [as _immediateCallback] (timers.js:714:5)
From previous event:
    at Engine.run (c:\sitespeed\sitespeed.io\node_modules\browsertime\lib\core\engine.js:405:8)
    at engine.start.then (c:\sitespeed\sitespeed.io\lib\plugins\browsertime\analyzer.js:95:26)
    at Server.once (events.js:342:8)
    at Server.listen (net.js:1422:10)
    at Object.startServer (c:\sitespeed\sitespeed.io\node_modules\browsertime\lib\support\extensionServer.js:15:19)
    at Engine.startExtensionServer (c:\sitespeed\sitespeed.io\node_modules\browsertime\lib\core\engine.js:145:8)
    at Engine.start (c:\sitespeed\sitespeed.io\node_modules\browsertime\lib\core\engine.js:175:12)
    at Object.analyzeUrl (c:\sitespeed\sitespeed.io\lib\plugins\browsertime\analyzer.js:94:8)
From previous event:
    at Object.analyzeUrl (c:\sitespeed\sitespeed.io\lib\plugins\browsertime\analyzer.js:95:8)
    at storageManager.createDirForUrl.then.then (c:\sitespeed\sitespeed.io\lib\plugins\browsertime\index.js:173:32)
    at c:\sitespeed\sitespeed.io\node_modules\mkdirp\index.js:30:20
    at FSReqWrap.oncomplete (fs.js:135:15)
From previous event:
    at Object.processMessage (c:\sitespeed\sitespeed.io\lib\plugins\browsertime\index.js:173:12)
    at queue.process.message (c:\sitespeed\sitespeed.io\lib\support\queueHandler.js:192:32)
    at drainItem (c:\sitespeed\sitespeed.io\node_modules\concurrent-queue\index.js:96:21)
    at Immediate.drain (c:\sitespeed\sitespeed.io\node_modules\concurrent-queue\index.js:77:84)
    at runCallback (timers.js:781:20)
    at tryOnImmediate (timers.js:743:5)
    at processImmediate [as _immediateCallback] (timers.js:714:5)

[soulgalore]

@mgasiorowski ah thanks, let me update the example the coming days. Best Peter

[soulgalore]

Hi @rschoessler short summary: Quantum (beta/nightly whatever) should work as long as you turn off collecting the HAR (–browsertime.skipHar). Firefox 55 broke the https://github.com/firebug/har-export-trigger/ that we use to collect the HAR. I’ve been asking for a while that there will be an version for 57 and they (Mozilla) has it on the 2do list for this quarter. The plan for us was to release 6.0 asap when Quantum was released, but the HAR will still be broken, so lets see how we do it. If you run our Docker container we still use 54 there.

It looks from your log that there maybe is something going on with the WebDriver (and we maybe need to update Geckodriver, I can check that later). How do you start your run (I see that you are in Windows)?

I tried out Browsertime directly and it works on my Mac: bin/browsertime.js --firefox.nightly --skipHar https://www.sitespeed.io

Best Peter