question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Tabs being throttled in parallel - how to give them all equal share of resources?

See original GitHub issue

Hello, there doesn’t seem to be a way to disable resource load scheduling in Puppeteer, or we’re doing it wrong. For our purposes, which entail a lot of pages being open in parallel, with no particular priority, we want to allocate an equal amount of resources to each page. We found that there should be a flag called

--enable-resource-load-scheduler=false

that would disable the throttling features of Chromium that were introduced recently. It doesn’t seem to have an effect, unfortunately.

Steps to reproduce

Tell us about your environment:

  • Puppeteer version: 1.13
  • Platform / OS version: Mac OS 10.14.3 / various Linuxes in Docker
  • Node.js version: 10 LTS

What steps will reproduce the problem?

const puppeteer = require('puppeteer');

(async function main() {
    const browser = await puppeteer.launch({
        args:['--enable-resource-load-scheduler=false'],
        headless: true,
    });

    const start = Date.now();
    const promises = Array(10).fill(null).map(async () => {
        const page = await browser.newPage();
        await page.goto('https://www.booking.com');
        console.log(`Page took ${Date.now() - start}ms to load.`);
    });
    await Promise.all(promises);
    console.log(`Opening of 10 pages took ${Date.now() - start}ms.`);
    await browser.close();
})();

What is the expected result? With the code above, I expect the 10 pages to load in very similar times (using booking.com because it’s quite a complex website that requires CPU power).

What happens instead? Pages load one after the other, the last one taking almost twice as much time as first one.

Page took 4520ms to load.
Page took 5151ms to load.
Page took 5346ms to load.
Page took 6411ms to load.
Page took 6733ms to load.
Page took 7451ms to load.
Page took 7498ms to load.
Page took 7570ms to load.
Page took 8187ms to load.
Page took 8755ms to load.
Opening of 10 pages took 8755ms.

The enable-resource-load-scheduler=false might not even be the way to go, but do you have any tips on how to achieve equal power to all tabs?

Thanks.

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Reactions:1
  • Comments:8 (5 by maintainers)

github_iconTop GitHub Comments

5reactions
aslushnikovcommented, Apr 2, 2019

@mnmkng I didn’t know about resource load scheduler before. Why do you think it’s the culprit?

FWIW, creating pages in separate browser contexts seems to be working fine:

const puppeteer = require('puppeteer');

(async function main() {
  const browser = await puppeteer.launch({
    args:['--enable-resource-load-scheduler=false'],
    headless: true,
  });

  const start = Date.now();
  const promises = Array(10).fill(null).map(async () => {
    const context = await browser.createIncognitoBrowserContext();
    const page = await context.newPage();
    await page.goto('https://www.booking.com');
    console.log(`Page took ${Date.now() - start}ms to load.`);
  });
  await Promise.all(promises);
  console.log(`Opening of 10 pages took ${Date.now() - start}ms.`);
  await browser.close();
})();
2reactions
JuanmaMenendezcommented, Sep 2, 2019

What about the RAM and CPU performance when there are several context / windows created? Won’t it suffer enough for example with 10 windows opened?

Would we have a similar performance having:

1 window with 10 tabs vs 10 windows with 1 tab each one?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Avoid getting throttled or blocked in SharePoint Online
You 're running an application - for example, to scan files in SharePoint Online - but you get throttled. Or even worse, you...
Read more >
Logic Apps and Throttling - Medium
You will encounter Logic Apps throttling if your Logic App exceed the throughput limits of Logic Apps. You can identify this by looking...
Read more >
Using the Chromium Network Tab for Performance Insights
In this tip, I cover how to use the Network Tab of Chromium F12 ... see the Resource Size for a file loaded...
Read more >
Tab throttling and more performance improvements in Chrome ...
We investigated how background tabs use system resources and found that JavaScript Timers represent >40% of the work in background tabs.
Read more >
Exploring the Network Tab in Chrome Dev Tools
The Network Throttling function allows you to see what a mobile user will experience. You can change the settings in the Throttling dropdown ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found