Asynchronous parallel screenshotting

Steps to reproduce

Tell us about your environment:

• Puppeteer version: 0.13.0
• Platform / OS version: Debian Jessie (Linux 5d08acbab1a9 4.13.12-200.fc26.x86_64) (from node:8-slim Docker image)

What steps will reproduce the problem?

I have set up very basic Express.js application which takes URL in request and returns screenshot in response. Here is the important code:

router.post('/screenshot', async function (req, res) {
  const page = await browser.newPage();
  await page.goto(req.body.url);
  const buffer = await page.screenshot();
  await page.close();
  res.set('Content-Type', 'image/png');
  res.write(buffer, 'binary');
  res.end(null, 'binary');
});

(browser is created only once).

What is the expected result? / What happens instead?

When I send multiple requests at the same time they all are completed synchronously, one after another. Node’s --trace-sync-io flag outputs a lot of info about sync I/O output too.

Question: Am I right in suggestion that it is not possible to use Puppeteer for asynchronous screenshotting with one Node process and I have to spawn multiple instances of Node?

Sorry if that should be obvious, I just want to confirm (Google did not help). Thanks in advance.

[intellix]

Also having massive issues with parallel screenshot handling and can’t seem to find a sweet spot or how to properly optimise this to perform well and not crash. I’ve currently got 12 arrays of pages to visit and screenshots to take (4 sites with desktop, mobile and tablet with different device ratios).

12 Browser with 1 page each causes server to have load of 13+ and eventually things start timing out and browsers start crashing outright 4 Browsers with 3 pages each causes load of around 6+ and again, things start to timeout. 1 Browser with 12 pages is super slow, seems to be ok, but if I only have 2 requests in, then a screenshot just stops… the whole thing just stops and there’s no error, like it just gave up and there was nothing thrown.

These are my params:

const browser = await puppeteer.launch({
  headless: true,
  args: ['--no-sandbox', '--disable-setuid-sandbox'],
});
this.page = await this.browser.newPage();
await this.page.setViewport({
  ...device.viewport,
  height: scrollHeight,
});
await this.page.screenshot({ fullPage: true });

Originally ran on an AWS t2.medium and then tried a c5.large. Nothing seems stable. I wouldn’t mind so much if it was just slower but it’s either a timeout fest or things just give up. Are there any recommendations for the flags when starting up perhaps or some extra params to setup the server with? I’ve set shared-mem to 1gb

[Oleg-Arkhipov]

@vonGameTheory thanks for your information about the fact that you made it to work, so I understood that it is something on “my” side, not Puppeteer’s. Indeed, the code is right, the problem is that I was testing by refreshing few tabs with requests in my own Chrome, and Chrome sends only one request to the same origin at the same time, haha. I then tested with ab tool and everything is right. Thanks!