Add --disable-dev-shm-usage to default launch flags

There are number of issues running pptr/headless in env like Docker where resources are limited.

Shall we add --disable-dev-shm-usage to the default launch flags so users run into fewer issue?

[Glennvd]

Hey Aslushnikov, I’ll see when I have time to run a clean benchmark.

My setup consists of puppeteer pods running on Kubernetes. These pods autoscale to an average of 10 concurrent puppeteer requests. With an absolute max of 20, and everything higher is queued, but this only happens on very high spikes while the autoscaler needs to scale. Each puppeteer request consists of an incoming HTTP request, that starts a new chromium browser, executes the script and returns either a rendered html or a screenshot.

If these pods are allocated enough CPU, the next bottleneck will be the disk IO, especially in a docker container. Running using --disable-dev-shm-usage will only impact this bottleneck further. To be honest to mitigate this bottleneck I even went further and moved the chromium profiles and chromium executable to /dev/shm completely. Since there we’re multiple browsers opening/closing every second, causing repetitive disk reads and writes.

This approach allowed me to scale for peaks up to 400 concurrent sessions, and if needed probably a lot higher.

Now while --disable-dev-shm-usage is a perfectly valid choice when you’re just running small tasks, and performance doesn’t matter that much. I ultimately think this is a choice that should be left to the developer itself. Especially since not everyone is running in docker to begin with. Otherwise we might start adding flags like --disable-gpu or --no-sandbox by default as well, since these also cause issues in docker.

My proposal would be to leave these flags out of any default configuration, but add a clear FAQ with docker caveats instead.

[Glennvd]

Any reason why this was suddenly implemented while the general consensus was not doing it? Correct me if I’m wrong but enabling this flag in docker containers will cause chromium to use /tmp instead of /dev/shm. Which means it will use the /tmp folder which is a regular filesystem in docker instead of /dev/shm which is shared RAM.

When optimising puppeteer for high throughput, one of the limits you hit is disk IO. This specific case is easily prevented by running docker with --shm-size="". I personally think it’s a weird choice to enable this by default, rather than advising that it exists or that docker should run with the shm-size flag.