question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Bugfix: Crome crashes inside of docker

See original GitHub issue

Describe the bug

PDF and Screenshot generation failing. (It seems like some of them complete eventually on subsequent attempts) 400MB+ “core” files are left in directories when those failures happen.

Steps to reproduce

This is a new install of ArchiveBox, (First time trying it) using the dockerHub image. (nikisweeting/archivebox) The default capture in the docker-compose fails as well: (command: bash -c 'echo "https://github.com/pirate/ArchiveBox" | /bin/archive; tail -f /dev/null')

  Archivebox:
    image: nikisweeting/archivebox
    container_name: archivebox
    restart: ${RESTART_MODE}
    command: bash -c 'echo "https://github.com/pirate/ArchiveBox" | /bin/archive; tail -f /dev/null'
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /mnt/data/Archive/ArchiveBox:/data
    environment:
      - USE_COLOR=False
      - SHOW_PROGRESS=False

Screenshots or log output

First entry succeeds (at least for pdf+screenshot) second does not:

[*] [2020-07-21 14:09:23] "Opinion: The unspoken truth about managing geeks | Computerworld"
    https://www.computerworld.com/article/2527153/opinion-the-unspoken-truth-about-managing-geeks.html
    √ /data/archive/1595363993
      > archive_org
        Failed: Failed to find "content-location" URL header in Archive.org response.
        Run to see full output:
            cd /data/archive/1595363993;
            curl --location --head --user-agent "ArchiveBox/6c4c6862e (+https://github.com/pirate/ArchiveBox/)" --max-time 60 https://web.archive.org/save/https://www.computerworld.com/article/2527153/opinion-the-unspoken-truth-about-managing-geeks.html

[*] [2020-07-21 14:10:17] "GitHub - pirate/ArchiveBox: 🗃 The open source self-hosted web archive. Takes browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more..."
    https://github.com/pirate/ArchiveBox
    √ /data/archive/1595353459
      > pdf
        Failed:Exception Failed to chmod: output.pdf does not exist (did the previous step fail?)
        Run to see full output:
            cd /data/archive/1595353459;
            google-chrome-unstable --headless --no-sandbox --disable-gpu "--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.75 Safari/537.36" --window-size=1440,2000 --timeout=60000 --print-to-pdf https://github.com/pirate/ArchiveBox
      > screenshot
        Failed:Exception Failed to chmod: screenshot.png does not exist (did the previous step fail?)
        Run to see full output:
            cd /data/archive/1595353459;
            google-chrome-unstable --headless --no-sandbox --disable-gpu "--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.75 Safari/537.36" --window-size=1440,2000 --timeout=60000 --screenshot https://github.com/pirate/ArchiveBox
      > archive_org
        Failed: Failed to find "content-location" URL header in Archive.org response.
        Run to see full output:
            cd /data/archive/1595353459;
            curl --location --head --user-agent "ArchiveBox/6c4c6862e (+https://github.com/pirate/ArchiveBox/)" --max-time 60 https://web.archive.org/save/https://github.com/pirate/ArchiveBox
[√] [2020-07-21 14:11:09] Update of 3 pages complete (3.00 min)
    - 0 links skipped
    - 0 links updated
    - 3 links had errors
    To view your archive, open: /data/index.html

Software versions

  • OS: Linux 5.7
  • Docker: Docker version 19.03.12, build 48a66213fe
  • ArchiveBox version: 10799e4
  • Python version: Python 3.5.3
  • Chrome version: unknown

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:7 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
piratecommented, Jul 22, 2020

Fixed in 8cb5302. You can remove the shm_size: '512m' falg from your docker setup, I disabled SHM usage inside containers using chrome cli args.

1reaction
1n5aN1aCcommented, Jul 22, 2020

Very sorry for forgetting to set a name for the issue!

Yes, --shm-size 512M does resolve the issue, thank you very much!

Here was my solution to configure this in docker-compose:

build:
  context: .
  shm_size: '512m'

However, it should be noted, (possibly added to documentation?) that this solution requires your docker-compose.yml file to be version: '3.5' at a minimum. (not just ‘3’)

Read more comments on GitHub >

github_iconTop Results From Across the Web

Chrome browser crashes when running inside Docker #327
I noticed chrome browser crashes with following errors in the log files: Exiting normally, received signal 15. ffmpeg: no process found ...
Read more >
Issue 715363: Chrome crashes/fails to load when /dev/shm ...
1. Create a Docker container with the default /dev/shm size (64MB) or smaller (ideally, say, 10MB) · 2. Launch Chrome inside the container...
Read more >
chrome/firefox crashed when running on docker containers
Context: I am using 2 local PCs to make a docker swarm cluster where I deploy 1 selenium Grid hub and 10 selenium...
Read more >
OR-Tools Release Notes
This page lists changes to OR-Tools, including new features, bug fixes, and improvements to the code and installation procedures.
Read more >
GKE release notes | Google Kubernetes Engine (GKE)
GKE does not support node images that use Docker as the runtime in GKE version ... With this change, Out Of Memory (OOM)...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found