Bug: docker build fails on dev branch

Describe the bug

Running docker build fails on the current dev branch. I’m using rootless containers with podman.

% docker build -t jamesob/archivebox .
STEP 1/29: FROM python:3.10-slim-bullseye
STEP 2/29: LABEL name="archivebox"     maintainer="Nick Sweeting <[e-mail]>"     description="All-in-one personal internet archiving container"     homepage="https://github.com/ArchiveBox/ArchiveBox"     documentation="https://github.com/ArchiveBox/ArchiveBox/wiki/Docker#docker"
--> Using cache f4047895f1e6209eccff50e68fed5e7bfaacc1e9baf4cc4f1891235415d7c33d
--> f4047895f1e
STEP 3/29: ENV TZ=UTC     LANGUAGE=en_US:en     LC_ALL=C.UTF-8     LANG=C.UTF-8     PYTHONIOENCODING=UTF-8     PYTHONUNBUFFERED=1     DEBIAN_FRONTEND=noninteractive     APT_KEY_DONT_WARN_ON_DANGEROUS_USAGE=1
--> Using cache 6484db555198c8e073c326811e8c0089b14b08e9268e58db6484ba96afa74942
--> 6484db55519
STEP 4/29: ENV CODE_DIR=/app     VENV_PATH=/venv     DATA_DIR=/data     NODE_DIR=/node     ARCHIVEBOX_USER="archivebox"
--> Using cache ec13199ab81e9b05438ad2af56b322d71fb7ce84439073f7061cf0b9d3d94085
--> ec13199ab81
STEP 5/29: RUN groupadd --system $ARCHIVEBOX_USER     && useradd --system --create-home --gid $ARCHIVEBOX_USER --groups audio,video $ARCHIVEBOX_USER
--> Using cache 37da61c9f6d0516891f6a4b7210b1a12a99acde386be971a3ea7890991d1934a
--> 37da61c9f6d
STEP 6/29: RUN apt-get update -qq     && apt-get install -qq -y --no-install-recommends         apt-transport-https ca-certificates gnupg2 zlib1g-dev         dumb-init gosu cron unzip curl     && rm -rf /var/lib/apt/lists/*
--> Using cache fad2cc145465c808ab5c1405e4c875a4d2372fca627d1736865f86f7139a5b64
--> fad2cc14546
STEP 7/29: RUN apt-get update -qq     && apt-get install -qq -y --no-install-recommends         wget curl chromium git ffmpeg youtube-dl ripgrep         fontconfig fonts-ipafont-gothic fonts-wqy-zenhei fonts-thai-tlwg fonts-kacst fonts-symbola fonts-noto fonts-freefont-ttf     && ln -s /usr/bin/chromium /usr/bin/chromium-browser     && rm -rf /var/lib/apt/lists/*
--> Using cache 4f456fa30824ba781dcecebc34a3c7f4eecf77a4f8901b557cc5bc45bd4d938d
--> 4f456fa3082
STEP 8/29: RUN curl -s https://deb.nodesource.com/gpgkey/nodesource.gpg.key | apt-key add -     && echo 'deb https://deb.nodesource.com/node_17.x buster main' >> /etc/apt/sources.list     && apt-get update -qq     && apt-get install -qq -y --no-install-recommends         nodejs     && rm -rf /var/lib/apt/lists/*
--> Using cache 723dbb4bf52634381b19df4bafe3804e97cd58a820cb6993a00a2465617e4a56
--> 723dbb4bf52
STEP 9/29: WORKDIR "$NODE_DIR"
--> Using cache 8612673bf7e4795982c8b81cebe1c968f0ec8159898b5c65731ee108a504fe9e
--> 8612673bf7e
STEP 10/29: ENV PATH="${PATH}:$NODE_DIR/node_modules/.bin"     npm_config_loglevel=error
--> Using cache 57a8c3cb2937aca44e4a94d24521c24065946351976f3b6f6171b63dd26d1c60
--> 57a8c3cb293
STEP 11/29: ADD ./package.json ./package.json
--> Using cache 3e22276594dfa564e0468a7c3e399d599598ef917c68492dd38bf95648b5a538
--> 3e22276594d
STEP 12/29: ADD ./package-lock.json ./package-lock.json
--> Using cache a00566ef859dc9b1b41ae361569641c08a804a61d023b4d1f5a0e90717ba387b
--> a00566ef859
STEP 13/29: RUN npm ci
--> Using cache 97a41b6ad463707a7aa96b33adefdca65d4840a81c251143ad3cf42cf7a2985c
--> 97a41b6ad46
STEP 14/29: WORKDIR "$CODE_DIR"
--> Using cache 14cc5250f8eb4626d5da2d4ab12e16a2bcf612e6ca349e7b51127ea0865c725d
--> 14cc5250f8e
STEP 15/29: ENV PATH="${PATH}:$VENV_PATH/bin"
--> Using cache 1075cca5ad2bfe1ae2ed24a6d63a8083dd1b0d489b0bf634e47a709d01d6ec1d
--> 1075cca5ad2
STEP 16/29: RUN python -m venv --clear --symlinks "$VENV_PATH"     && pip install --upgrade pip setuptools     && mkdir -p "$CODE_DIR/archivebox"
Requirement already satisfied: pip in /usr/local/lib/python3.10/site-packages (22.0.4)
Collecting pip
  Downloading pip-22.1-py3-none-any.whl (2.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 43.7 MB/s eta 0:00:00
Requirement already satisfied: setuptools in /usr/local/lib/python3.10/site-packages (58.1.0)
Collecting setuptools
  Downloading setuptools-62.3.1-py3-none-any.whl (1.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 76.5 MB/s eta 0:00:00
Installing collected packages: setuptools, pip
  Attempting uninstall: setuptools
    Found existing installation: setuptools 58.1.0
    Uninstalling setuptools-58.1.0:
      Successfully uninstalled setuptools-58.1.0
  Attempting uninstall: pip
    Found existing installation: pip 22.0.4
    Uninstalling pip-22.0.4:
ERROR: Could not install packages due to an OSError: [Errno 39] Directory not empty: 'urllib3'

Error: error building at STEP "RUN python -m venv --clear --symlinks "$VENV_PATH"     && pip install --upgrade pip setuptools     && mkdir -p "$CODE_DIR/archivebox"": error while running runtime: exit status 1

[jamesob]

Okay, looks like the archivebox user didn’t have read permissions on /app, which I think is explained by my host’s umask (umask 077). So the complete patch set I needed to get this working is:

diff --git a/Dockerfile b/Dockerfile
index 7d42262..0f0430c 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -74,7 +74,7 @@ RUN npm ci
 
 # Install Python dependencies
 WORKDIR "$CODE_DIR"
-ENV PATH="${PATH}:$VENV_PATH/bin"
+ENV PATH="${VENV_PATH}/bin:${PATH}"
 RUN python -m venv --clear --symlinks "$VENV_PATH" \
     && pip install --upgrade --quiet pip setuptools \
     && mkdir -p "$CODE_DIR/archivebox"
@@ -103,7 +103,7 @@ RUN apt-get update -qq \
 # Install ArchiveBox Python package and its dependencies
 WORKDIR "$CODE_DIR"
 ADD . "$CODE_DIR"
-RUN pip install -e .
+RUN pip install -e . && chmod a+rx -R /app
 
 # Setup ArchiveBox runtime config
 WORKDIR "$DATA_DIR"

plus the submodule init.

I can create PR if you think these changes are good, plus mentioning the submodule init in a README.

[FunctionDJ]

@caj-larsson I didn’t. Can you point me to some resources on how that works? I’ve never built a docker image before…