Question: Mounting symlink'd index.sqlite3 on SSD when using docker-compose

Hello,

I’ve tried many ways to make it work, but I can’t achieve to have the database hosted outside of the /data directory. I’m running the project on ZFS (hard drives), the performance is O.K, but it start to get slow on the DB side, and when trying to add multiple links error 500 starts occurring. You suggest running the database at the least on SSD https://github.com/ArchiveBox/ArchiveBox#storage-requirements , but doesn’t seem to work with docker-compose, or I’m unable to mount it properly.

version: '2.4'   # '3.9' or greater also works

services:
    archivebox:
        # build: .                              # for developers working on archivebox
        image: ${DOCKER_IMAGE:-archivebox/archivebox:master}
        command: server --quick-init 0.0.0.0:8000
        ports:
            - 8000:8000
        environment:
            - ALLOWED_HOSTS=*                   # add any config options you want as env vars
            - SAVE_MEDIA=False
            #- MEDIA_MAX_SIZE=750m
            # - SEARCH_BACKEND_ENGINE=sonic     # uncomment these if you enable sonic below
            # - SEARCH_BACKEND_HOST_NAME=sonic
            # - SEARCH_BACKEND_PASSWORD=SecretPassword
        volumes:
            - /docker/archivebox/data:/data

My home directory is an NVMe SSD, I would like to have the index.sqlite3 on /home/user/data/index.sqlite3

Tried symbolic link, but does fail. Compose doesn’t start, looping errors. Tried to add a volume like this, but keeps recreating the database from scratch.

 volumes:
            - /docker/archivebox/data:/data
            - /home/user/data/index.sqlite3:/data/index.sqlite3

Any proper way of mounting it?

[akhilleusuggo]

@pirate I don’t want to reopen the issue, but your last example logically works, but when it comes to running the database, how would I do that?

How can I tell Archivebox that the database (index.sqlite3) is mounted on /media/someSSD/archivebox/index.sqlite and not on the default location data/index.sqlite3

Just by creating a symbolic link, doesn’t mean the data will be written to the location of the symlink. Will still be written on the default /data/index.sqlite3 witch I have on spinning drives with ZFS (compression/dedup ON), and you know performance is horrible compared to without.

Your example maybe was meant to fix the docker symlink issue; but not Archivebox issue.

If more than 1 project has a database that requires mounting on /data/index.sqlite3, this solution will not work. I’m not saying that archivebox issue, but would be nice if we had a variable to point where the database is located.

I know that maybe the last resort would that each one would be rebuilding the image to reflect the location where he wants to store the db.

[akhilleusuggo]

Ok, I see what you mean. Like a reverse symlink. Thank you, I’ll try this way out.