[BUG] The same video is displayed in several.

Describe the bug The webpage is not responding. The same video is displayed in several.

To Reproduce Steps to reproduce the behavior:

1. git clone https://github.com/Tzahi12345/YoutubeDL-Material.git
2. docker build -t ytdl:0.1 .
3. edit docker-compose.yml file (container name, mount path)
4. goto YTDL setting. select yt-dlp, use cookies
5. then Subscribe to about 15 channels. (It happened when I subscribed to a new channel or new video)
6. After a few hours the webpage becomes unresponsive.
7. So I restarted the container.

Expected behavior The same video appears.

Screenshots

Environment

• After downloading git clone, I made an image with docker build.

Additional context I don’t know if it’s a virtual machine problem or a Docker problem. (I’m running an Ubuntu virtual machine on xcp-ng, and I’ve set up 6 cpu and 8 GB of ram. Also, I am using a Synology NAS, and it is mounted through smb.) The webpage goes unresponsive within a few hours of running the container. So I had no choice but to restart the container. I don’t know if that’s the reason, but the same video appears multiple times.

version: "2"
services:
    ytdl_material:
        environment:
            ALLOW_CONFIG_MUTATIONS: 'true'
            ytdl_mongodb_connection_string: 'mongodb://ytdl-mongo-db:27017'
            ytdl_use_local_db: 'false'
            write_ytdl_config: 'true'
        restart: always
        volumes:
            - ./appdata:/app/appdata
            - /mnt/z/audio:/app/audio
            - /mnt/z/video:/app/video
            - /mnt/z/subscriptions:/app/subscriptions
            - ./users:/app/users
        ports:
            - "8998:17442"
        container_name: ytdl
        image: ytdl:0.1
    ytdl-mongo-db:
        image: mongo:latest
        restart: always
        ports:
            - "27017:27017"
        logging:
            driver: "none"
        container_name: mongo-db
        volumes:
            - ./db/:/data/db

[Tzahi12345]

I pushed another change that will hopefully fix this. Beforehand, the logic to avoid duplicates used the path to the file which was inconsistent as mentioned above due to different formatting. Now this is done just using the video URL (see commit here).

Note that #420 should fix this problem regardless once that gets merged.

[GlassedSilver]

You might have pulled a slightly outdated nightly build there before, because there have been two builds in rapid succession and sometimes DockerHub takes a little while to pick up on a new build.

Glad to hear it seems solved for you. As for UI responsiveness, that as well is something that yours truly is constantly trialing with a database of… oh… you know… over 30k videos 😄 (and atm 48 subscriptions)

It’s something that is currently being looked into a lot more, so maybe that helps you rest a little easier. 😉

#420 will definitely help a lot in this regard as well. Overall YoutubeDL-Material has undergone already one MASSIVE increase in performance on large databases both thanks to migrating to MongoDB which is a fair step up from running off of json files as well as the recent change to load the UI right away on container start rather than waiting for a bunch of background processes to finish which for heavy users like me meant something between 10-20 minutes of start up time or even more until I could get into the UI. (and more it was towards the end for sure…)

We’ve come a long way and I’m very optimistic about more optimizations happening that’ll make using the application even more fun and reliable. 😃