[FEATURE] Download management

Forgive me for breaking the template here, but this is more than just a single feature I’m looking to add.

The download management hasn’t been overhauled since #57, and it’s been a long time since then. The main issues are:

1. The GUI page is unwieldy to use
2. It does not fit nicely into our current DB system
3. There is no “order” to the downloads, they are simply split by session
4. Retries are not possible
5. Pausing is not possible
6. All downloads start as soon as they can, without a system to queue them up
7. Download state (e.g. progress) is inconsistent and not clear (how much is downloaded in MBs? What’s the ETA?)

These can be fixed with two new database tables. First, a download_queue which stores all info necessary to begin a download. It will keep track of the state (started or not, progress, paused or not, queue position)

• Caveat 1: how often do we want to update this? currently downloads get updated every second or so, but this may be too frequent
• Caveat 2: how do we store queue position? In the local DB, this is easy as items in a “table” is really just an array. In MongoDB, I don’t believe this isn’t the case. Perhaps we just store the start timestamp and sort when we pull the records.

Next, we can have a finished_downloading table, where we store downloads that finished or failed (with an error log). Here we can include a retry button, and an option to clear the “log” (not the file itself).

The GUI should be overhauled as well, showing first the download queue, and under it the finished downloads. This can be in a table format if space allows.

Related feature request: #379 regarding rate limiting downloads, which will almost certainly merge before this even gets worked on.

---

Tangential to this #157 which should make the process of importing videos easier. The reason these occupy the same space in my brain is I imagine a queue system that can work for both (unprocessed upload -> processed upload, likewise not yet downloaded -> downloaded)

[Tzahi12345]

Backend work is mostly done! Here’s the UI so far (WIP):

https://user-images.githubusercontent.com/707413/128816053-e21809c1-6923-4264-a8bc-94eee432283b.mp4

[jonathanm-ur]

This may belong as a separate request, but it fits with the general theme of download management.

I would like a list of all videos in a channel, with the ability to refresh that list but not automatically download new videos, just offer the ability to check them off and add them to the queue.

This could be implemented in the current GUI layout, when you browse the specific subscription or playlist it could have a refresh button that would update the list of currently available videos, show a blank thumbnail with text or download the thumbnail only and overlay a red X or something that would denote that it’s not available locally yet, clicking on it would offer the ability to immediately download or add it to a queue. It would also be nice to highlight videos that are local only with no online counterpart with a distinct border or something, so you can see at a glance if one of the previous downloads is no longer available online.

There are some channels I watch that occasionally don’t automatically download all new content, it would be nice to know if the youtubedl list isn’t picking up the video, or it picked it up and couldn’t download for some reason.