Mitigation of attack vectors from external provided links for generic extractor

Checklist

• I’m asking a question
• I’ve looked through the README and FAQ for similar questions
• I’ve searched the bugtracker for similar questions including closed ones

Question

1. Are there any mitigation possibilities in place for attacks such as Slowloris or a 3 party url that will indefinitely send data without a content-length?

2. Does the downloader respect the content-length header and error out if the downloaded size is a mismatch?

3. I have noticed that there is an option for --max-filesize but does that use content-length or the actual downloaded bytes so far?

4. Is there anything I could do to kill the process if the download speed is lower than a threshold? Or is there any option that would help me achieve this?

[dstftw]

I’ve re-checked question 2 and looks like it actually silently truncates data at Content-Length border if there are more data. This happens somewhere in python internals or deeper. Browsers seem to have similar behavior. So that the actual answer to 2 is “no”.

[legraphista]

Thank you for your time & answers.

I have come to the conclusion that my requirements are out of the scope of youtube-dl.

We can write a custom downloader that aligns with our needs and use that when the extractor is generic, else let youtube-dl handle the heavy work.

I’ll most likely leverage --load-info-json to cache the metadata and not send out duplicate requests after the extractor checks.