Identify onion services with their respective (already yt-dlp supported) websites

Checklist

• I’m reporting a new site support request
• I’ve verified that I’m running yt-dlp version 2022.03.08.1. (update instructions)
• I’ve checked that all provided URLs are alive and playable in a browser
• I’ve checked that none of provided URLs violate any copyrights or contain any DRM to the best of my knowledge
• I’ve searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
• I’ve read the guidelines for opening an issue
• I’ve read about sharing account credentials and am willing to share it if required

Region

Tor Network

Example URLs

• Twitter has an onion service: https://twitter3e4tixl4xyajtrzo62zg5vztmjuricljdp2c5kshju4avyoid.onion/TwitterBlue/status/1484226494708662273 (source: https://en.wikipedia.org/wiki/Twitter#External_links )
• Facebook has an onion service: https://www.facebookwkhpilnemxj7asaniu7vnjjbiltxjqhye3mhbshg7kx5tfyd.onion/ (source: https://en.wikipedia.org/wiki/Facebook_onion_address )
• BBC: https://www.bbcnewsd73hkzno2ini43t4gblxvycyac5aw4gnv7t2rccijh7745uqd.onion/

Description

Identify onion services with their respective (already yt-dlp supported) websites (for example twitter is already supported by yt-dlp but its onion service is dealt with by the fallback generic extractor instead of the twitter specific one)

Verbose log

Example for the Twitter onion service:


emily@emily:~/Videos$ yt-dlp https://twitter3e4tixl4xyajtrzo62zg5vztmjuricljdp2c5kshju4avyoid.onion/TwitterBlue/status/1484226494708662273
[generic] 1484226494708662273: Requesting header
WARNING: [generic] Falling back on generic information extractor.
[generic] 1484226494708662273: Downloading webpage
[generic] 1484226494708662273: Extracting information
ERROR: Unsupported URL: https://twitter3e4tixl4xyajtrzo62zg5vztmjuricljdp2c5kshju4avyoid.onion/TwitterBlue/status/1484226494708662273

[dirkf]

This List of Real-World Onion Sites might be consulted for other candidates, eg DW, NYT.

[gamer191]

@resulin They are marketed as a way to avoid trusting exit nodes*, and to let people create accounts for free, anonymously. Also, clearnet sites only have 3 nodes in a circuit.

*As the hyperlinked article notes, using https achieves that already (it’s slightly less secure, but it’s probably good enough). However, python isn’t as good at ssl verification as tor browser (https://revoked.badssl.com/ doesn’t raise an ssl error, for example).

Actually, now that I think about it, I’m not sure whether yt-dlp would block exit nodes from performing MitM attacks on onion sites 👀