question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Include playlist description and view count when using youtube-dl with `--flat-playlist -J` on a YouTube playlist

See original GitHub issue

Checklist

  • I’m reporting a feature request
  • I’ve verified that I’m running youtube-dl version 2021.06.06
  • I’ve searched the bugtracker for similar feature requests including closed ones

Description

I have been trying to find a way to quickly get information and video URLs/titles from a YouTube playlist, so far I have found that using youtube-dl <playlist URL> -J --flatplaylist is the closest to what I want, returning a list of all videos as well as the uploader of the playlist and the playlist name.
However, it would be extremely useful if it could also return even more information about the playlist, like the playlist’s Description & view count.

Issue Analytics

  • State:open
  • Created 2 years ago
  • Comments:5 (2 by maintainers)

github_iconTop GitHub Comments

3reactions
nose-gnomecommented, Oct 27, 2021

I have made pull request #30161 with the changes you have shown, and the changes I made

2reactions
nose-gnomecommented, Oct 27, 2021

Thank you for your response showing the solution for the missing playlist description.

On the matter of obtaining extra metadata like view_count, I found that they are stored inside:

 {
     'sidebar': {
         'metadata': {...} // Where the Title & Description are currently being extracted from.
 
         'playlistSidebarRenderer': {
             'items': [
                 0: {                
                     // As well as containing stats, items[0] also contains re-occurring information like Title, Description & thumbnail
 
                     'stats': [
                         0: {...}, // 0 Contains the number for videos in the playlist
                         1: {...}, // 1 Contains how many views the playlist has
                         2: {...} //  2 Contains the date of the last time the playlist was updated.
                     ]
                 }
             ]
         }
     }
 }

So, by making the following changes, I found how to include the view_count and the last_updated date in the output:

--- a/youtube_dl/extractor/common.py
+++ b/youtube_dl/extractor/common.py
@@ -968,7 +968,8 @@ class InfoExtractor(object):
             urls, playlist_id=playlist_id, playlist_title=playlist_title)
 
     @staticmethod
-    def playlist_result(entries, playlist_id=None, playlist_title=None, playlist_description=None):
+    def playlist_result(entries, playlist_id=None, playlist_title=None, playlist_description=None,
+                        playlist_view_count=None, playlist_last_update=None):
         """Returns a playlist"""
         video_info = {'_type': 'playlist',
                       'entries': entries}
@@ -978,6 +979,10 @@ class InfoExtractor(object):
             video_info['title'] = playlist_title
         if playlist_description:
             video_info['description'] = playlist_description
+        if playlist_view_count:
+            video_info['view_count'] = playlist_view_count
+        if playlist_last_update:
+            video_info['last_updated'] = playlist_last_update
         return video_info
 
     def _search_regex(self, pattern, string, name, default=NO_DEFAULT, fatal=True, flags=0, group=None):
--- a/youtube_dl/extractor/youtube.py
+++ b/youtube_dl/extractor/youtube.py
@@ -2785,6 +2785,18 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
                 data, lambda x: x['metadata']['playlistMetadataRenderer'], dict)
             if renderer:
                 title = renderer.get('title')
+                description = renderer.get('description')
+
+                stats = try_get(
+                    data, lambda x: x['sidebar']['playlistSidebarRenderer']['items'][0]['playlistSidebarPrimaryInfoRenderer']['stats'])
+                view_count_text = try_get(
+                    stats, lambda x: x[1]['simpleText'], compat_str) or ''
+                view_count = str_to_int(self._search_regex(
+                    r'^([\d,]+)', re.sub(r'\s', '', view_count_text),
+                    'view count', default=None))
+
+                last_updated_text = try_get(stats, lambda x: x[2]['runs'][1]['text'])
+                last_updated = unified_strdate(last_updated_text)
             else:
                 renderer = try_get(
                     data, lambda x: x['header']['hashtagHeaderRenderer'], dict)
@@ -2793,7 +2805,9 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
         playlist = self.playlist_result(
             self._entries(selected_tab, item_id, webpage),
             playlist_id=playlist_id, playlist_title=title,
-            playlist_description=description)
+            playlist_description=description,
+            playlist_view_count=view_count,
+            playlist_last_update=last_updated)
         playlist.update(self._extract_uploader(data))
         return playlist

Then:

youtube-dl --flat-playlist -J PLZHQObOWTQDMXMi3bUMThGdYqos36X_lA

->

{
  "_type": "playlist",
  "entries": [
    {
      "_type": "url",
      "ie_key": "Youtube",
      "id": "GNcFjFmqEc8",
      "url": "GNcFjFmqEc8",
      "title": "But why is a sphere's surface area four times its shadow?",
      "description": null,
      "duration": 1021,
      "view_count": null,
      "uploader": "3Blue1Brown"
    },
    {
      "_type": "url",
      "ie_key": "Youtube",
      "id": "OkmNXy7er84",
      "url": "OkmNXy7er84",
      "title": "The hardest problem on the hardest test",
      "description": null,
      "duration": 675,
      "view_count": null,
      "uploader": "3Blue1Brown"
    },
    {
      "_type": "url",
      "ie_key": "Youtube",
      "id": "AmgkSdhK4K8",
      "url": "AmgkSdhK4K8",
      "title": "Who cares about topology?   (Inscribed rectangle problem)",
      "description": null,
      "duration": 1096,
      "view_count": null,
      "uploader": "3Blue1Brown"
    },
    {
      "_type": "url",
      "ie_key": "Youtube",
      "id": "pQa_tWZmlGs",
      "url": "pQa_tWZmlGs",
      "title": "Why slicing a cone gives an ellipse",
      "description": null,
      "duration": 772,
      "view_count": null,
      "uploader": "3Blue1Brown"
    },
    {
      "_type": "url",
      "ie_key": "Youtube",
      "id": "yuVqxCSsE7c",
      "url": "yuVqxCSsE7c",
      "title": "Sneaky Topology | The Borsuk-Ulam theorem and stolen necklaces",
      "description": null,
      "duration": 1190,
      "view_count": null,
      "uploader": "3Blue1Brown"
    },
    {
      "_type": "url",
      "ie_key": "Youtube",
      "id": "zwAD6dRSVyI",
      "url": "zwAD6dRSVyI",
      "title": "Thinking outside the 10-dimensional box",
      "description": null,
      "duration": 1627,
      "view_count": null,
      "uploader": "3Blue1Brown"
    },
    {
      "_type": "url",
      "ie_key": "Youtube",
      "id": "K8P8uFahAgc",
      "url": "K8P8uFahAgc",
      "title": "Circle Division Solution",
      "description": null,
      "duration": 533,
      "view_count": null,
      "uploader": "3Blue1Brown"
    }
  ],
  "id": "PLZHQObOWTQDMXMi3bUMThGdYqos36X_lA",
  "title": "Geometry",
  "view_count": 125486,
  "last_updated": "20191121",
  "uploader": "3Blue1Brown",
  "uploader_id": "UCYO_jab_esuFRV4b17AJtAw",
  "uploader_url": "https://www.youtube.com/c/3blue1brown",
  "extractor": "youtube:tab",
  "webpage_url": "https://www.youtube.com/playlist?list=PLZHQObOWTQDMXMi3bUMThGdYqos36X_lA",
  "webpage_url_basename": "playlist",
  "extractor_key": "YoutubeTab"
}
Read more comments on GitHub >

github_iconTop Results From Across the Web

youtube-dl; How download ONLY the playlist, NOT the files ...
Try using this command: youtube-dl -j --flat-playlist "https://<yourYoutubePlaylist>" | jq -r '.id' | sed 's_^_https://youtu.be/_' > result.
Read more >
youtube-dl/README.md at master - GitHub
Command-line program to download videos from YouTube.com and other video sites - youtube-dl/README.md at master · ytdl-org/youtube-dl.
Read more >
Generate list of videos (--flat-playlist) : r/youtubedl - Reddit
I want to generate a list of all of videos in a playlist, I found the option --flat-playlist in the Readme. It starts...
Read more >
How Does YouTube Count Playlist Views? - TubeRanker
YouTube counts playlist views differently than video views. Finding the number of views a playlist has is easy, but understanding how it got ......
Read more >
Efficiently get every video of every playlist of a channel
If you only manipulate your YouTube channel through your script then you can keep track of current playlists' videos state.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found