Error saving metadata for large playlist: ([Errno 24] Too many open files)

System OS

Linux

Python Version

3.9 (CPython)

Install Source

pip / PyPi

Install version / commit hash

v4.0.0rc2

Expected Behavior vs Actual Behavior

Expected the command to generate a save file with YouTube URLs for all tracks of the playlist – minus those with no match found.

What actually happens is that the process fails due to too many open files.

Perhaps the HTTPSConnection objects accumulate during the track iteration?

Steps to reproduce - Ensure to include actual links!

spotdl --save-file studio_piano_jazz.spotdl --preload save 'https://open.spotify.com/playlist/6147ogdoSLuOiYlPLHEZ9k?si=22b5ba535d164791'

Traceback

Found url for Gerald Clayton - My Ideal 2: https://youtube.com/watch?v=kb47y2BcKUI
Found url for Gerald Clayton - My Ideal 1: https://youtube.com/watch?v=tu4PE1czJMM
Found url for Gerald Clayton - There Is Music Where You're Going My Friends: https://youtube.com/watch?v=_NDUgdSuigA
Found url for Gerald Clayton - Nobody Else But Me: https://youtube.com/watch?v=kXZwvt2IjQM
Found url for Gerald Clayton - Hank: https://youtube.com/watch?v=VtvBrC_cHxg
...
Found url for Don Grusin - So Nan Desu: https://youtube.com/watch?v=18AF8aaTqZc
Found url for Don Grusin - Jam Suite 2 - Colorado Peach Jam: https://youtube.com/watch?v=V0i0QHMUBXA
Song(name='Flora', artists=['Don Grusin'], artist='Don Grusin', album_name='Old Friends & Relatives', album_artist='Don Grusin', genres=[], disc_number=1, disc_count=1, duration=283.133, year=1996, date='1996-01-01', track_number=4, tracks_count=14, 
song_id='604SkI7RzRtpClr4j8fvML', cover_url='https://i.scdn.co/image/ab67616d0000b2732e761d41d601ee6181e5455e', explicit=False, publisher='Don Grusin', url='https://open.spotify.com/track/604SkI7RzRtpClr4j8fvML', isrc='JPB520600223', copyright_text='1996 Don Grusin 
Music', download_url=None, song_list=None, lyrics=None) generated an exception: HTTPSConnectionPool(host='music.youtube.com', port=443): Max retries exceeded with url: / (Caused by SSLError(OSError(24, 'Too many open files')))
Song(name='Foreign Service', artists=['Don Grusin'], artist='Don Grusin', album_name='Old Friends & Relatives', album_artist='Don Grusin', genres=[], disc_number=1, disc_count=1, duration=307.906, year=1996, date='1996-01-01', track_number=5, tracks_count=14, 
song_id='1Tf7Hgq28QYS9vpdU8Krdi', cover_url='https://i.scdn.co/image/ab67616d0000b2732e761d41d601ee6181e5455e', explicit=False, publisher='Don Grusin', url='https://open.spotify.com/track/1Tf7Hgq28QYS9vpdU8Krdi', isrc='JPB520600224', copyright_text='1996 Don Grusin 
Music', download_url=None, song_list=None, lyrics=None) generated an exception: HTTPSConnectionPool(host='music.youtube.com', port=443): Max retries exceeded with url: / (Caused by SSLError(OSError(24, 'Too many open files')))
Song(name='Estate', artists=['Don Grusin'], artist='Don Grusin', album_name='Old Friends & Relatives', album_artist='Don Grusin', genres=[], disc_number=1, disc_count=1, duration=315.466, year=1996, date='1996-01-01', track_number=6, tracks_count=14, 
song_id='1KJAVAs299rrW5JGokUuum', cover_url='https://i.scdn.co/image/ab67616d0000b2732e761d41d601ee6181e5455e', explicit=False, publisher='Don Grusin', url='https://open.spotify.com/track/1KJAVAs299rrW5JGokUuum', isrc='JPB520600225', copyright_text='1996 Don Grusin 
Music', download_url=None, song_list=None, lyrics=None) generated an exception: HTTPSConnectionPool(host='music.youtube.com', port=443): Max retries exceeded with url: / (Caused by SSLError(OSError(24, 'Too many open files')))
Song(name='Circles', artists=['Don Grusin'], artist='Don Grusin', album_name='Old Friends & Relatives', album_artist='Don Grusin', genres=[], disc_number=1, disc_count=1, duration=292.826, year=1996, date='1996-01-01', track_number=7, tracks_count=14, 
song_id='3Y9BIFK8FXugkdbVH6HyWV', cover_url='https://i.scdn.co/image/ab67616d0000b2732e761d41d601ee6181e5455e', explicit=False, publisher='Don Grusin', url='https://open.spotify.com/track/3Y9BIFK8FXugkdbVH6HyWV', isrc='JPB520600226', copyright_text='1996 Don Grusin 
Music', download_url=None, song_list=None, lyrics=None) generated an exception: HTTPSConnectionPool(host='music.youtube.com', port=443): Max retries exceeded with url: / (Caused by SSLError(OSError(24, 'Too many open files')))
...
(errors spew out, all the same 'Too many open files')
...
Song(name='My Foolish Heart', artists=['McCoy Tyner'], artist='McCoy Tyner', album_name='Jazz Roots', album_artist='McCoy Tyner', genres=['avant-garde jazz', 'contemporary post-bop', 'free jazz', 'hard bop', 'jazz', 'jazz fusion', 'jazz piano', 'stride'], disc_number=1, 
disc_count=1, duration=257.506, year=2000, date='2000-01-01', track_number=3, tracks_count=14, song_id='4z9kyF5zFf7VmdLgfBcAC4', cover_url='https://i.scdn.co/image/ab67616d0000b2732658a8306b9c1e92a5cef638', explicit=False, publisher='Telarc', 
url='https://open.spotify.com/track/4z9kyF5zFf7VmdLgfBcAC4', isrc='USTE10450703', copyright_text='© 2000 Telarc International', download_url=None, song_list=None, lyrics=None) generated an exception: HTTPSConnectionPool(host='music.youtube.com', port=443): Max retries 
exceeded with url: / (Caused by SSLError(OSError(24, 'Too many open files')))
Saved 971 songs to studio_piano_jazz.spotdl

Other details

There are 2,209 songs on this playlist.

Reproduces on Mac OS X as well.

[xnetcat]

https://github.com/spotDL/spotify-downloader/commit/182ed3f22c87c2bf99e077021b7a0b9c62ce0c47

[Silverarmor]

not stale - for @xnetcat to review I believe?

Or possibly @almostimplemented you can open a PR if your solution works 😃