Expecting ',' delimiter: line 1 column 1576 (char 1575)

This may be out of scope as I am using youtube-transcript-api with parallel processing and the issue only happens with it. However I believe it is the way the youtube-transcript-api that is handling that error that is the bug.

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10 : 10.0.19041
youtube-transcript-api Version: 0.4.1
Python version: 3.9.6

I originally believe it to be an issue with the parallel processing package, however after speaking with that maintainer of that package. The guess is that it could be:

by you, he’s referring to me

my guess would be you bombed some server with too many concurrent requests, _fetch_video_html gave up and returned some 500 Internal Server Error or so response (some non-200 response which does not contain a payload and so can not be json decoded). Maybe they forgot to add a response.raise_for_status() which would have made this traceback more verbose.

Heres the remote traceback

---------------------------------------------------------------------------
RemoteTraceback                           Traceback (most recent call last)
RemoteTraceback: 
"""
Traceback (most recent call last):
  File "c:\Users\manue\Documents\Github\data-science-venv\.venv\lib\site-packages\multiprocess\pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "c:\Users\manue\Documents\Github\data-science-venv\.venv\lib\site-packages\pathos\helpers\mp_helper.py", line 15, in <lambda>
    func = lambda args: f(*args)
  File "c:\Users\manue\Documents\Github\data-science-venv\.venv\lib\site-packages\mapply\mapply.py", line 105, in run_apply
    return df_or_series.apply(func, args=args, **kwargs)
  File "c:\Users\manue\Documents\Github\data-science-venv\.venv\lib\site-packages\pandas\core\frame.py", line 8740, in apply
    return op.apply()
  File "c:\Users\manue\Documents\Github\data-science-venv\.venv\lib\site-packages\pandas\core\apply.py", line 688, in apply
    return self.apply_standard()
  File "c:\Users\manue\Documents\Github\data-science-venv\.venv\lib\site-packages\pandas\core\apply.py", line 812, in apply_standard
    results, res_index = self.apply_series_generator()
  File "c:\Users\manue\Documents\Github\data-science-venv\.venv\lib\site-packages\pandas\core\apply.py", line 828, in apply_series_generator
    results[i] = self.f(v)
  File "C:\Users\manue\AppData\Local\Temp/ipykernel_10772/780513497.py", line 5, in get_transcripts
  File "c:\Users\manue\Documents\Github\data-science-venv\.venv\lib\site-packages\youtube_transcript_api\_api.py", line 128, in get_transcript
    return cls.list_transcripts(video_id, proxies, cookies).find_transcript(languages).fetch()
  File "c:\Users\manue\Documents\Github\data-science-venv\.venv\lib\site-packages\youtube_transcript_api\_api.py", line 70, in list_transcripts
    return TranscriptListFetcher(http_client).fetch(video_id)
  File "c:\Users\manue\Documents\Github\data-science-venv\.venv\lib\site-packages\youtube_transcript_api\_transcripts.py", line 36, in fetch
    self._extract_captions_json(self._fetch_video_html(video_id), video_id)
  File "c:\Users\manue\Documents\Github\data-science-venv\.venv\lib\site-packages\youtube_transcript_api\_transcripts.py", line 50, in _extract_captions_json
    captions_json = json.loads(
  File "C:\Users\manue\.pyenv\pyenv-win\versions\3.9.6\lib\json\__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "C:\Users\manue\.pyenv\pyenv-win\versions\3.9.6\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Users\manue\.pyenv\pyenv-win\versions\3.9.6\lib\json\decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 1576 (char 1575)
"""

Let me know if anything else is needed.

[jdepoix]

I forgot to mention: the improved error message has been released with version 0.4.2

[thoughtfuldata]

Thanks!

This helps me out