RegexNotFoundError: Unable to extract video url - Happens only on Web Server

Checklist

• I’m asking a question
• I’ve looked through the README and FAQ for similar questions
• I’ve searched the bugtracker for similar questions including closed ones

Question

I am trying to get the json response using the following command

python3 /usr/local/bin/youtube-dl -j "https://www.instagram.com/p/CJYpUabHXX7/?igshid=qmznpbi11tuk" --verbose

This works just fine on my local system but when I try it on a web server it says RegexNotFoundError: Unable to extract video url. I tried with Ubuntu 18.04, Ubuntu 20.04, and CentOS 8.3. Also, some of the urls work just fine on web server as well.

Here is the log using --verbose parameter

[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-j', 'https://www.instagram.com/p/CJYpUabHXX7/?igshid=qmznpbi11tuk', '--verbose']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.01.08
[debug] Python version 3.6.8 (CPython) - Linux-4.18.0-240.1.1.el8_3.x86_64-x86_64-with-centos-8.3.2011
[debug] exe versions: none
[debug] Proxy map: {}
ERROR: Unable to extract video url; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 803, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 824, in __extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 532, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/instagram.py", line 216, in _real_extract
    video_url = self._og_search_video_url(webpage, secure=False)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 1128, in _og_search_video_url
    return self._html_search_regex(regexes, html, name, **kargs)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 1019, in _html_search_regex
    res = self._search_regex(pattern, string, name, default, fatal, flags, group)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 1010, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
youtube_dl.utils.RegexNotFoundError: Unable to extract video url; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.```

[xihuny]

Tried with different user agents and using cookies gives the same error. I used a chrome extension to generate cookies file for instagram website and used --cookies cookies.txt

Fixed by using authenticated cookies. Previously I was using the cookies I got without login into Instagram account.

[xihuny]

Tried with different user agents and using cookies gives the same error. I used a chrome extension to generate cookies file for instagram website and used --cookies cookies.txt