question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Puppeteer request interceptor not loggin m3u8 requests

See original GitHub issue

so i have noticed that the interceptor will not log any file with extension .m3u8 and i was wondering why this is ? ill give you an example on twitch. Update i just realized that when this is enabled it just wont allow m3u8 to load period?

const puppeteer = require('puppeteer-extra');



(async () => {
  process.setMaxListeners(Infinity)
  const browser = await puppeteer.launch({
    headless: false,
    args: [  '--window-size=0x0','--ignore-certificate-errors', '--no-sandbox',
    '--disable-setuid-sandbox',
    '--disable-dev-shm-usage',
    '--disable-accelerated-2d-canvas',
    '--no-first-run',
    '--no-zygote',
    '--disable-gpu']
  }); 
  const page = await browser.newPage();
  await page.setRequestInterception(true);
  const urls = [];
  page.on('request', request => {
    urls.push(request.url());
    console.log(urls)
    request.continue();
  });

// you may have to change the channel because idk if this guy will be on when this is looked at
  await page.goto('https://player.twitch.tv/?channel=r3ar3ntry', {
    timeout: 0,
    waitUntil: 'load'
  });
  try {
    await page.waitForSelector('button#mature-link', /*{ timeout: 5000 }*/)
    await page.click('button#mature-link')
    await page.addStyleTag({content: '.player{display: none}'})
  } catch (error) {
    await page.addStyleTag({content: '.player{display: none}'})
  }

})()

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Reactions:6
  • Comments:13

github_iconTop GitHub Comments

3reactions
pironside44commented, Jan 4, 2021

I just ran into this issue as well – I believe this is an issue with how puppeteer does request interception (something with page navigation after the first intercepted request). A way around this is to use a CDP session. Here’s a working example I used to grab the m3u8 from a stream:

https://gist.github.com/pironside44/307248e1c081ef4d27ec38fa0148d4d7

0reactions
XeO-techcommented, Aug 1, 2022

A tip on getting video stream requests from iframes. Iframe is a separate target in terms of CDP, so you need to attach a CDP session to the iframe. You can use browser.targets() after page is loaded, then find one you need, attach a session to it, and intercept requests. Hope this will be helpful for newcomers to this topic.
Additional info: https://stackoverflow.com/questions/63357370/capture-requests-xhr-js-css-from-embedded-iframes-using-devtool-protocol

Read more comments on GitHub >

github_iconTop Results From Across the Web

Puppeteer hangs when implementing request.respond with an ...
I realised I was not using got correctly. const resp = await got(options) // Return retrieved response to interceptedRequest ...
Read more >
Request Interception - Puppeteer
An example of a naïve request interceptor that aborts all image requests: import puppeteer from 'puppeteer'; (async () => {
Read more >
Simple index - piwheels
... newio-requests wix-protos-acdc-msm-data-protos rdt-identity dodayutils function-controler minette-symphony django-cadence rancidtoolkit2 http-noah ...
Read more >
Using man-in-the-middle proxy to intercept requests in ...
This article demonstrates how to setup a reliable interception of HTTP requests in headless Chrome / Puppeteer using a local proxy.
Read more >
glove_vocab.250k.txt - Bar Ilan NLP Lab
8th explained requests bug t-shirt aircraft lie alabama ai revealed celebrate ... occurred carrier animation requested copper kong routine comics processor ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found