Scraping more than 50 / 100 Comment [Issue]
See original GitHub issueI noticed when scraping more than 50 or 100, it returns the JSON pair comment_full
as NONE.
Please consider my issue @kevinzg and @neon-ninja
I think comment rendering issue or else timeout
Issue Analytics
- State:
- Created 2 years ago
- Comments:13 (5 by maintainers)
Top Results From Across the Web
YouTube comment scraper returns limited results
The problem is that it doesn't return all the comments. In fact, it always returns a vector with 283 elements, regardless of how...
Read more >How Web Scrape Multiple Pages with ONE Function with Python
Many websites use the same template for multiple pages of data, and this video shows you can create a single function to scrape...
Read more >How to Scrape Websites Without Getting Blocked - ScrapeHero
Check if Website is Changing Layouts. Some websites make it tricky for scrapers, serving slightly different layouts. For example, in a website ...
Read more >DOs and DON'Ts of Web Scraping - ZenRows
Some actions are independent of the website you are scraping: get HTML, parse it, queue new links to crawl, store content, and more....
Read more >How to Scrape Multiple Pages of a Website Using Python?
The above technique is absolutely wonderful, but what if you need to scrape different pages, and you don't know their page numbers? You'll...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
For comments, yes. See what happens when you try view comments on https://m.facebook.com/story.php?story_fbid=1517650235239786&id=144655535872603 in an incognito window.
With the code:
I get the output:
510 comments extracted in 86s