question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. Itย collects links to all the places you might be looking at while hunting down a tough bug.

And, if youโ€™re still stuck at the end, weโ€™re happy to hop on a call to see how we can help out.

INFO:facebook_scraper.page_iterators:Page parser did not find next page URL

See original GitHub issue

Hi, The NPM package for Facebook Scraper has been updated to version 0.23.43. Since then, I have not scrapped any group. The script just stops when I see this message with debug mode:

INFO:facebook_scraper.page_iterators:Page parser did not find next page URL

What could be the cause of this? I use 100 proxy servers and Facebook cookies. Everything went well yesterday.

I appreciate your help.

Issue Analytics

  • State:open
  • Created 2 years ago
  • Comments:15

github_iconTop GitHub Comments

2reactions
maorkavodcommented, Jun 28, 2021

The property works great! I appreciate your help. Compared to paid services such as proxycrawel, your facebook scrapper works better ๐Ÿ˜ƒ

0reactions
neon-ninjacommented, Jun 27, 2021

Yes, it works fine for me. This code:

pprint(next(get_posts(group="Pishpeshuk.Nadlan", options={"allow_extra_requests": False})))

returns

{'available': True,
 'comments': 0,
 'comments_full': None,
 'factcheck': None,
 'image': None,
 'image_lowquality': 'https://scontent.fakl1-2.fna.fbcdn.net/v/t1.6435-9/fr/cp0/e15/q65/105610023_3682686048416784_7502958421092527351_n.jpg?_nc_cat=109&ccb=1-3&_nc_sid=ca434c&_nc_ohc=GcadebOLb5UAX9IjgIs&_nc_ht=scontent.fakl1-2.fna&tp=14&oh=b2e5736d4f5d51be28780a439874160c&oe=60DDE0F1',
 'images': None,
 'images_description': None,
 'images_lowquality': ['https://scontent.fakl1-2.fna.fbcdn.net/v/t1.6435-9/fr/cp0/e15/q65/105610023_3682686048416784_7502958421092527351_n.jpg?_nc_cat=109&ccb=1-3&_nc_sid=ca434c&_nc_ohc=GcadebOLb5UAX9IjgIs&_nc_ht=scontent.fakl1-2.fna&tp=14&oh=b2e5736d4f5d51be28780a439874160c&oe=60DDE0F1'],
 'images_lowquality_description': ['No photo description available.'],
 'is_live': False,
 'likes': 25,
 'link': None,
 'post_id': '3009648419116144',
 'post_text': 'ื‘ืจื•ื›ื™ื ื”ื‘ืื™ื ืœืคืฉืคืฉื•ืง ื“ื™ืจื•ืช-ืœื”ืฉื›ืจื”/ืžื›ื™ืจื” , ื‘ืงื‘ื•ืฆื” ื ื™ืชืŸ ืœืคืจืกื '
              'ื“ื™ืจื•ืช ืœืžื›ื™ืจื”/ืœื”ืฉื›ืจื”, ืกืื‘ืœื˜ ื•ืฉื•ืชืคื™ื ื‘ื›ืœ ื”ืืจืฅ, ื“ื™ืจื•ืช ื’ืŸ, '
              "ืคื ื˜ื”ืื•ื–ื™ื, ื“ื™ืจื•ืช ื™ื•ืงืจื”, ืžื—ืกื ื™ื, ื‘ืชื™ื, ื•ื™ืœื•ืช , ืงื•ื˜ื’'ื™ื ื•ื ื›ืกื™ื "
              'ืขืกืงื™ื™ื.\n'
              '\n'
              '*ืžืชื•ื•ื›ื™ื ืฉื™ืžื• ืœื‘ - ืžืชื•ื•ื›ื™ื ืžื•ืจืฉื™ื ืœืคืจืกื ืžื•ื“ืขื” ืื—ืช ื‘ื™ื•ื ื‘ืœื‘ื“, '
              'ื‘ืžื™ื“ื” ื•ืžืชื•ื•ืš ื™ืคืจืกื ืžืขืœ ืžื•ื“ืขื” ืื—ืช - ื”ื•ื ื™ื•ืกืจ ืžื”ืœื•ื—.\n'
              '\n'
              '*ืืœ ืชื—ืกื›ื• ื‘ืคืจื˜ื™ื - ื›ืชื‘ื• ื”ืื ืžื“ื•ื‘ืจ ื‘ืžื›ื™ืจื”/ื”ืฉื›ืจื”, ืกื•ื’ ื”ื ื›ืก, ืื–ื•ืจ '
              'ื”ื ื›ืก, ืžืฆื‘ ื”ื ื›ืก ื•ืขื•ื“ ืคืจื˜ื™ื ืฉื™ื•ื›ืœื• ืœืขื–ื•ืจ ืœืจื•ื›ืฉื™ื.\n'
              '\n'
              '*ืื ื—ื ื• ื ื’ื“ ืกืคืื - ืคืจืกื•ืžื™ื ืฉืœื ืงืฉื•ืจื™ื ืœืจื•ื— ื”ืงื‘ื•ืฆื”, ื™ื•ืกืจื• ื•ื”ื’ื•ืœืฉ '
              'ื™ื•ืจื—ืง ืœืฆืžื™ืชื•ืช.\n'
              '\n'
              '*ื™ืฉ ืœื”ืงืคื™ื“ ืขืœ ืคืจืกื•ื ืชืžื•ื ื•ืช + ืžื—ื™ืจ ืœืžืขืŸ ื ื•ื—ื™ื•ืช ื”ื’ื•ืœืฉื™ื .\n'
              '\n'
              'ื’ืœื™ืฉื” ื ืขื™ืžื” ื•ืคืฉืคื•ืฉ ืžื”ื ื” ืœื›ื•ืœื,\n'
              '\n'
              'ืคืฉืคืฉื•ืง ื“ื™ืจื•ืช ืœื”ืฉื›ืจื”/ืžื›ื™ืจื”',
 'post_url': 'https://m.facebook.com/groups/Pishpeshuk.Nadlan/permalink/3009648419116144/',
 'reaction_count': None,
 'reactions': None,
 'reactors': None,
 'shared_post_id': None,
 'shared_post_url': None,
 'shared_text': '',
 'shared_time': None,
 'shared_user_id': None,
 'shared_username': None,
 'shares': 1,
 'text': 'ื‘ืจื•ื›ื™ื ื”ื‘ืื™ื ืœืคืฉืคืฉื•ืง ื“ื™ืจื•ืช-ืœื”ืฉื›ืจื”/ืžื›ื™ืจื” , ื‘ืงื‘ื•ืฆื” ื ื™ืชืŸ ืœืคืจืกื ื“ื™ืจื•ืช '
         'ืœืžื›ื™ืจื”/ืœื”ืฉื›ืจื”, ืกืื‘ืœื˜ ื•ืฉื•ืชืคื™ื ื‘ื›ืœ ื”ืืจืฅ, ื“ื™ืจื•ืช ื’ืŸ, ืคื ื˜ื”ืื•ื–ื™ื, ื“ื™ืจื•ืช '
         "ื™ื•ืงืจื”, ืžื—ืกื ื™ื, ื‘ืชื™ื, ื•ื™ืœื•ืช , ืงื•ื˜ื’'ื™ื ื•ื ื›ืกื™ื ืขืกืงื™ื™ื.\n"
         '\n'
         '*ืžืชื•ื•ื›ื™ื ืฉื™ืžื• ืœื‘ - ืžืชื•ื•ื›ื™ื ืžื•ืจืฉื™ื ืœืคืจืกื ืžื•ื“ืขื” ืื—ืช ื‘ื™ื•ื ื‘ืœื‘ื“, ื‘ืžื™ื“ื” '
         'ื•ืžืชื•ื•ืš ื™ืคืจืกื ืžืขืœ ืžื•ื“ืขื” ืื—ืช - ื”ื•ื ื™ื•ืกืจ ืžื”ืœื•ื—.\n'
         '\n'
         '*ืืœ ืชื—ืกื›ื• ื‘ืคืจื˜ื™ื - ื›ืชื‘ื• ื”ืื ืžื“ื•ื‘ืจ ื‘ืžื›ื™ืจื”/ื”ืฉื›ืจื”, ืกื•ื’ ื”ื ื›ืก, ืื–ื•ืจ ื”ื ื›ืก, '
         'ืžืฆื‘ ื”ื ื›ืก ื•ืขื•ื“ ืคืจื˜ื™ื ืฉื™ื•ื›ืœื• ืœืขื–ื•ืจ ืœืจื•ื›ืฉื™ื.\n'
         '\n'
         '*ืื ื—ื ื• ื ื’ื“ ืกืคืื - ืคืจืกื•ืžื™ื ืฉืœื ืงืฉื•ืจื™ื ืœืจื•ื— ื”ืงื‘ื•ืฆื”, ื™ื•ืกืจื• ื•ื”ื’ื•ืœืฉ ื™ื•ืจื—ืง '
         'ืœืฆืžื™ืชื•ืช.\n'
         '\n'
         '*ื™ืฉ ืœื”ืงืคื™ื“ ืขืœ ืคืจืกื•ื ืชืžื•ื ื•ืช + ืžื—ื™ืจ ืœืžืขืŸ ื ื•ื—ื™ื•ืช ื”ื’ื•ืœืฉื™ื .\n'
         '\n'
         'ื’ืœื™ืฉื” ื ืขื™ืžื” ื•ืคืฉืคื•ืฉ ืžื”ื ื” ืœื›ื•ืœื,\n'
         '\n'
         'ืคืฉืคืฉื•ืง ื“ื™ืจื•ืช ืœื”ืฉื›ืจื”/ืžื›ื™ืจื”',
 'time': datetime.datetime(2020, 6, 26, 6, 11, 47),
 'user_id': '407003609318394',
 'user_url': 'https://facebook.com/pishpeshuk.co.il/?refid=18&_ft_=top_level_post_id.3009648419116144%3Acontent_owner_id_new.407003609318394%3Apage_id.407003609318394%3Astory_location.6%3Astory_attachment_style.photo%3Atds_flgs.3%3Aott.AX9SGtYoZmErvghw%3Apage_insights.%7B%22407003609318394%22%3A%7B%22page_id%22%3A407003609318394%2C%22page_id_type%22%3A%22page%22%2C%22actor_id%22%3A407003609318394%2C%22dm%22%3A%7B%22isShare%22%3A0%2C%22originalPostOwnerID%22%3A0%7D%2C%22psn%22%3A%22EntGroupMallPostCreationStory%22%2C%22post_context%22%3A%7B%22object_fbtype%22%3A657%2C%22publish_time%22%3A1593108707%2C%22story_name%22%3A%22EntGroupMallPostCreationStory%22%2C%22story_fbid%22%3A%5B3009648419116144%5D%7D%2C%22role%22%3A1%2C%22sl%22%3A6%2C%22targets%22%3A%5B%7B%22actor_id%22%3A407003609318394%2C%22page_id%22%3A407003609318394%2C%22post_id%22%3A3009648419116144%2C%22role%22%3A1%2C%22share_id%22%3A0%7D%5D%7D%2C%22303803599700653%22%3A%7B%22page_id%22%3A303803599700653%2C%22page_id_type%22%3A%22group%22%2C%22actor_id%22%3A407003609318394%2C%22dm%22%3A%7B%22isShare%22%3A0%2C%22originalPostOwnerID%22%3A0%7D%2C%22psn%22%3A%22EntGroupMallPostCreationStory%22%2C%22post_context%22%3A%7B%22object_fbtype%22%3A657%2C%22publish_time%22%3A1593108707%2C%22story_name%22%3A%22EntGroupMallPostCreationStory%22%2C%22story_fbid%22%3A%5B3009648419116144%5D%7D%2C%22role%22%3A1%2C%22sl%22%3A6%7D%7D&__tn__=C-R',
 'username': 'ืคืฉืคืฉื•ืง',
 'video': None,
 'video_duration_seconds': None,
 'video_height': None,
 'video_id': None,
 'video_quality': None,
 'video_size_MB': None,
 'video_thumbnail': None,
 'video_watches': None,
 'video_width': None,
 'w3_fb_url': None}
Read more comments on GitHub >

github_iconTop Results From Across the Web

Python getting incomplete next page URL (BeautifulSoup ...
The part i am struggling is with the WHILE loop in "for link in new_links". I am mostly looking for any example that...
Read more >
Python Scrapy tutorial for beginners - How to go to the next page
Right-click on the next button: The next page URL is inside an a tag, within a li tag. You know how to extract...
Read more >
How to get the next page on Beautiful Soup - Medium
Today, we are going to learn how to fetch all the items while Web Scraping by reaching to the next pages.
Read more >
EXPLOITING URL PARSERS: THE GOOD, BAD, AND ...
The confusion in URL parsing can cause unexpected behavior in the software ... An attacker-hosted class would not be loaded and the vulnerability...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found