AssertionError: Please file a bug report with PRAW

Describe the Bug

I have been trying to iterate over the comments in a submission and an assertion error is raised on every attempt.

Desired Result

The script should be able to iterate over every available comment.

Relevant Logs

Traceback (most recent call last):
  File "test.py", line 8, in <module>
    submission.comments.replace_more(limit=None)
  File "/home/henry/.local/lib/python3.8/site-packages/praw/util/deprecate_args.py", line 43, in wrapped
    return func(**dict(zip(_old_args, args)), **kwargs)
  File "/home/henry/.local/lib/python3.8/site-packages/praw/models/comment_forest.py", line 183, in replace_more
    new_comments = item.comments(update=False)
  File "/home/henry/.local/lib/python3.8/site-packages/praw/util/deprecate_args.py", line 43, in wrapped
    return func(**dict(zip(_old_args, args)), **kwargs)
  File "/home/henry/.local/lib/python3.8/site-packages/praw/models/reddit/more.py", line 70, in comments
    assert self.children, "Please file a bug report with PRAW."
AssertionError: Please file a bug report with PRAW.

Code to reproduce the bug

import praw

submission = reddit.submission('z1c9z')

print(submission.title)

comments = []
submission.comments.replace_more(limit=None)
for comment in submission.comments.list():
    comments.append(comment.id)

print(len(comments))

My code example does not include the Reddit() initialization to prevent credential leakage.

Yes

This code has previously worked as intended.

No

Operating System/Environment

Ubuntu 20.04.5 LTS on Windows 10 x86_64

Python Version

python3.8.10

PRAW Version

7.6.1

Prawcore Version

2.3.0

Anything else?

I’ve run this script against other submissions with a large number of comments and I get the same error each time. Additional post IDs:

• 2nbslo
• 39bpam
• 29qfnm

When I run the same script against a selection of small posts I do not get this error and the script executes as expected.

[LilSpazJoekp]

This appears to be fixed in my testing earlier today. Though, I couldn’t test fully to 100% confirm it is fixed.

[LilSpazJoekp]

So here’s a summary of my findings: (TL;DR Reddit broke something in their /api/morecomments endpoint and there isn’t much we can do about it at the moment)

When you call replace_more PRAW will start replacing all MoreComments instances with its children comments. These can be either Comment or MoreComments instances. In posts with a bunch of comments (1k+ comments), the last comment will basically be the overflow of the rest of the comments (which is a MoreComments instance itself). This last instance of MoreComments can have thousands of children comments. Normally, this isn’t an issue because you will just need to request the comments for that instance with the /api/morecomments endpoint and you’ll get back more comments (which can be a mixture of Comment and MoreComment instances) and the last one will be another large MoreComments instance. This continues until all the instances are replaced or the limit (the number of MoreComments it will replace, by default this is 32) in the replace_more is reached. Side note, the reason why many people are seeing this is because PRAW starts with the biggest MoreComments first.

Now here is where it falls apart, the first time PRAW takes this last MoreComments and hits the /api/morecomments endpoint to fill those in. It gets the expected comments back, however, the last child of that response is another MoreComments instance (notice the count of 17524) that is corrupted/incomplete:

{
    "json": {
        "data": {
            "things": [
                ...,
                {
                    "data": {
                        "children": [],
                        "count": 17524,
                        "depth": 0,
                        "id": "s",
                        "name": "t1_s",
                        "parent_id": "t3_2nbslo"
                    },
                    "kind": "more"
                }
            ]
        },
        "errors": []
    }
}

The assertion that everyone is seeing is there to make sure there is children to actually fetch. And with this incomplete MoreComments instance, this will prevent PRAW from fetching the majority of the comments (basically PRAW can get the first and second page of comments you can see on the site).

We have a few solutions to this:

• Remove the assertion
  • this will result in an error later on because PRAW tries to fetch an empty set of comments and it isn’t written to handle that
• Remove the bad MoreComments instance
  • This will cause the replace_more to only replace a small subset of MoreComments.

Both of these solutions are not good and our best bet is that Reddit will fix this bug, and soon. I suspect Reddit has made a change without considering the public API because the website seems unaffected by this and there appears to be different format (c1:t1_c60n8gi,t1_xxxxx,t1_xxxxx,) that Reddit is requesting from the /api/morecomments endpoint.

It seems @bboe beat me to the punch on my comment but it just confirms my findings.