question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

DjangoConnectionField.resolve_connection replaces resolved queryset with clone, wiping query cache (causing N+1)

See original GitHub issue

I’m new to graphene-django so please bear with me if my issue is nonsensical.

I’ve been spending some time trying to figure out why, when using a resolution method in combination with a DjangoFilterConnectionField any prefetches are being unused. I’ve traced it down to a few things.

Within DjangoFilterConnectionField.merge_querysets an & operator is used to merge two Django QuerySets. The __and__ method on QuerySet creates a new instance of its class, thereby losing any internal caching that may have been accomplished when using prefetch_related(). There doesn’t seem to be much of a way around this without accessing private attributes of the QuerySet object, which is clearly bad practice.

I then noticed that DjangoConnectionField.resolve_connection checks to see if iterable is not default_manager. If it’s not (which as far as I can tell it never will be if you have a resolve_things method on your Query class) it will then follow down the aforementioned path which winds up wiping cache because it creates a new QuerySet.

I wonder, what is the purpose of the following code in resolve_connection? If I disable it, the queryset my resolve_things method returns is actually used and thus the prefetch works.

            if iterable is not default_manager:
                default_queryset = maybe_queryset(default_manager)
                iterable = cls.merge_querysets(default_queryset, iterable)

For reference, my implementation is as follows:

class PostType(DjangoObjectType):
    class Meta:
        model = Post
        filter_fields = ['id', 'title']
        interfaces = (graphene.relay.Node, )


class Query(graphene.ObjectType):
    posts = DjangoFilterConnectionField(PostType)
    
    def resolve_posts(self, info, **kwargs):
        return Post.objects.prefetch_related('customer_posts').all()

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Reactions:2
  • Comments:15 (2 by maintainers)

github_iconTop GitHub Comments

2reactions
zbyte64commented, Oct 2, 2019

merge_queryset is also impacting #758 & #787

I don’t see an easy way to fix this. Maybe instead of trying to union querysets we pass through a single queryset.

1reaction
dani0805commented, Aug 31, 2018

We ended up overriding the method like this:

class PrefetchingConnectionField(DjangoConnectionField):

    @classmethod
    def merge_querysets(cls, default_queryset, queryset):
        return queryset

Now in your model you redefine the field and implement resolve.

So far it works, but I would love to know what am I breaking with this:

# Child has a FK to Parent and the reverse label is "children"
class ParentNode(DjangoObjectType, CommonInteractionNode):
    ...
    children = PrefetchingConnectionField(ChildNode)

    class Meta:
        model = Parent
        interfaces = (Node,)
        filter_fields = []

    def resolve_children(self, info):
        return self.children.all()

# Parent has a FK to Root and you want to prefetch in resolve_parents
class RootNode(DjangoObjectType):
    class Meta:
        model = Root
        interfaces = (Node,)
        filter_fields = []
        
    def resolve_parents(self, info):
        return Parents.objects.filter(root__id=self.id).select_related(
            "some_fk",
        ).prefetch_related(
            "children"
       )
  

Read more comments on GitHub >

github_iconTop Results From Across the Web

No results found

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found