question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

queryset.count() support field and distinct parameter

See original GitHub issue

Filter model with fk field then use distinct().count() get wrong count num I have two model like:

class Client(Model):
    uuid = fields.UUIDField(pk=True, description='ID')
    name = fields.CharField(max_length=255, description='name')

class Ability(Model):
    id = fields.UUIDField(pk=True, description='ID')
    client = fields.ForeignKeyField(model_name='models.Client', related_name='client_abilities',
                                    description='client')
    type = fields.SmallIntField(description='type ')
    name = fields.CharField(max_length=255, description='name')

then i create there client and 6 ability like:

c1 = Client(name='c1')
c2 = Client(name='c2')
c3 = Client(name='c3')

a1 = Ability(client=c1, type=1, name='a1')
a2 = Ability(client=c1, type=1, name='a2')
a3 = Ability(client=c2, type=1, name='a3')
a4 = Ability(client=c2, type=1, name='a4')
a5 = Ability(client=c3, type=1, name='a5')
a6 = Ability(client=c3, type=1, name='a6')

then I filter Client like:

await Client.filter(client_abilities__type=1).distinct() it will return [c1,c2,c3] await Client.filter(client_abilities__type=1).distinct().count() it will return 6, but the expected num is 3.

then i print sql for this two orm statement: Client.filter(client_abilities__type=1).distinct().sql() get something like:

SELECT DISTINCT `client`.`uuid`,`client`.`name`  FROM `client` LEFT OUTER JOIN `ability` ON `client`.`uuid`=`ability`.`client_id` WHERE `ability`.`type`=1

exec the sql i get this

uuid	name
5afaf55a-e175-4a38-8b11-205d4d99ad5e	demo
892b2ee2-cb5a-498b-a408-6c431553b721	demo
db94723c-ce25-4fc4-bf90-0f23800c2aae	demo

await Client.filter(client_abilities__type=1).distinct().count().sql() get something like:

SELECT COUNT(*) FROM `client` LEFT OUTER JOIN `ability` ON `client`.`uuid`=`ability`.`client_id` WHERE `client`.`delete`=false AND `ability`.`type`=1

exec the sql i get this

COUNT(*)
6

Describe the solution you’d like exce sql:

SELECT COUNT(DISTINCT `client`.`uuid`) FROM `client` LEFT OUTER JOIN `ability` ON `client`.`uuid`=`ability`.`client_id` WHERE `client`.`delete`=false AND `ability`.`type`=1

i can get right count:

COUNT(DISTINCT `client`.`uuid`)
3

so,i want tortoise-orm support something like(if this is no better way): Client.filter(**filters).distinct().count('uuid', distinct=True)

Describe alternatives you’ve considered I think the key point is venv/Lib/site-packages/tortoise/queryset.py Line 899

    def _make_query(self) -> None:
        self.query = copy(self.model._meta.basequery)
        self.resolve_filters(
            model=self.model,
            q_objects=self.q_objects,
            annotations=self.annotations,
            custom_filters=self.custom_filters,
        )
        self.query._select_other(Count("*"))

i try change self.query._select_other(Count("*")) to self.query._select_other(Count(field)).distinct(), but it not work,i dont know how to change uuid to client`.`uuid

looking forward to your reply! thanks

Issue Analytics

  • State:open
  • Created 3 years ago
  • Reactions:3
  • Comments:5 (1 by maintainers)

github_iconTop GitHub Comments

4reactions
asitm9commented, Dec 11, 2020

Queryset.count() support field and distinct parameter

To enable this feature the following changes are required:

    if param and distinct:
        self.query._select_other(Count(Field(param)).distinct())
    self.query._select_other(Count("*"))

The query would look like Client.filter(**filters).count('uuid', distinct=True)

I have recently started exploring the tortoise-orm code, if we have any better alternative to pass the argument to the count method, please let me know.

I can modify the test code for the change & can create a PR.

0reactions
jonathanspwcommented, Mar 6, 2021

You can just use annotate instead, like:

await Client.filter(client_abilities__type=1).annotate(count=Count("uuid", distinct=True)).values("count")

Works beautifully, thanks!

Read more comments on GitHub >

github_iconTop Results From Across the Web

django - Get distinct values of Queryset by field - Stack Overflow
What you want is: Visit.objects.filter(stuff).values("ip_address").annotate(n=models.Count("pk")). What this does is get all ip_addresses ...
Read more >
QuerySet API reference | Django documentation
Django provides a count() method for precisely this reason. ... The query parameter to QuerySet exists so that specialized query subclasses can reconstruct ......
Read more >
Query API - SQLAlchemy 1.4 Documentation
The ORM-level distinct() call includes logic that will automatically add columns from the ORDER BY of the query to the columns clause of...
Read more >
Aggregation — Django 4.1.3 documentation
Book.objects.count() 2452 # Total number of books with publisher=BaloneyPress > ... however, the Count aggregate has a distinct parameter that may help:.
Read more >
Legacy SQL Functions and Operators | BigQuery - Google Cloud
Returns the exact number of non-NULL, distinct values for the specified field. FIRST(), Returns the first sequential value in the scope of the...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found