Several SQL queries for each related item instead of one aggregated query

Bug report

Describe the bug

When executing very simple GraphQL query with querying data from one relationship field, Keystone executes separate query for each relation, instead of one aggregated query with list of needed ids

To Reproduce

1. Launch the “examples/basic” demo, enable database logging via enableLogging: true.
2. Fill 2 authors and 4 posts to database (some linked with author, some - not).
3. Execute this GraphQL query:

query {
  allPosts {
    id, author {id, name}
  }
}

1. Lookup to the output, you will see several SQL queries:

prisma:query SELECT `main`.`Post`.`id`, `main`.`Post`.`title`, `main`.`Post`.`status`, `main`.`Post`.`content`, `main`.`Post`.`publishDate`, `main`.`Post`.`author` FROM `main`.`Post` WHERE 1=1 LIMIT ? OFFSET ?
prisma:query SELECT 1
prisma:query SELECT `main`.`Author`.`id`, `main`.`Author`.`name`, `main`.`Author`.`email` FROM `main`.`Author` WHERE (`main`.`Author`.`id`) IN (SELECT `t0`.`id` FROM `main`.`Author` AS `t0` INNER JOIN `main`.`Post` AS `j0` ON (`j0`.`author`) = (`t0`.`id`) WHERE (`j0`.`id` = ? AND `t0`.`id` IS NOT NULL)) LIMIT ? OFFSET ?
prisma:query SELECT `main`.`Author`.`id`, `main`.`Author`.`name`, `main`.`Author`.`email` FROM `main`.`Author` WHERE (`main`.`Author`.`id`) IN (SELECT `t0`.`id` FROM `main`.`Author` AS `t0` INNER JOIN `main`.`Post` AS `j0` ON (`j0`.`author`) = (`t0`.`id`) WHERE (`j0`.`id` = ? AND `t0`.`id` IS NOT NULL)) LIMIT ? OFFSET ?
prisma:query SELECT `main`.`Author`.`id`, `main`.`Author`.`name`, `main`.`Author`.`email` FROM `main`.`Author` WHERE (`main`.`Author`.`id`) IN (SELECT `t0`.`id` FROM `main`.`Author` AS `t0` INNER JOIN `main`.`Post` AS `j0` ON (`j0`.`author`) = (`t0`.`id`) WHERE (`j0`.`id` = ? AND `t0`.`id` IS NOT NULL)) LIMIT ? OFFSET ?
prisma:query SELECT `main`.`Author`.`id`, `main`.`Author`.`name`, `main`.`Author`.`email` FROM `main`.`Author` WHERE (`main`.`Author`.`id`) IN (SELECT `t0`.`id` FROM `main`.`Author` AS `t0` INNER JOIN `main`.`Post` AS `j0` ON (`j0`.`author`) = (`t0`.`id`) WHERE (`j0`.`id` = ? AND `t0`.`id` IS NOT NULL)) LIMIT ? OFFSET ?

Problems:

1. Why each author queried via separate SQL query? And even when author is empty!
2. Why all fields are queried from “Post” table, when we need only id and author for query results?
3. Why author email is queried using inner query and even inner join inside it?

Those problems give large performance degradation on databases with many items!

Expected behavior

1. To optimize performance, Keystone must collect id of all relationships, and query them via one SQL query with list of need ids.
2. We should query only requested fields, not all available fields from table.
3. There must be no inner joins and subqueries for such simple query.

System information

Keystone version: 21.0.2

[molomby]

Partial fix underway in #6639 🥳

[molomby]

Ok, I’ve talked this though with @timleslie (apparently you and him have been chatting recently too). My current understanding is that, yes, Prisma does proactively apply a data loader pattern to avoid N+1 problems but only does so when it’s really obvious that it can (only findUnique calls using the the same parameters). The access control that Keystone applies unintentionally prevents this from occurring – we call findFirst instead of findUnique so we can supply the additional access filters.

There may be some low hanging fruit here whereby, maybe we can only call findFirst when access filters are actual in effect. This would allow some lists to take advantage of the automatic data loading Prisma provides. It still only solves for querying via many-to-one relationships; coming the direction (one-to-many) will trigger N+1 queries in any case. Otherwise, the longer term solution is to add a data loader layer in our own code and be smarter about how queries are constructed.

Regardless, you shouldn’t be relying on Keystone to always build optimal queries. If you have complex queries that are used regularly in your main frontend, you’re probably better moving to custom mutations that do exactly what’s needed. The CRUD schema Keystone provides is intended to support the Admin UI and as a starting point for general app dev.