Inefficient OneToMany Relations Query

Hard to tell which project is responsible for your ?join= query building, be it NestJSX or NestJS or TypeORM…but I noticed 2 queries are run instead of one.

Example: I setup a super basic 2 entity system. Posts and Spaces. A Space is simply a “category”. A Post can have ONE Space, so post.entity.ts is setup like so

@ManyToOne(type => Space, space => space.posts, { nullable: false })
space: Space;

While the inverse relation on space.entity.ts is setup properly like so

@OneToMany(type => Post, post => post.space)
posts: Post[];

I have my posts controller setup to JOIN spaces properly like so

@Crud({
  model: {
    type: Post,
  },
  query: {
    join: {
      space: {eager: true},
    },
    maxLimit: 25,
  },
})

Now when I run my basic GET entpoint via curl it will JOIN in the space but I believe it is doing so inefficiently.

The NestJS console output shows 2 queries like so

query1: SELECT DISTINCT `distinctAlias`.`Post_id` as "ids_Post_id" FROM (SELECT `Post`.`id` AS `Post_id`, `Post`.`slug` AS `Post_slug`, `Post`.`title` AS `Post_title`, `Post`.`body` AS `Post_body`, `Post`.`hidden` AS `Post_hidden`, `Post`.`deleted` AS `Post_deleted`, `Post`.`indexed_at` AS `Post_indexed_at`, `Post`.`created_at` AS `Post_created_at`, `Post`.`updated_at` AS `Post_updated_at`, `space`.`id` AS `space_id`, `space`.`name` AS `space_name`, `space`.`section` AS `space_section`, `space`.`slug` AS `space_slug`, `space`.`order` AS `space_order`, `Post`.`parent_id`, `Post`.`spaceId`, `Post`.`format_id`, `Post`.`created_by`, `Post`.`updated_by` FROM `posts` `Post` INNER JOIN `spaces` `space` ON `space`.`id`=`Post`.`spaceId`) `distinctAlias` ORDER BY `Post_id` ASC LIMIT 25

query2: SELECT `Post`.`id` AS `Post_id`, `Post`.`slug` AS `Post_slug`, `Post`.`title` AS `Post_title`, `Post`.`body` AS `Post_body`, `Post`.`hidden` AS `Post_hidden`, `Post`.`deleted` AS `Post_deleted`, `Post`.`indexed_at` AS `Post_indexed_at`, `Post`.`created_at` AS `Post_created_at`, `Post`.`updated_at` AS `Post_updated_at`, `space`.`id` AS `space_id`, `space`.`name` AS `space_name`, `space`.`section` AS `space_section`, `space`.`slug` AS `space_slug`, `space`.`order` AS `space_order`, `Post`.`parent_id`, `Post`.`spaceId`, `Post`.`format_id`, `Post`.`created_by`, `Post`.`updated_by` FROM `posts` `Post` INNER JOIN `spaces` `space` ON `space`.`id`=`Post`.`spaceId` WHERE `Post`.`id` IN (1, 2, 3, 4, 5, 6)

So the first query is simply trying to get a list of post IDs, which are 1, 2, 3, 4, 5, 6

Then the second query is performing an IN statement with those same IDs. There are 2 problems with that. One is the first query is inefficient because it is querying ALL COLUMNS in its select. All it needs is just Post.id is its sub select because that is all its counting.

Second is, I don’t get the point of the first query at all. Since this is a basic OneToMany simply running one single query with a JOIN and NO IN statement would do the same thing. Basically your second query is right but you don’t need the IN statement, just

SELECT
	`Post`.`id` AS `Post_id`,
	`Post`.`slug` AS `Post_slug`,
	`Post`.`title` AS `Post_title`,
	`Post`.`body` AS `Post_body`,
	`Post`.`hidden` AS `Post_hidden`,
	`Post`.`deleted` AS `Post_deleted`,
	`Post`.`indexed_at` AS `Post_indexed_at`,
	`Post`.`created_at` AS `Post_created_at`,
	`Post`.`updated_at` AS `Post_updated_at`, 
	`space`.`id` AS `space_id`,
	`space`.`name` AS `space_name`,
	`space`.`section` AS `space_section`,
	`space`.`slug` AS `space_slug`,
	`space`.`order` AS `space_order`,
	`Post`.`parent_id`,
	`Post`.`spaceId`,
	`Post`.`format_id`,
	`Post`.`created_by`,
	`Post`.`updated_by`
FROM
	`posts` `Post`
	INNER JOIN `spaces` `space` ON `space`.`id`=`Post`.`spaceId`

Maybe I am missing the point to the first query, but its just getting a list of IDs to use later in the IN statement which I can’t see the point too.

[mreschke]

Not sure as I have since switched away from NestJS as I feel my API’s should be in python. Why? Because my APIs always get more complicated than just a CRUD endpoint. Endpoints that build reports, query other systems (including hardware), perform complex data analysis…python has more libraries and is a bit more powerful for me. Nothing wrong with NestJS or the node ecosystem, love the project and will use it in the future when applicable.

[michaelyali]

@mreschke

Endpoints that build reports, query other systems (including hardware), perform complex data analysis

That not the issue with NestJs at all. Your mistake was to rely on CRUD lib which just creates several endpoints with dumb af logic because it’s called CRUD lol 😃 I’ve never used my lib when I’m building really big and complex projects because it’s really obvious that I would need something more than just CRUD endpoints.