Massive performance issue querying entities with many relations
See original GitHub issueIssue Description
I have a query builder I am running to query for items in a table and loading their relations
let deviceQuery = deviceRepo
.createQueryBuilder('device')
.leftJoinAndSelect('device.tankType', 'tankType')
.leftJoinAndSelect('device.customer', 'customer')
.leftJoinAndSelect('device.organization', 'organization')
.leftJoinAndSelect('device.location', 'location')
const devices: Device[] = await deviceQuery.getMany()
This table only has 7 items on it. When executing this query it takes upwards of 2500ms to return. If I replace .getMany(
) with .getRawMany()
the query only takes 140ms. I can also take the generated SQL from typeorm and paste it into PGAdmin and the query will also only take 140ms there. Something is going terribly wrong with the mapping process to the entity.
Expected Behavior
For .getMany()
to be performant when querying for entities that have only 4 relations on them.
Actual Behavior
.getMany()
is 15x slower than .getRawMany()
.
Steps to Reproduce
- Create an entity with 4+
@ManyToOne()
/@OneToMany()
relations - compare
.getMany()
to.getRawMany()
My Environment
Dependency | Version |
---|---|
Operating System | Windows |
Node.js version | v12.18.3 |
Typescript version | v3.9.7 |
TypeORM version | v0.2.29 |
Additional Context
Relevant Database Driver(s)
-
aurora-data-api
-
aurora-data-api-pg
-
better-sqlite3
-
cockroachdb
-
cordova
-
expo
-
mongodb
-
mysql
-
nativescript
-
oracle
-
postgres
-
react-native
-
sap
-
sqlite
-
sqlite-abstract
-
sqljs
-
sqlserver
Are you willing to resolve this issue by submitting a Pull Request?
- Yes, I have the time, and I know how to start.
- Yes, I have the time, but I don’t know how to start. I would need guidance.
- No, I don’t have the time, although I believe I could do it if I had the time…
- No, I don’t have the time and I wouldn’t even know how to start.
Issue Analytics
- State:
- Created 3 years ago
- Reactions:5
- Comments:9 (5 by maintainers)
Top Results From Across the Web
Efficient Querying - EF Core - Microsoft Learn
Performance guide for efficient querying using Entity Framework Core. ... As more one-to-many relationships are loaded, the amount of ...
Read more >Entity-framework code is slow when using Include() many times
If at least one entity does not have a relationship to some of other entities, then EF will not able to construct one...
Read more >10 Common Hibernate Mistakes That Cripple Your Performance
Learn how to fix your performance problems by avoiding the most common Hibernate mistakes.
Read more >3 Common Hibernate Performance Issues in Your Log
A lot of Hibernate performance issues can be found in the log file. In this article, I show you 3 commons ones and...
Read more >Five levels of performance tuning for an EF Core query
In my book “Entity Framework Core in Action, 2 edition” I use this Book App as an example of using various EF Core...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
It was the
@RelationId
s that caused all the problems. When I would execute a.getMany()
it would run about 11 queries. Picking one of these extra queries out and running it in pgadmin I noticed it was just giving me lists of Ids. This was a massive misunderstanding on my part on the cost of@RelationId
. Running postgres locally was just so lightning fast it masked the “real” query time. I thought they were free, and added paired it with every relationship I had even if it wasn’t needed at the time.Thank you so much for leading me to the right conclusion.
I think the loading of RelationIds can be improved though. When digging into all of this it seemed that when running the 11 queries instead of running in parallel they were waiting for the previous to finish. This is evident by the total query time seeming to be the sum of all 11 queries. These queries are not dependent on each other and should be able to be executed at the same time making this take as long as the slowest query.
Glad you got it figured out.
@RelationId
is still an experimental feature so hopefully that will be improved, but there should probably be more information about how it works in the background to avoid confusion like this. Generally the ORM hides the fact that one-to-many relations are much more complex than many-to-one. I personally avoid using them except for single item queries.