question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

getCount(). Perfs issues for multi columns primary key

See original GitHub issue

Issue type:

[X ] question

Database system/driver: [X ] postgres

Version 0.2.18

I’m facing a perf issue when running getCount() Why when counting all records of a query, queryBuilder creates a SQL query that concatenates the columns that are part of the primary key ?

Typescript :

const query: SelectQueryBuilder<TEntity> = dbConnection
					.getRepository<TEntity>(entity)
					.createQueryBuilder(entity.name)
					.where(`"${entity.name}"."${id}"= :val`, {
						val: value.toLowerCase(),
					});

				const totalRecords: number = await query.getCount();

generated sql

SELECT COUNT(DISTINCT(
	CONCAT(
		"u"."a", 
		"u"."b",
		"u"."c")
	)
) as "cnt" FROM "sc"."tbl" "u"

That is destroying the performances for a table of 140K records. Creating an index on CONCAT using is not efficient, hard to maintain with 100 of tables, knowing that for some tables, some primay comumns are timestamp…

Is there a way or a parameter to set to avoid the concat and just running a kind of (select count(1) FROM "sc"."tbl" "u") ?

Rgds

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:7 (3 by maintainers)

github_iconTop GitHub Comments

2reactions
dsbertcommented, May 1, 2020

Note this also causes incorrect counts to be returned.

Here is an example.

You have two number columns set as primary keys - field1 and field2.

The following two rows will return the incorrect count.

{
	field1: 11,
	field2: 101
	// concat(field1, field2) = '11101'
},
{
	field1: 1,
	field2: 1101
	// concat(field1, field2) = '11101'
},

Count should be 2, but returns 1.

0reactions
imnotjamescommented, Oct 8, 2020

Can you confirm that #6870 fixes the issues you’re seeing?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Counting DISTINCT over multiple columns - Stack Overflow
My specific problem required me to divide a SUM by the COUNT of the distinct combination of various foreign keys and a date...
Read more >
Is a two column primary key the most performant way to go if ...
The primary key is not there as a mechanism to improve performance (by adding an index) it is there to guarantee data consistency....
Read more >
Primary Key Constraint | CockroachDB Docs
The PRIMARY KEY constraint specifies that the constrained columns' values must uniquely identify each row. A table can only have one primary key, ......
Read more >
Is a composite primary key a good idea? - Ask TOM
Is a composite primary key a good idea? Tom,I have a table that has 3 fields that, when considered in combination, could uniquely...
Read more >
The right column order in multi-column indexes
In that case the database creates an index on all primary key columns—a so-called concatenated ... but it causes serious performance problems in...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found