Use of Double and Decimal .Net Type via NpgsqlCommand Parameters causes massive query performance degradation

Steps to reproduce

The issue

The same parameterized sql query runs orders of magnitude slower on millions of rows (see below for timing) if Double and Decimal .Net types are used as NpgsqlCommand Parameters as opposed to .Net Short and Int32 types.

The sql query look something like:

select * from product 
WHERE (product.type_id = @type_id) 
ORDER BY product.created_date DESC NULLS LAST
LIMIT @limit OFFSET @offset

The type_id in PostgreSQL is a smallint NOT NULL column, and is indexed. The created_date in PostgreSQL is a created_date timestamp without time zone NOT NULL column, and is indexed.

There is also another composite index:

CREATE INDEX product_created_date_type_id_idx
    ON product USING btree
    (created_date DESC NULLS LAST, type_id DESC NULLS LAST);	

Search using .Net Decimal - Run [1] - CreatedDate Desc - [19928.7159]ms
Search using .Net Decimal - Run [2] - CreatedDate Desc - [19132.9099]ms
Search using .Net Decimal - Run [3] - CreatedDate Desc - [19578.5421]ms
Search using .Net Decimal - Run [4] - CreatedDate Desc - [22294.7393]ms
Search using .Net Decimal - Run [5] - CreatedDate Desc - [19940.8091]ms

Search using .Net Int32 - Run [1] - CreatedDate Desc - [2673.4479]ms
Search using .Net Int32 - Run [2] - CreatedDate Desc - [2706.07]ms
Search using .Net Int32 - Run [3] - CreatedDate Desc - [3031.2587]ms
Search using .Net Int32 - Run [4] - CreatedDate Desc - [2584.9249]ms
Search using .Net Int32 - Run [5] - CreatedDate Desc - [2866.3663]ms

Search using .Net Double - Run [1] - CreatedDate Desc - [19027.7723]ms
Search using .Net Double - Run [2] - CreatedDate Desc - [19355.4194]ms
Search using .Net Double - Run [3] - CreatedDate Desc - [18845.8106]ms
Search using .Net Double - Run [4] - CreatedDate Desc - [19691.8726]ms
Search using .Net Double - Run [5] - CreatedDate Desc - [19467.7988]ms

Further technical details

Npgsql version: 4.0.3 PostgreSQL version: 10.5 Operating system: Windows 10

Other details about my project setup:

Have also tested using type_id as a .Net Short type and just as fast as Int32 type.

[YohDeadfall]

Npgsql doesn’t parse primitive numeric types. It changes the byte order using a single instruction and sends the value. The decimal type isn’t primitive, but its processing is fast too, but the timing depends on the provided value.

So as I said before it’s not a Npgsql problem. In your case you should swap columns in the index, so type_id will be first. Then PostgreSQL will use the index to filter values first and just read the required part of the index.

Check the query plans using EXPLAIN and compare them.

[roji]

In addition to all of @YohDeadfall’s correct comments above, it’s a pretty bad idea to benchmark this kind of thing with StopWatch. Consider making a quick benchmark with BenchmarkDotNet - it’s extremely easy to do and provides reliable measurements.

I’m sure that if you benchmark the right thing without any extra unrelated operations (e.g. parsing string to decimal), you’d find that Npgsql does things quite quickly. If you still think there’s a perf issue, please open a separate issue with a BenchmarkDotNet code fragment that demonstrates it.