question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Update performance for nullable int/decimal types

See original GitHub issue

As mentioned in #3253 the fix for that issue did not solve what we are seeing in AWS. I used @aarondk-dev solution as a starting point and with @zacbre’s help were able to isolate the problem. Here’s a solution to help reproduce the issue:

https://github.com/pebezo/Linq2DbUpdate

The good news is that it’s not dependent on MySQL, Docker, or anything in AWS. The examples below use Sqlite. Also, the issue seems to happen only on nullable int and nullable decimals. Here are the results for version 3.5.0

Method Mean Error StdDev Median Ratio RatioSD Allocated
Compiled 244.5 us 14.57 us 60.90 us 223.0 us 1.00 0.00 63 KB
String 446.1 us 23.54 us 98.62 us 407.5 us 1.93 0.63 128 KB
String_Nullable 450.1 us 25.08 us 105.91 us 406.1 us 1.93 0.58 127 KB
Int 285.0 us 12.94 us 52.34 us 270.0 us 1.23 0.31 83 KB
Int_Nullable 13,986.1 us 335.66 us 1,417.55 us 13,766.7 us 60.39 15.29 1,542 KB
DateTime 286.9 us 10.67 us 41.91 us 276.8 us 1.24 0.29 85 KB
DateTime_Nullable 343.9 us 20.75 us 81.53 us 316.1 us 1.48 0.43 89 KB
Bool 320.4 us 18.58 us 77.66 us 293.5 us 1.38 0.43 82 KB
Bool_Nullable 285.6 us 10.80 us 43.54 us 281.2 us 1.24 0.32 82 KB
Decimal 305.5 us 16.75 us 69.44 us 288.0 us 1.31 0.38 84 KB
Decimal_Nullable 14,308.3 us 292.09 us 1,217.45 us 14,305.0 us 61.68 14.33 1,545 KB
Float 328.3 us 19.04 us 79.78 us 302.3 us 1.41 0.44 83 KB
Float_Nullable 286.0 us 10.14 us 40.68 us 285.6 us 1.24 0.29 86 KB

The Compiled run includes all the columns, that’s why some of the other tests are able to beat it. Notice how the Int_Nullable and Decimal_Nullable take 60x more than the rest.

In our AWS app we have a combination of int?/decimal? columns which explains that 100x slowness.

For kicks we ran the exact same benchmark against version 2.9.8:

Method Mean Error StdDev Median Ratio RatioSD Allocated
Compiled 156.1 us 0.37 us 1.53 us 155.8 us 1.00 0.00 52 KB
String 4,656.2 us 9.77 us 40.40 us 4,652.4 us 29.82 0.37 1,317 KB
String_Nullable 4,731.0 us 17.20 us 71.32 us 4,718.4 us 30.30 0.55 1,320 KB
Int 3,163.1 us 26.51 us 110.80 us 3,160.0 us 20.26 0.74 820 KB
Int_Nullable 3,101.4 us 22.99 us 95.07 us 3,098.2 us 19.86 0.64 816 KB
DateTime 136.3 us 0.48 us 1.99 us 136.1 us 0.87 0.02 58 KB
DateTime_Nullable 139.9 us 0.48 us 2.01 us 139.9 us 0.90 0.02 58 KB
Bool 121.7 us 0.27 us 1.12 us 121.5 us 0.78 0.01 53 KB
Bool_Nullable 126.3 us 0.28 us 1.17 us 126.1 us 0.81 0.01 54 KB
Decimal 3,562.3 us 17.69 us 70.31 us 3,565.6 us 22.82 0.49 823 KB
Decimal_Nullable 3,638.4 us 19.10 us 80.02 us 3,649.0 us 23.31 0.56 820 KB
Float 127.4 us 0.25 us 1.05 us 127.3 us 0.82 0.01 53 KB
Float_Nullable 135.3 us 0.42 us 1.72 us 135.1 us 0.87 0.01 57 KB

It looks like some scenarios are actually better than in the latest version; something that may be worth investigating…

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Reactions:2
  • Comments:10 (10 by maintainers)

github_iconTop GitHub Comments

2reactions
pebezocommented, Oct 22, 2021

Awesome work @MaceWindu 👏 🚀

master

Method Mean Error StdDev Ratio RatioSD Allocated
Compiled 234.7 us 30.07 us 30.88 us 1.00 0.00 56 KB
String 290.8 us 23.00 us 23.62 us 1.26 0.18 85 KB
String_Nullable 291.4 us 31.75 us 33.97 us 1.26 0.22 85 KB
Int 274.6 us 57.13 us 65.79 us 1.18 0.26 57 KB
Int_Nullable 256.6 us 50.08 us 57.68 us 1.07 0.24 57 KB
DateTime 298.4 us 54.63 us 60.72 us 1.26 0.20 61 KB
DateTime_Nullable 270.6 us 44.73 us 51.51 us 1.20 0.23 58 KB
Bool 242.3 us 31.01 us 35.71 us 1.02 0.17 56 KB
Bool_Nullable 248.0 us 39.53 us 42.30 us 1.04 0.18 57 KB
Decimal 250.7 us 39.16 us 43.52 us 1.09 0.25 58 KB
Decimal_Nullable 333.6 us 85.63 us 98.61 us 1.40 0.38 58 KB
Float 228.6 us 15.89 us 18.30 us 1.00 0.14 57 KB
Float_Nullable 262.8 us 37.60 us 41.79 us 1.11 0.20 57 KB

3.5.0

Method Mean Error StdDev Ratio RatioSD Allocated
Compiled 247.6 us 35.80 us 36.76 us 1.00 0.00 56 KB
String 417.5 us 31.60 us 35.13 us 1.73 0.28 128 KB
String_Nullable 463.0 us 47.42 us 46.57 us 1.89 0.30 126 KB
Int 357.7 us 58.69 us 67.59 us 1.44 0.30 83 KB
Int_Nullable 13,723.4 us 621.97 us 691.31 us 56.76 7.20 731 KB
DateTime 318.1 us 27.44 us 30.50 us 1.31 0.18 85 KB
DateTime_Nullable 387.8 us 82.01 us 94.44 us 1.60 0.47 88 KB
Bool 320.0 us 36.69 us 39.26 us 1.32 0.22 85 KB
Bool_Nullable 348.5 us 58.83 us 60.42 us 1.43 0.31 85 KB
Decimal 328.3 us 50.00 us 57.58 us 1.33 0.31 84 KB
Decimal_Nullable 13,366.7 us 517.98 us 554.23 us 54.75 7.33 730 KB
Float 314.2 us 25.24 us 28.05 us 1.29 0.17 83 KB
Float_Nullable 405.1 us 63.31 us 70.37 us 1.69 0.39 83 KB

2.9.8

Method Mean Error StdDev Ratio RatioSD Allocated
Compiled 157.7 us 2.11 us 2.16 us 1.00 0.00 48 KB
String 4,797.1 us 55.11 us 58.97 us 30.44 0.45 1,314 KB
String_Nullable 4,763.2 us 33.59 us 38.68 us 30.19 0.45 1,320 KB
Int 3,202.0 us 80.86 us 93.12 us 20.28 0.76 820 KB
Int_Nullable 3,193.4 us 68.28 us 78.63 us 20.20 0.62 816 KB
DateTime 136.6 us 1.17 us 1.30 us 0.87 0.01 61 KB
DateTime_Nullable 143.2 us 0.60 us 0.69 us 0.91 0.01 61 KB
Bool 124.8 us 0.57 us 0.61 us 0.79 0.01 53 KB
Bool_Nullable 128.1 us 1.93 us 2.15 us 0.81 0.02 54 KB
Decimal 3,725.9 us 57.68 us 66.43 us 23.62 0.58 820 KB
Decimal_Nullable 3,711.4 us 83.71 us 93.05 us 23.55 0.61 817 KB
Float 130.4 us 1.75 us 1.87 us 0.83 0.01 53 KB
Float_Nullable 134.3 us 1.48 us 1.59 us 0.85 0.02 54 KB
1reaction
MaceWinducommented, Oct 22, 2021

@pebezo nullable fix merged

Read more comments on GitHub >

github_iconTop Results From Across the Web

C# Decimal datatype performance
Accuracy is 100% for long datatypes, and unless you're working with vast numbers (use a 64-bit long type) you'll be ok. If you...
Read more >
Nullable micro-optimizations, part one
The first thing to notice is that a nullable value type's ability to represent a “null” integer or decimal or whatever is not...
Read more >
Why is an AVG Function on DECIMAL values throwing a ...
I've added WHERE clauses to ensure that it doesn't capture an empty value or the jsonb column value itself is not null. SELECT;...
Read more >
Optimize NULL values storage consumption using SQL ...
When you store a NULL value in a fixed-length column such as a column with INT data type, the NULL value will consume...
Read more >
Chapter 4, Optimizing Schema and Data Types
The performance improvement from changing NULL columns to NOT NULL is ... items int NOT NULL, time float NOT NULL, cost decimal(9,5) NOT...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found