[QUERY] [EventHub] PartitionKey HashCode Generation
See original GitHub issueLibrary name and version
Azure.Messaging.EventHubs
Query/Question
When converting PartitionResolver.ComputeHash to span and BinaryPrimitives it would be possible to squeeze out some perf. See the below benchmark
BenchmarkDotNet=v0.13.1, OS=Windows 10.0.22000
11th Gen Intel Core i7-1185G7 3.00GHz, 1 CPU, 8 logical and 4 physical cores
.NET SDK=6.0.102
[Host] : .NET 6.0.2 (6.0.222.6406), X64 RyuJIT
DefaultJob : .NET 6.0.2 (6.0.222.6406), X64 RyuJIT
Method | Mean | Error | StdDev | StdErr | Median | Min | Q1 | Q3 | Max | Op/s | Ratio | RatioSD |
---|---|---|---|---|---|---|---|---|---|---|---|---|
ComputeHashBitConverter | 9.599 ns | 0.3469 ns | 1.007 ns | 0.1022 ns | 9.421 ns | 7.989 ns | 8.835 ns | 10.279 ns | 12.61 ns | 104,174,105.6 | 1.00 | 0.00 |
ComputeHashSpan | 8.792 ns | 0.4335 ns | 1.216 ns | 0.1274 ns | 8.444 ns | 6.987 ns | 7.875 ns | 9.409 ns | 12.60 ns | 113,736,404.7 | 0.93 | 0.16 |
Is that something you’d be interested in?
Another thing. I noticed currently it is required to allocate a byte array to compute the hash and the size of the byte array is proportional to the size of the partition key. I see the following options:
- Pool the buffer
- Stackalloc the buffer for short string and for longer pool the buffer
- If the
EnqueueEventOptions
are most likely to be reused per partition, it would be possible to turn the partition key into bytes there and the just pass those bytes intoComputeHash
method (then the options act as sort of root for the byte array)
Just as an example, pooling the byte array there is slightly slower but gets rid of all the allocations on that path.
BenchmarkDotNet=v0.13.1, OS=Windows 10.0.22000
11th Gen Intel Core i7-1185G7 3.00GHz, 1 CPU, 8 logical and 4 physical cores
.NET SDK=6.0.102
[Host] : .NET 6.0.2 (6.0.222.6406), X64 RyuJIT
DefaultJob : .NET 6.0.2 (6.0.222.6406), X64 RyuJIT
Method | Mean | Error | StdDev | StdErr | Min | Q1 | Median | Q3 | Max | Op/s | Ratio | RatioSD | Gen 0 | Allocated |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Pooled | 67.84 ns | 1.171 ns | 1.038 ns | 0.277 ns | 66.13 ns | 67.12 ns | 67.67 ns | 68.67 ns | 69.39 ns | 14,740,063.2 | 1.15 | 0.09 | - | - |
NonPooled | 55.45 ns | 1.686 ns | 4.892 ns | 0.497 ns | 47.37 ns | 51.71 ns | 54.57 ns | 58.57 ns | 68.62 ns | 18,033,322.7 | 1.00 | 0.00 | 0.0102 | 64 B |
Environment
No response
Issue Analytics
- State:
- Created 2 years ago
- Comments:10 (7 by maintainers)
Top Results From Across the Web
What hash function is used for partitionkey mapping to ...
The partition key is a sender-supplied value passed into an event hub. It is processed through a static hashing function, which creates the ......
Read more >[Event Hubs] Fix endianness when hashing partition keys
The hashing algorithm is sensitive to the endianness of the local machine: ...
Read more >Features and terminology in Azure Event Hubs
The partition key is a sender-supplied value passed into an event hub. It is processed through a static hashing function, which creates the ......
Read more >Azure Event Hub Implementation Using .Net Core Console ...
In this article, you will learn about Azure Event Hub implementation ... Partition Key. Used to generate the hash key of the event...
Read more >Azure Event Hubs - Apache Camel
Sets a hashing key to be provided for the batch of events, which instructs the Event Hubs service to map this key to...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
As always, thank you, @danielmarbach. I’m going to mark this one as addressed, but feel free to reach out if there’s any additional context that I can help with.
Cool will think about it and once I have sufficient time I’ll send a PR