question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[QUERY] [EventHub] PartitionKey HashCode Generation

See original GitHub issue

Library name and version

Azure.Messaging.EventHubs

Query/Question

When converting PartitionResolver.ComputeHash to span and BinaryPrimitives it would be possible to squeeze out some perf. See the below benchmark


BenchmarkDotNet=v0.13.1, OS=Windows 10.0.22000
11th Gen Intel Core i7-1185G7 3.00GHz, 1 CPU, 8 logical and 4 physical cores
.NET SDK=6.0.102
  [Host]     : .NET 6.0.2 (6.0.222.6406), X64 RyuJIT
  DefaultJob : .NET 6.0.2 (6.0.222.6406), X64 RyuJIT


Method Mean Error StdDev StdErr Median Min Q1 Q3 Max Op/s Ratio RatioSD
ComputeHashBitConverter 9.599 ns 0.3469 ns 1.007 ns 0.1022 ns 9.421 ns 7.989 ns 8.835 ns 10.279 ns 12.61 ns 104,174,105.6 1.00 0.00
ComputeHashSpan 8.792 ns 0.4335 ns 1.216 ns 0.1274 ns 8.444 ns 6.987 ns 7.875 ns 9.409 ns 12.60 ns 113,736,404.7 0.93 0.16

Is that something you’d be interested in?

Another thing. I noticed currently it is required to allocate a byte array to compute the hash and the size of the byte array is proportional to the size of the partition key. I see the following options:

  • Pool the buffer
  • Stackalloc the buffer for short string and for longer pool the buffer
  • If the EnqueueEventOptions are most likely to be reused per partition, it would be possible to turn the partition key into bytes there and the just pass those bytes into ComputeHash method (then the options act as sort of root for the byte array)

Just as an example, pooling the byte array there is slightly slower but gets rid of all the allocations on that path.


BenchmarkDotNet=v0.13.1, OS=Windows 10.0.22000
11th Gen Intel Core i7-1185G7 3.00GHz, 1 CPU, 8 logical and 4 physical cores
.NET SDK=6.0.102
  [Host]     : .NET 6.0.2 (6.0.222.6406), X64 RyuJIT
  DefaultJob : .NET 6.0.2 (6.0.222.6406), X64 RyuJIT


Method Mean Error StdDev StdErr Min Q1 Median Q3 Max Op/s Ratio RatioSD Gen 0 Allocated
Pooled 67.84 ns 1.171 ns 1.038 ns 0.277 ns 66.13 ns 67.12 ns 67.67 ns 68.67 ns 69.39 ns 14,740,063.2 1.15 0.09 - -
NonPooled 55.45 ns 1.686 ns 4.892 ns 0.497 ns 47.37 ns 51.71 ns 54.57 ns 58.57 ns 68.62 ns 18,033,322.7 1.00 0.00 0.0102 64 B

Environment

No response

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:10 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
jsquirecommented, Mar 3, 2022

As always, thank you, @danielmarbach. I’m going to mark this one as addressed, but feel free to reach out if there’s any additional context that I can help with.

1reaction
danielmarbachcommented, Feb 24, 2022

Cool will think about it and once I have sufficient time I’ll send a PR

Read more comments on GitHub >

github_iconTop Results From Across the Web

What hash function is used for partitionkey mapping to ...
The partition key is a sender-supplied value passed into an event hub. It is processed through a static hashing function, which creates the ......
Read more >
[Event Hubs] Fix endianness when hashing partition keys
The hashing algorithm is sensitive to the endianness of the local machine: ...
Read more >
Features and terminology in Azure Event Hubs
The partition key is a sender-supplied value passed into an event hub. It is processed through a static hashing function, which creates the ......
Read more >
Azure Event Hub Implementation Using .Net Core Console ...
In this article, you will learn about Azure Event Hub implementation ... Partition Key. Used to generate the hash key of the event...
Read more >
Azure Event Hubs - Apache Camel
Sets a hashing key to be provided for the batch of events, which instructs the Event Hubs service to map this key to...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found