Distributed tracing story with Confluent Kafka client

Greetings!

I’m using OpenTelemetry to instrument an application, primarily to watch the performance of all its dependencies and find bottlenecks. OpenTelemetry is automatically capturing Http requests coming into my system and then tracing outgoing Http, SQL, & Redis calls made while doing the processing. All of that data is exported into one of the many tools OpenTelemetry supports for visualization. What I would like to do is mod the Confluent Kafka client to also participate in that process.

The easiest way to do that would be to use DiagnosticSource to allow other code in the process to subscribe to events that are fired as messages are produced or consumed. DiagnosticSource only fires when there is a subscriber so the cost to existing users not using the events would be ~0.

If we are open to this, I can take a stab at a PR, documentation, etc., but I wanted to check first.

/cc @SergeyKanzhelev @MikeGoldsmith @cijothomas

[MoienTajik]

Are there any plans to implement this feature?

[fedeoliv]

Hey everyone, JFYI I created a PR #1838 with a proposal for the Producer instrumentation.