Gap in Kafka metrics instrumentation

Description

~The instrumentation module com.newrelic.instrumentation.kafka-clients-metrics-2.0.0 instruments KafkaConsumer#poll(Duration). This method is overloaded (KafkaConsumer#poll(long)) and both call a private method.~

~This means that if a customer is to call the overloaded method, no metrics will be recorded. So we should instrument the private method instead.~

~Also, in newer versions of Kafka (3?) the private method has changed its signature, so we will probably have to create a new instrumentation module.~

While investigating the GTSE it was believed that the instrumentation module kafka-clients-metrics-2.0.0 would apply to Kafka 3, as it is indicated in the build.gradle file. But actually this module is an increment over the kafka-clients-metrics-0.10.0.0 and only instruments the new methods added in Kafka 2.

We confirmed that the customer from the GTSE is using Kafka 3 and so a new instrumentation module is needed.

Reference

https://newrelic.atlassian.net/browse/GTSE-13566

Aha! Link: https://newrelic.aha.io/features/JAVA-385

[averkhovtsev-xm]

The title of the ticket is a bit misleading. It is not a gap, none of the producer/consumer metrics are being sent to Newrelic. the kafka-clients-metrics-0.10.0.0 module is not applied to 3+ version of kafka clients, the kafka-clients-metrics-2.0.0 module instruments only 1 method in KafkaConsumer which is not metrics related. The reason I’m bringing this up is to increase the priority (and hopefully ETA) of this ticket.

[meiao]

This was delivered in the 7.8.0 agent.