Update Kafka producer instrumentation

[sharvath-newrelic]

Overview

Updated Kafka producer instrumentation to use modern distributed tracing APIs with W3C Trace Context support, and added an additional test to validate the exclude_newrelic_header configuration option.

Related Github Issue

#696

Kafka instrumentation uses deprecated DT APIs that only adds the newrelic header. This requires customers to take manual steps for full distributed tracing functionality (see https://github.com/newrelic/newrelic-java-examples/tree/main/newrelic-java-agent/distributed-tracing/kafka-examples).

Testing

Added W3C header verification to existing test: produceConsumeTest()

Added new test: produceConsumeTestExcludeNewRelicHeader():

• Tests the exclude_newrelic_header: true configuration
• Verifies W3C headers are present when proprietary header is excluded
• Validates distributed tracing works correctly with only W3C headers
• Uses KafkaHeadersAdapter (test-scoped adapter) to manually accept W3C headers

Added new test configuration to validate exclusion of the newrelic header in span_events_test.

Ran through a few e2e tests to validate the following scenarios:

• Packaged the 8.25.0-SNAPSHOT jar and replaced the jars in the kafka-examples application. Commented out the addDistributedTraceHeadersToKafkaRecord function fired off by produce() with exclude_new_relic_header set to false. Verified consumption of the newrelic, traceparent and tracestate record headers.
• Ran the aforementioned scenario with exclude_new_relic_header set to true. Verified consumption of the traceparent and tracestate record headers.
• Ran the 'old' (6.4.0) scenario with addDistributedTraceHeadersToKafkaRecord commented out with exclude_new_relic_header set to false and verified consumption of only the newrelic record header.

Checks

• Your contributions are backwards compatible with relevant frameworks and APIs.
• Your code does not contain any breaking changes. Otherwise please describe.
• Your code does not introduce any new dependencies. Otherwise please describe.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 70.34%. Comparing base (56fe81b) to head (462a857).
⚠️ Report is 19 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff            @@
##               main    #2516   +/-   ##
=========================================
  Coverage     70.34%   70.34%           
- Complexity    10058    10062    +4     
=========================================
  Files           847      847           
  Lines         40896    40913   +17     
  Branches       6198     6201    +3     
=========================================
+ Hits          28769    28781   +12     
- Misses         9326     9332    +6     
+ Partials       2801     2800    -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.