Sigv4 Authentication in OtlpHttp{Signal}Exporter

[arun-annamalai]

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is.
Context: AWS X-ray has released an Otel Native endpoint: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-OTLPEndpoint.html. However, this endpoint requires that the http requests be signed using sigv4: https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_sigv.html.

Situation: As of right now, trace senders are limited to using the otel collector (with Sigv4 extension) to send traces from their application to Xray otlp endpoint.

Is there a way to build a sigv4 extension for the java language SDKs so that applications can directly send to cloud otlp endpoint from the SDK? This essentially cuts an expensive middle man for faas environments like lambda. I was trying to look into this, but it doesnt look like the Java Otel SDK allows for http clients to be injected.

Describe the solution you'd like
A clear and concise description of what you want to happen.
I just need a mechanism to hook into the HTTP call workflow. Ideally in native HTTP clients these are interceptors (like the OkHTTP application interceptor here: https://square.github.io/okhttp/features/interceptors/#choosing-between-application-and-network-interceptors).

The interceptor would contain context about the HTTP request (body, params, headers, etc) and allow mutation of the HTTP request. I would then use the AWS SDK inside a custom interceptor to sign the HTTP request. I could also add this custom interceptor to the java-contrib repo and allow all otel/aws users to access this.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
I have considered the setHeaders() API and that does not solve the usecase since it does not have access to the HTTP request.

Alternatively, if we are allowed to inject in HTTP clients or specify that we would like to use an OkHTTP client (with our interceptor attached in) then we might be able to perform the signing.

Ideally, a separate OTLP exporter just for exporter to Amazon, would require alot of redundant code to maintain. I would assume other data storage vendors would also want some way to hook into the authentication process as well.

Additional context
Add any other context or screenshots about the feature request here.
Discussion with Jack over slack: https://cloud-native.slack.com/archives/C014L2KCTE3/p1734483601460489

[jack-berg]

We discussed this topic at length in the 1/16/24 java SIG. Please check out the last 15 minutes of the meeting recording for the full discussion.

The take away for me was that this is challenging to support in our OTLP exporters for the reasons discussed in the slack thread. An additional point that was brought up was that Sigv4 seems to require having a reference to the payload bytes in memory, while our OTLP exporters write payload bytes incrementally to the request's OutputStream. Its not clear how Sigv4 would work given this design.

I think AWS should seriously consider adding AWS specific implementations of OTLP exports to the AWS java SDK. If you did this, you could do this without taking on the complexity of proto serialization by leveraging our OTLP serialization logic. Its currently internal, but we have a growing number of use cases for making our OTLP serialization logic public, and so making it public with strong backwards compatibility guarantees is a realistic option.

Also, if its important to have sigv4 implementations across a variety of otel language implementations, and I presume that it is because the sigv4 requirement can't be specific to otel java, then I would highly recommend bringing this topic up at the specification sig, which meets on tuesdays at 8:00 PT. Whatever challenges you face in opentelemetry-java, you're likely to face in other languages as well. Would be good to hear what the maintainers of other languages think about this.

Given the complexities and the fact that this is a vendor-specific request, if you do want to propose extending our OTLP exporters with some sort of extension point to support this, I'd ask that you do the legwork to sketch out a prototype of how that might work. If you choose to do this, keep in mind the sender abstraction - specifically, we can't expose any types from any underlying client implementation in the more general OTLP configuration surface area. For example, we can't have an configuration option in OtlpHttpSpanExporterBuilder reference okhttp3.Headers (i.e. the headers representation from OkHttp).