Add support for lambda traces via threadLocal #6295
Conversation
RanVaknin
added
the
changelog-not-required
There was a problem hiding this comment.
Will async clients be addressed in a separate PR? This won't work for async retries (which are delegated to a retries thread pool), calls that happen within future chaining (which are delegated to a future completion executor thread pool), etc.
We'll want to find where calls might switch threads (i.e. anywhere we're using futures) and then preserve the thread local setting across that thread change.
| @@ -32,6 +33,7 @@ | |||
| public class TraceIdExecutionInterceptor implements ExecutionInterceptor { | |||
| private static final String TRACE_ID_HEADER = "X-Amzn-Trace-Id"; | |||
| private static final String LAMBDA_FUNCTION_NAME_ENVIRONMENT_VARIABLE = "AWS_LAMBDA_FUNCTION_NAME"; | |||
| private static final String CONCURRENT_TRACE_ID_KEY = "AWS_LAMBDA_X_TraceId"; | |||
There was a problem hiding this comment.
This capitalization is weird. Was this specific format chosen for a reason?
There was a problem hiding this comment.
I don't have context about why it was chosen, but according to the Lambda team ,this is the format they are going to use.
There was a problem hiding this comment.
Happy to follow AWS SDK standards - We can tweak our string accordingly
There was a problem hiding this comment.
Did we agree on a string? ex. AWS_LAMBDA_X_TRACEID?
There was a problem hiding this comment.
AWS_LAMBDA_X_TRACEID or AWS_LAMBDA_X_TRACE_ID sounds good to me!
There was a problem hiding this comment.
Let's go with AWS_LAMBDA_X_TRACE_ID🤝🏻
.../src/main/java/software/amazon/awssdk/awscore/internal/interceptor/TracingSystemSetting.java
Outdated
Show resolved
Hide resolved
RanVaknin
changed the title
|
| @@ -32,27 +34,40 @@ | |||
| public class TraceIdExecutionInterceptor implements ExecutionInterceptor { | |||
| private static final String TRACE_ID_HEADER = "X-Amzn-Trace-Id"; | |||
| private static final String LAMBDA_FUNCTION_NAME_ENVIRONMENT_VARIABLE = "AWS_LAMBDA_FUNCTION_NAME"; | |||
| private static final String CONCURRENT_TRACE_ID_KEY = "AWS_LAMBDA_X_TraceId"; | |||
| private static final ExecutionAttribute<String> CACHED_TRACE_ID = new ExecutionAttribute<>("CachedTraceId"); | |||
There was a problem hiding this comment.
Probably just TRACE_ID? This is cached in the sense that anything stored in memory is cached. Probably not the right term here.
| @@ -133,5 +206,6 @@ private SdkHttpRequest modifyHttpRequest(Context.ModifyHttpRequest context) { | |||
| private void resetRelevantEnvVars(EnvironmentVariableHelper env) { | |||
| env.remove("AWS_LAMBDA_FUNCTION_NAME"); | |||
| env.remove("_X_AMZN_TRACE_ID"); | |||
| env.remove("AWS_LAMBDA_MAX_CONCURRENCY"); | |||
| .responseBody(AbortableInputStream.create(new StringInputStream("{}"))) | ||
| .build()); | ||
|
|
||
| client.allTypes().join(); |
There was a problem hiding this comment.
We should also add a test sending a request in the result future:
client.allTypes()
.thenRun(() -> client.allTypes() /* Did this request have the right trace ID? */)I suspect we'll need to plumb the thread local context to there as well (and clear it, so that reuses of that thread don't have it set).
Motivation and Context
This implementation uses SLF4J MDC (which uses ThreadLocal under the hood) to read the trace ID value stored in
AWS_LAMBDA_X_TraceIdthat would be set by the Lambda runtime (this would be added later, probably here).