error on CloudWatchClient

[vinaynathbandi]

Describe the bug

Got error while using CloudWatchClient to list or get metrics

CloudWatchClient cw = CloudWatchClient.builder() .region(region) .build();

Error:
Exception in thread "main" java.lang.NullPointerException: Credentials must not be null.
at software.amazon.awssdk.utils.Validate.paramNotNull(Validate.java:156)
at software.amazon.awssdk.auth.signer.params.Aws4SignerParams.(Aws4SignerParams.java:42)
at software.amazon.awssdk.auth.signer.params.Aws4SignerParams$BuilderImpl.build(Aws4SignerParams.java:206)
at software.amazon.awssdk.auth.signer.internal.BaseAws4Signer.sign(BaseAws4Signer.java:34)
at software.amazon.awssdk.core.internal.http.pipeline.stages.SigningStage.lambda$signRequest$0(SigningStage.java:71)
at software.amazon.awssdk.core.internal.util.MetricUtils.measureDuration(MetricUtils.java:51)
at software.amazon.awssdk.core.internal.http.pipeline.stages.SigningStage.signRequest(SigningStage.java:70)
at software.amazon.awssdk.core.internal.http.pipeline.stages.SigningStage.execute(SigningStage.java:55)
at software.amazon.awssdk.core.internal.http.pipeline.stages.SigningStage.execute(SigningStage.java:40)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:73)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42)
at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:78)
at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:40)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:50)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:36)
at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:81)
at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56)
at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:48)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:31)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
at software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:193)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:103)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:167)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:82)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:175)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:76)
at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:55)
at software.amazon.awssdk.services.cloudwatch.DefaultCloudWatchClient.listMetrics(DefaultCloudWatchClient.java:1883)
at com.msk.metrics.ListMetricData.listMets(ListMetricData.java:95)
at com.msk.metrics.ListMetricData.main(ListMetricData.java:75)

Expected Behavior

Need to take default credentials from Environment variables

Current Behavior

Getting

Exception in thread "main" java.lang.NullPointerException: Credentials must not be null.
at software.amazon.awssdk.utils.Validate.paramNotNull(Validate.java:156)
at software.amazon.awssdk.auth.signer.params.Aws4SignerParams.(Aws4SignerParams.java:42)
at software.amazon.awssdk.auth.signer.params.Aws4SignerParams$BuilderImpl.build(Aws4SignerParams.java:206)
at software.amazon.awssdk.auth.signer.internal.BaseAws4Signer.sign(BaseAws4Signer.java:34)
at software.amazon.awssdk.core.internal.http.pipeline.stages.SigningStage.lambda$signRequest$0(SigningStage.java:71)
at software.amazon.awssdk.core.internal.util.MetricUtils.measureDuration(MetricUtils.java:51)
at software.amazon.awssdk.core.internal.http.pipeline.stages.SigningStage.signRequest(SigningStage.java:70)
at software.amazon.awssdk.core.internal.http.pipeline.stages.SigningStage.execute(SigningStage.java:55)
at software.amazon.awssdk.core.internal.http.pipeline.stages.SigningStage.execute(SigningStage.java:40)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:73)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42)
at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:78)
at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:40)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:50)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:36)
at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:81)
at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56)
at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:48)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:31)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
at software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:193)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:103)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:167)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:82)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:175)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:76)
at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:55)
at software.amazon.awssdk.services.cloudwatch.DefaultCloudWatchClient.listMetrics(DefaultCloudWatchClient.java:1883)
at com.msk.metrics.ListMetricData.listMets(ListMetricData.java:95)
at com.msk.metrics.ListMetricData.main(ListMetricData.java:75)

Reproduction Steps

package com.msk.metrics;

import java.util.Properties;

import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.AwsCredentials;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProviderChain;
import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
import software.amazon.awssdk.auth.credentials.EnvironmentVariableCredentialsProvider;
import software.amazon.awssdk.auth.credentials.ProfileCredentialsProvider;
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
import software.amazon.awssdk.auth.credentials.SystemPropertyCredentialsProvider;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.regions.providers.DefaultAwsRegionProviderChain;
import software.amazon.awssdk.services.cloudwatch.CloudWatchClient;
import software.amazon.awssdk.services.cloudwatch.model.CloudWatchException;
import software.amazon.awssdk.services.cloudwatch.model.ListMetricsRequest;
import software.amazon.awssdk.services.cloudwatch.model.ListMetricsResponse;
import software.amazon.awssdk.services.cloudwatch.model.Metric;

public class ListMetricData {

public static void main(String[] args) {
	
	 
        String namespace = "AWS/Kafka";
        Region region = Region.US_EAST_1;
        System.out.println("region: "+ region);
        
       
        
        
        CloudWatchClient cw = CloudWatchClient.builder()
        		.region(region)
        		.build();
       

        listMets(cw, namespace) ;
        cw.close();
    }

    // snippet-start:[cloudwatch.java2.list_metrics.main]
    public static void listMets( CloudWatchClient cw, String namespace) {

        boolean done = false;
        String nextToken = null;

        try {
            while(!done) {

                ListMetricsResponse response;

                if (nextToken == null) {
                   ListMetricsRequest request = ListMetricsRequest.builder()
                        .namespace(namespace)
                        .build();

                 response = cw.listMetrics(request);
                } else {
                  ListMetricsRequest request = ListMetricsRequest.builder()
                        .namespace(namespace)
                        .nextToken(nextToken)
                        .build();

                response = cw.listMetrics(request);
            }

            for (Metric metric : response.metrics()) {
                System.out.printf(
                        "Retrieved metric %s", metric.metricName());
                System.out.println();
            }

            if(response.nextToken() == null) {
                done = true;
            } else {
                nextToken = response.nextToken();
            }
        }

        } catch (CloudWatchException e) {
            System.err.println(e.awsErrorDetails().errorMessage());
            System.exit(1);
        } 
    }

}

Possible Solution

No response

Additional Information/Context

No response

AWS Java SDK version used

AWS SDK for Java API Reference - 2.17.169

JDK version used

java version "1.8.0_131" Java(TM) SE Runtime Environment (build 1.8.0_131-b11) Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

Operating System and version

Windows 10

[vinaynathbandi]

ABle to resolve credentials nullpointer exception. But i got another error "Request is missing Authentication Token" when using

CloudWatchClient cw = CloudWatchClient.builder() .region(region) .credentialsProvider(StaticCredentialsProvider.create(awsCreds)) .overrideConfiguration(ClientOverrideConfiguration.builder().advancedOptions(signerOption).build()) .build();

[debora-ito]

@vinaynathbandi How are you defining signerOption?

[vinaynathbandi]

@debora-ito Thank you for response.

Below is the 'signerOption' used

Map<SdkAdvancedClientOption<?>, Signer> signerOption = new HashMap<>(); signerOption.put(SdkAdvancedClientOption.SIGNER, new NoOpSigner());

[debora-ito]

So you're overriding the default request signer with NoOpSigner, that's probably the reason you're seeing the error.

You don't need to define a custom signerOption for regular api calls:

CloudWatchClient cw = CloudWatchClient.builder() 
                                      .region(region) 
                                      .credentialsProvider(StaticCredentialsProvider.create(awsCreds)) 
                                      .build();

[vinaynathbandi]

Used above code but got error

Exception in thread "main" java.lang.NullPointerException: Credentials must not be null. at software.amazon.awssdk.utils.Validate.paramNotNull(Validate.java:156) at software.amazon.awssdk.auth.signer.params.Aws4SignerParams.(Aws4SignerParams.java:42) at software.amazon.awssdk.auth.signer.params.Aws4SignerParams$BuilderImpl.build(Aws4SignerParams.java:206) at software.amazon.awssdk.auth.signer.internal.BaseAws4Signer.sign(BaseAws4Signer.java:34) at software.amazon.awssdk.core.internal.http.pipeline.stages.SigningStage.lambda$signRequest$0(SigningStage.java:71) at software.amazon.awssdk.core.internal.util.MetricUtils.measureDuration(MetricUtils.java:51) at software.amazon.awssdk.core.internal.http.pipeline.stages.SigningStage.signRequest(SigningStage.java:70) at software.amazon.awssdk.core.internal.http.pipeline.stages.SigningStage.execute(SigningStage.java:55) at software.amazon.awssdk.core.internal.http.pipeline.stages.SigningStage.execute(SigningStage.java:40) at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)

[vinaynathbandi]

@debora-ito Any update on this issue?

[debora-ito]

Have you set the aws credentials in any way? There are several ways you can set them, you can read more about it in https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/setup-credentials.html.

The error indicates that the credential the SDK picked up is null.

[debora-ito]

In the sample code, the SDK will pick the static credentials defined in awsCreds, can you show how they are being created? Can they be null?

Please don't show your actual credentials, redact them when giving the sample code.

CloudWatchClient cw = CloudWatchClient.builder() 
                                      .region(region) 
                                      .credentialsProvider(StaticCredentialsProvider.create(awsCreds)) 
                                      .build();

[vinaynathbandi]

`AwsBasicCredentials awsCreds = AwsBasicCredentials.create(
"XXXXXXXXXXXXXXXXXXX",
"XXXXXXXXXXXXXXXXXXXXXXXXXXXXX");

CloudWatchClient cw = CloudWatchClient.builder()
.region(region)
.credentialsProvider(StaticCredentialsProvider.create(awsCreds))
//.overrideConfiguration(ClientOverrideConfiguration.builder().advancedOptions(signerOption).build())
.build();`

[debora-ito]

That is weird, using AwsBasicCredentials should be sufficient if you pass them to the client builder as the credentialsProvider.

Can you tell us more about the application environment? Are you running in EC2 or ECS/EKS?
You said you've set environment variables and .aws/configuration, is there any chance that anywhere in the code a different CloudWatchClient is using the default credential provider chain and not the AwsBasicCredentials?

[vinaynathbandi]

I'm using for AWS/Kafka metrics.

String namespace = "AWS/Kafka";
Region region = Region.US_EAST_1;

[debora-ito]

I mean where you're running the application? It's on an EC2 or virtual environment? Or on a local linux machine, for example?

[debora-ito]

Just a heads up, I'm moving this to Github Discussions, that's where we are keeping guidance questions and answers.