Skip to content

Fix a bug in Service Connect error handling#4588

Merged
amogh09 merged 1 commit into
aws:devfrom
amogh09:sc-error
Apr 23, 2025
Merged

Fix a bug in Service Connect error handling#4588
amogh09 merged 1 commit into
aws:devfrom
amogh09:sc-error

Conversation

@amogh09
Copy link
Copy Markdown
Contributor

@amogh09 amogh09 commented Apr 23, 2025

Summary

Service connect tasks go through an "augmenting" phase handled by serviceconnect.Manager's AugmentTaskContainer method. This method makes some changes to tasks such as populating APPNET_CONTAINER_IP_MAPPING environment variable in service connect sidecar container. However, the method implementation currently swallows any errors that happen during the augmentation process. This PR fixes this bug by surfacing any such errors as CannotCreateContainer errors.

Testing

Ran a Service Connect bridge mode service task on an instance faked to have IPv6-only compatibility and without IPv6 enabled on docker bridge network. This causes task augmentation to fail because an IPv6 address is expected for each task container but it is not found. Verified the task stop error in DescribeTasks response that is reproduced below.

$ aws ecs describe-tasks --cluster tutorial --task a2eafcdda65348178cc6e0da96cb2cdc | jq '.tasks[0].containers'
[
  {
    "containerArn": "arn:aws:ecs:us-west-2:979604884904:container/tutorial/a2eafcdda65348178cc6e0da96cb2cdc/27ff7458-2088-4ae0-8cbe-a8e38ab01a45",
    "taskArn": "arn:aws:ecs:us-west-2:979604884904:task/tutorial/a2eafcdda65348178cc6e0da96cb2cdc",
    "name": "webserver",
    "image": "public.ecr.aws/docker/library/nginx:latest",
    "imageDigest": "sha256:5ed8fcc66f4ed123c1b2560ed708dc148755b6e4cbd8b943fab094f2c6bfa91e",
    "lastStatus": "STOPPED",
    "networkInterfaces": [],
    "healthStatus": "UNKNOWN",
    "managedAgents": [
      {
        "name": "ExecuteCommandAgent",
        "reason": "Received Container Stopped event",
        "lastStatus": "STOPPED"
      }
    ],
    "cpu": "100"
  },
  {
    "containerArn": "arn:aws:ecs:us-west-2:979604884904:container/tutorial/a2eafcdda65348178cc6e0da96cb2cdc/fb80fa71-e919-48bd-8070-33561163435f",
    "taskArn": "arn:aws:ecs:us-west-2:979604884904:task/tutorial/a2eafcdda65348178cc6e0da96cb2cdc",
    "name": "ecs-service-connect-dVSKI",
    "lastStatus": "STOPPED",
    "reason": "CannotCreateContainerError: CannotCreateContainerError: instance is IPv6-only but no IPv6 address found for container 'webserver'",
    "networkInterfaces": [],
    "healthStatus": "UNKNOWN",
    "managedAgents": [
      {
        "name": "ExecuteCommandAgent",
        "reason": "Received Container Stopped event",
        "lastStatus": "STOPPED"
      }
    ]
  }
]

New tests cover the changes: yes

Description for the changelog

Bugfix: Surface task augmentation errors for Service Connect tasks

Additional Information

Does this PR include breaking model changes? If so, Have you added transformation functions?

No

Does this PR include the addition of new environment variables in the README?

No

Licensing

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@amogh09 amogh09 marked this pull request as ready for review April 23, 2025 01:11
@amogh09 amogh09 requested a review from a team as a code owner April 23, 2025 01:11
@amogh09 amogh09 enabled auto-merge (squash) April 23, 2025 16:24
@amogh09 amogh09 merged commit 0f959b3 into aws:dev Apr 23, 2025
40 of 41 checks passed
timj-hh pushed a commit to timj-hh/amazon-ecs-agent that referenced this pull request Jul 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants