Skip to content

Provision SQS interruption queue for Karpenter - #18607

Open
shreyaabaranwal wants to merge 1 commit into
kubernetes:masterfrom
shreyaabaranwal:fix-karpenter-interruption-queue
Open

Provision SQS interruption queue for Karpenter#18607
shreyaabaranwal wants to merge 1 commit into
kubernetes:masterfrom
shreyaabaranwal:fix-karpenter-interruption-queue

Conversation

@shreyaabaranwal

Copy link
Copy Markdown

Summary

When Karpenter is enabled, kOps doesn't provision an SQS queue for it, so Karpenter never receives interruption events (spot interruptions, instance state changes, scheduled maintenance, rebalance recommendations). nodeTerminationHandler can't be enabled alongside Karpenter either, since validation blocks that combination, so clusters running Karpenter currently get no interruption handling at all.

  • Adds a KarpenterBuilder that provisions an SQS queue plus the EventBridge rules/targets for the four interruption event types, mirroring what's already done for NodeTerminationHandler's queue.
  • Passes the queue name into the addon via a new KarpenterQueueName template function, used in both the Helm values (settings.interruptionQueue) and the static manifest (INTERRUPTION_QUEUE env var).
  • Adds the SQS permissions Karpenter needs to read from the queue (DeleteMessage, GetQueueUrl, GetQueueAttributes, ReceiveMessage) to AddKarpenterPermissions, added unconditionally since SQS actions don't support resource-level conditions.

Fixes #18016

Test plan

  • go build ./...
  • gofmt -l . (clean aside from pre-existing vendor files)
  • go vet ./pkg/model/... ./cmd/kops/... ./upup/pkg/fi/cloudup/...
  • go test ./pkg/model/iam/... ./pkg/model/awsmodel/... ./pkg/model/components/addonmanifests/karpenter/... ./upup/pkg/fi/cloudup/...
  • go test ./cmd/kops/... (updated TestKarpenter golden fixtures to include the new SQS queue, EventBridge rules/targets, and IAM policy changes)

@kubernetes-prow kubernetes-prow Bot added the do-not-merge/invalid-commit-message Indicates that a PR should not merge because it has an invalid commit message. label Jul 18, 2026
@kubernetes-prow
kubernetes-prow Bot requested a review from hakman July 18, 2026 12:27
@kubernetes-prow
kubernetes-prow Bot requested a review from olemarkus July 18, 2026 12:27
@kubernetes-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign olemarkus for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubernetes-prow kubernetes-prow Bot added area/provider/aws Issues or PRs related to aws provider cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Jul 18, 2026
@kubernetes-prow

Copy link
Copy Markdown
Contributor

Hi @shreyaabaranwal. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@kubernetes-prow kubernetes-prow Bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jul 18, 2026
When Karpenter is enabled, kOps doesn't set up an SQS queue for it,
so it never receives interruption events (spot interruptions,
instance state changes, scheduled maintenance, rebalance
recommendations). nodeTerminationHandler can't be enabled alongside
Karpenter either, since validation blocks that combination. Net
result: clusters running Karpenter get no interruption handling at
all right now.

This adds a KarpenterBuilder that sets up an SQS queue plus the
EventBridge rules/targets for the four interruption event types,
mirroring what we already do for NodeTerminationHandler's queue. The
queue name is passed into the addon through a new
KarpenterQueueName template function, used in both the Helm values
(settings.interruptionQueue) and the static manifest
(INTERRUPTION_QUEUE env var).

Also adds the SQS permissions Karpenter needs to actually read from
that queue (DeleteMessage, GetQueueUrl, GetQueueAttributes,
ReceiveMessage). These are added unconditionally since SQS actions
don't support resource-level conditions -- same reasoning already
used for AddNodeTerminationHandlerSQSPermissions.
@shreyaabaranwal
shreyaabaranwal force-pushed the fix-karpenter-interruption-queue branch from 87102e7 to ddaa2fe Compare July 18, 2026 12:31
@kubernetes-prow kubernetes-prow Bot removed the do-not-merge/invalid-commit-message Indicates that a PR should not merge because it has an invalid commit message. label Jul 18, 2026
@shreyaabaranwal

shreyaabaranwal commented Jul 18, 2026

Copy link
Copy Markdown
Author

"Hi @hakman @olemarkus, just checking in on this , happy to make any changes needed.

@shreyaabaranwal

Copy link
Copy Markdown
Author

Hi @hakman @olemarkus, following up again - happy to split this into smaller PRs if that'd make review easier, or address any specific concerns. Let me know what would help move this forward.

@kubernetes-prow kubernetes-prow Bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 30, 2026
@kubernetes-prow

Copy link
Copy Markdown
Contributor

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/addons area/provider/aws Issues or PRs related to aws provider cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Karpenter add-on missing INTERRUPTION_QUEUE: NodeTerminationHandler must be disabled, leaving the cluster without AWS interruption events

1 participant