Skip to content

Enhance support for Argo Rollouts - #1380

Merged
nadaverell merged 9 commits into
skyhook-io:mainfrom
albertrdixon:feat/argo-rollouts-actions
Aug 19, 2026
Merged

Enhance support for Argo Rollouts#1380
nadaverell merged 9 commits into
skyhook-io:mainfrom
albertrdixon:feat/argo-rollouts-actions

Conversation

@albertrdixon

@albertrdixon albertrdixon commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Description

This adds enhanced support for Argo Rollouts in the UI. The following is a list of buttons added to Argo Rollouts details pages and their effect.

  • Restart only executes a rolling restart of Pods, not a full Rollout with new Analysis (spec.restartAt mutation)
  • Rollback button and modal. Shows a popup with versions that can be rolled back to. Optional 'Full promotion' check box to push the rollback through without a full Rollout.
  • Promote and Full Promote buttons to allow operators to manage Rollout stage promotion
  • Abort button to abort a Rollout

Type of change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update

How has this been tested?

Describe the tests you ran to verify your changes.

I included scripting and artifacts that I used for running a local demo/testing in kind. Also tested against out own remote clusters. Added unit tests were it made sense.

  • Tested locally with minikube/kind
  • Tested against a remote cluster
  • Added/updated unit tests

Checklist

  • My code follows the project's coding standards
  • I have performed a self-review of my code
  • I have added comments where necessary
  • My changes generate no new warnings
  • Any dependent changes have been merged

Related issues

Fixes #1379


Note

Medium Risk
Introduces cluster-mutating rollout controls (status subresource patches and rollback/restart semantics) behind capability probes and RBAC, with broad surface area across API, MCP, and UI but solid test coverage and explicit error mapping.

Overview
Adds first-class Argo Rollouts support end-to-end: operators and agents can drive canary/blue-green progression, inspect why a rollout is stuck, and roll back with Rollout-specific semantics.

API & engine: New POST /api/rollouts/{ns}/{name}/{abort,retry,promote,promote-full,skip-step} and GET .../capabilities, with actions gated separately on patch rollouts vs patch rollouts/status. Restart, scale, revisions, and rollback for rollouts go through the existing /api/workloads routes, delegating to pkg/rollouts. Rollback auth follows workloadRef when the Rollout does not own an inline template.

MCP: New write tool manage_rollout; manage_workload and get_resource (include=revisions) cover Rollout restart/scale/rollback with explicit notes that rollback restarts the full strategy (vs abort / promote-full).

UI: Rollout detail gains RBAC-aware Promote, Promote full, Skip step, Abort/Retry, analysis-run links, richer canary steps and problem banners. AnalysisRun list/detail views surface metric verdicts (including inconclusive vs failed). Shared revision history adds Stable / Rolling out badges and an optional promote fully after rollback path for hotfixes.

Observability & AI: Rollout-specific audit/history diffs (steps, weights, pauses, abort) so timelines are not empty while phase stays Progressing. AI summaries/issues name pause reasons, AnalysisRuns, and failing metrics (dry-run metrics excluded).

DX: make rollouts-demo and expanded docs/MCP catalog checks for new write tools.

Reviewed by Cursor Bugbot for commit 2f33930. Bugbot is set up for automated code reviews on this repo. Configure here.

Comment thread packages/k8s-ui/src/components/shared/ResourceActionsBar.tsx Outdated
Comment thread packages/k8s-ui/src/components/shared/ResourceActionsBar.tsx
Comment thread pkg/k8score/workload.go
Comment thread internal/server/server.go
statusPatch = []byte(fmt.Sprintf(clearPauseConditionsAndControllerPausePatch, next))
unifiedPatch = []byte(fmt.Sprintf(unpauseAndClearPauseConditionsPatchWithStep, next))
default:
statusPatch = []byte(clearPauseConditionsPatch)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this diverges from the Argo Rollouts CLI behavior: when the current step is a running analysis, the CLI advances to the next step. this would return success but leave the Rollout on the same step. Reproduced on the demo cluster.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great catch, this was definitely a bug!

Comment thread internal/server/rollouts_handlers.go Outdated
strategy := rollouts.StrategyOf(ro)
terminating := ro.GetDeletionTimestamp() != nil && !ro.GetDeletionTimestamp().IsZero()

// The status verbs fall back to a main-resource patch on Rollouts <= v0.9, so

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

argo rollouts 0.9 is from 2020 and EOL, no need to support so far back, just adds code complexity here. unless you have reason to expect people to still be running rollouts so old?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P.S. so I think just require patch rollouts/status for modern installs

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haha, no reason. This also eliminated a bunch of code.

@nadaverell

Copy link
Copy Markdown
Contributor

Promote full and Abort are one-click production traffic changes. Can we put those two behind ConfirmDialog? Normal Promote and Skip Step can stay direct. we have various examples of ConfirmDialog in the codebase for sensitive actions

@nadaverell nadaverell left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work - thanks for this PR!

A few small things before merging, see the comments

@nadaverell

Copy link
Copy Markdown
Contributor

Great work - thanks for this PR!

A few small things before merging, see the comments

P.S. also I recommend checking if the BugBot findings are valid or not, usually it's pretty decent at spotting potential issues, just always double-chck.

@albertrdixon
albertrdixon force-pushed the feat/argo-rollouts-actions branch from d2d8231 to aae1bb6 Compare August 10, 2026 22:27
Comment thread pkg/ai/context/summary_crd.go
Comment thread packages/k8s-ui/src/components/resources/ResourcesView.tsx
Comment thread pkg/rollouts/revisions.go
@albertrdixon
albertrdixon force-pushed the feat/argo-rollouts-actions branch from 90bcea9 to ab2c31f Compare August 10, 2026 23:39
@albertrdixon
albertrdixon force-pushed the feat/argo-rollouts-actions branch from ab2c31f to 2fc032b Compare August 13, 2026 16:31
Comment thread internal/server/server.go
@albertrdixon
albertrdixon force-pushed the feat/argo-rollouts-actions branch from 2fc032b to f789912 Compare August 13, 2026 16:55
Comment thread internal/server/rollouts_handlers.go
Comment thread packages/k8s-ui/src/components/resources/resource-utils.ts
@albertrdixon
albertrdixon force-pushed the feat/argo-rollouts-actions branch 2 times, most recently from b905466 to d00a1a3 Compare August 13, 2026 17:20
@albertrdixon

Copy link
Copy Markdown
Contributor Author

@nadaverell I think all the issues have been addressed. Cursorbot has been on me!

@albertrdixon
albertrdixon force-pushed the feat/argo-rollouts-actions branch from d00a1a3 to 7325261 Compare August 13, 2026 21:30
@albertrdixon
albertrdixon force-pushed the feat/argo-rollouts-actions branch from 7325261 to c1dccbd Compare August 13, 2026 22:06
Comment thread pkg/rollouts/operations.go
@albertrdixon
albertrdixon force-pushed the feat/argo-rollouts-actions branch 2 times, most recently from e951ecc to c832272 Compare August 14, 2026 00:29
Comment thread internal/mcp/tools_workloads.go
pkg/rollouts implements the five status verbs (abort, retry, promote,
promote-full, skip-step) plus revision history and rollback, over the dynamic
client with no internal/ imports.

Three mechanics worth knowing, all verified against a live cluster:

- The verbs patch the `rollouts/status` subresource and fall back to the main
  resource on NotFound, because Rollouts <= v0.9 have no status subresource.
  `patch rollouts` does not imply `patch rollouts/status`, so callers must
  authorize both.
- Rollback strips `rollouts-pod-template-hash` from the restored template; the
  controller derives that hash from template contents and a stale one wedges the
  rollout.
- Promoting past an inconclusive analysis needs an explicit step-index advance —
  clearing the pause alone leaves the controller on the same step.

Restart goes through `spec.restartAt`, which evicts pods older than the
timestamp: a rolling pod restart, not a new revision and not a re-run of the
canary steps or analysis.

Also caches AnalysisRun, AnalysisTemplate, ClusterAnalysisTemplate, and
Experiment so the analysis surfaces have data to read.
POST /api/rollouts/{ns}/{name}/{abort,retry,promote,promote-full,skip-step} plus
a capabilities probe the UI reads to decide which verbs to offer.

Rollback and revision history deliberately stay on the existing
/api/workloads/{kind}/... routes: same operation shape, shared revision UI. That
meant normalizing the rollbackable-kind gate so a singular kind can no longer 400
on a technicality.

The capabilities endpoint runs a SelfSubjectAccessReview for `rollouts` and
`rollouts/status` separately, since the subresource grant is independent, and
reports the strategy so the UI can hide step-relative verbs on blueGreen.
Registers a Rollout diff function so mid-canary transitions become timeline
events: phase, abort, promoteFull, step index, pod hash, stable RS, blueGreen
selectors, canary/stable weights, pause conditions, replica counts. Pause reasons
are sorted before comparison so the controller reordering them is not a change.

Registering a kind means empty diffs get dropped, so the coverage has to be
complete or updates vanish silently — hence a subtest per transition.

Topology gains the active AnalysisRuns as nodes hanging off their Rollout,
labelled by trigger (step, background, pre-promotion, post-promotion). Only
current runs: every historical AnalysisRun would grow the graph without bound.
They are excluded from the generic owner-ref pass so there is exactly one
producer, and the edge is EdgeManages — EdgeUses would file the Rollout under
"Autoscaler" on its own AnalysisRun's detail page.
The Rollout verbs join manage_workload rather than becoming new tools, keeping
them inside the description budget and the existing write-tool annotations. Also
updates the setup-dialog catalog, which CI pins against the registry.

AI context minification learns why a Rollout is blocked — abort takes precedence,
then pause reasons, then named analysis verdicts — and summarizes AnalysisRuns by
naming only the failing or inconclusive metrics, so an agent gets the deciding
metric instead of "InconclusiveAnalysisRun".
The Rollout detail page gains an action row and an Analysis section; AnalysisRun
gets its own renderer. Both are built around one question: why is this rollout
stuck, and what do I do about it.

Gating distinguishes the two reasons a verb is unavailable. A verb the capability
probe denied is absent entirely — a greyed button reads as "you could do this" and
hides whether RBAC or state is the cause. A verb blocked by state is present and
carries the reason.

The revision table separates `Current` from `Stable`, which diverge mid-canary:
revision 5 can be rolling out while 3 still serves traffic. Rolling back re-enters
the strategy — canary replays every step, blueGreen parks the revision in preview
with the active Service untouched — so the dialog offers promoting straight
through. That option is kind-gated, not strategy-gated, and hidden when the probe
denies promote-full.

AnalysisRun detail splits the three verdicts an operator must act on differently:
Error could not run, Failed will abort the rollout, Inconclusive is waiting on a
human. Each metric is paired with its condition from the spec, so a stuck rollout
shows `latest: 2` against `success if: result == 1`.

Problem banners use AlertBanner rather than hand-rolled colors, which wash out in
light mode, and dedupe on message — an aborted Rollout is also Degraded and both
render status.message.
`make rollouts-demo` bootstraps a kind cluster parked in the five states the
control surface has to handle: a manual canary pause, an inconclusive analysis, a
blueGreen pause, a canary aborted by a failing analysis, and a workloadRef canary
whose template lives on a Deployment.

Generating a deterministic Inconclusive without Prometheus needed a lever: with
both successCondition and failureCondition set, a result matching neither
evaluates Inconclusive. So the metric provider is an nginx serving static JSON —
`2` against `result == 1` / `result == 0`.

The parking checks run before waiting for Healthy so re-running against an
already-parked cluster is idempotent rather than timing out.
@albertrdixon
albertrdixon force-pushed the feat/argo-rollouts-actions branch from c832272 to a9b88e5 Compare August 14, 2026 01:00
Comment thread internal/server/rollouts_handlers.go Outdated
auth.AuditLog(r, namespace, name)
client := s.getDynamicClientForRequest(r)
if client == nil {
log.Printf("[rollouts] Dynamic client unavailable for %s Rollout %s/%s", action, sanitizeForLog(namespace), sanitizeForLog(name))
Comment thread internal/server/rollouts_handlers.go Outdated
default:
status = http.StatusInternalServerError
}
log.Printf("[rollouts] %s %s/%s -> %d: %v", action, sanitizeForLog(namespace), sanitizeForLog(name), status, err)
Comment thread internal/server/rollouts_handlers.go Outdated
default:
status = http.StatusInternalServerError
}
log.Printf("[rollouts] %s %s/%s -> %d: %v", action, sanitizeForLog(namespace), sanitizeForLog(name), status, err)
albertrdixon and others added 2 commits August 17, 2026 10:55
Conflict in web/src/components/workload/WorkloadView.tsx was additive on
both sides — this branch registers RolloutRenderer while main added the
Kyverno coverage/queued and CNPG renderers to the same import list and
rendererOverrides map. Kept both.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The action segment reached log.Printf raw while namespace and name were
already passed through sanitizeForLog, so a crafted request could inject
newlines and forge log records. %q escapes them and is the sanitizer
CodeQL's go/log-injection query recognizes, matching how the rest of
internal/server logs untrusted strings.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

There are 15 total unresolved issues (including 13 from previous reviews).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 645705d. Configure here.

Comment thread pkg/rollouts/operations.go
Comment thread pkg/ai/context/summary_crd.go

@nadaverell nadaverell left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed the full iteration - looks great!
Merging

@nadaverell
nadaverell merged commit 273d21c into skyhook-io:main Aug 19, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enhance support for Argo Rollouts

3 participants