🚨 Breaking Change - Dataplane - Sane defaults

[EngHabu]

[:rotating_light: Breaking Change]

Major overhaul of the dataplane chart to establish sane, low-privilege defaults that work out of the box for new deployments while preserving backward compatibility via values-legacy.yaml.

Default mode changes

• Low-privilege mode (low_privilege: true) is now the default — namespace-scoped RBAC, no cluster-wide permissions except where strictly required (fluentbit DaemonSet, knative-operator)
• V2 executor is the default — flytepropeller.enabled: false, executor.idl2Executor: true, flyteconnector.enabled: true
• Cluster resource sync disabled by default (clusterresourcesync.enabled: false)
• Common service account (union-system) shared across all components by default
• values-legacy.yaml created to restore all previous defaults with a single overlay file

Prometheus consolidation

• Replaced the static prometheus deployment (templates/prometheus/) and the prometheus-simple subchart with a single community prometheus chart aliased as prometheus
• Removed standalone kube-state-metrics dependency (now a subchart of prometheus)
• Namespace-scoped RBAC for prometheus and kube-state-metrics in low-privilege mode via prometheus-rbac.yaml
• Scrape configs for flytepropeller, serving-envoy, and dcgm-exporter are now unconditional (no-op when targets don't exist)
• cAdvisor scraping moved to values-legacy.yaml (requires ClusterRole)

Knative operator improvements

• Split CRDs into knative-operator-crds subchart — solves chicken-and-egg CRD validation error on fresh installs. CRDs are in the crds/ directory so Helm installs them before templates
• CRDs chart conditioned on knative-operator-crds.enabled
• Removed single_namespace support — the operator requires cluster-scoped RBAC and can't be namespace-restricted
• Fixed _example key in operator ConfigMaps that caused webhook validation failures on helm upgrade
• Added tpl support in knative-operator.namespace helper for namespaceOverride
• Fixed CRD conversion webhook namespace to use the helper instead of .Release.Namespace

Image builder auto-configuration

• imageBuilder.defaultRepository auto-generates from cloud provider:
  • AWS: <account_id>.dkr.ecr.<region>.amazonaws.com/<registryName>
  • GCP: <region>-docker.pkg.dev/<projectId>/<registryName>
  • Azure: <registryName>.azurecr.io
• imageBuilder.authenticationType auto-detects (aws, google, azure, noop)
• New imageBuilder.registryName value (default: union-dataplane)
• Added global.AWS_ACCOUNT_ID for ECR URL generation

Other changes

• Fixed fluentbit.serviceAccount.name to use common SA (union-system) by default
• Fixed union-serviceaccount.yaml missing tpl calls (pre-existing bug)
• Webhook templates moved from nodeexecutor/ to webhook/ directory with Helm-managed certificates
• Added GCP test case (dataplane.gcp.yaml)
• Removed values-low-privilege.yaml and values.v2.yaml (superseded by new defaults)

Webhook certificate management

• Helm-managed MutatingWebhookConfiguration — the webhook configuration is now created by Helm (with flytepropellerwebhook.managedConfig: true) instead of self-registered by the webhook binary at runtime. This removes the need for the webhook to have cluster-scoped RBAC for mutatingwebhookconfigurations
• Configurable certificate providers (flytepropellerwebhook.certificate.provider):
  • helm (default) — generates self-signed certs using Helm's crypto functions, preserved on upgrade if the secret exists
  • certManager — uses cert-manager to provision and manage certificates
  • external — user-provided certificates via caCert, tlsCrt, tlsKey values
  • legacy — original behavior where the webhook binary generates its own certs via init container
• Static test certificates (values-test-certs.yaml) for deterministic test snapshot generation
• Webhook templates restructured from nodeexecutor/ to webhook/ directory

🚨 Preserving current behavior

In order to preserve current behavior, dataplane/values-legacy.yaml is added to revert all defaults to current values. This can be used in conjunction with any other values file like this:

helm upgrade ..... -f values-legacy.yaml -f myvalues.yaml

Test plan

• Verify helm template renders correctly for default values
• Verify helm template with low_privilege: true produces namespace-scoped resources and no cluster-wide permissions
• Verify build-image-config configmap is created only in single-namespace mode
• Verify depot-token imagePullSecret appears in task pod template when Depot is enabled
• Verify webhook certificates render correctly for all providers (helm, certManager, external, legacy)
• Verify generated test fixtures match expected output
• Test upgrade path from existing deployments (webhook secret reuse, resource renames)

• main
  • 🚨 Breaking Change - Dataplane - Sane defaults 👈
    • Fix up comments in values.yaml to style #259

[aviator-app]

Current Aviator status

Aviator will automatically update this comment as the status of the PR changes.
Comment /aviator refresh to force Aviator to re-examine your PR (or learn about other /aviator commands).

This PR was merged using Aviator.

---

See the real-time status of this PR on the Aviator webapp.

Use the Aviator Chrome Extension to see the status of your PR within GitHub.

[davidmirror-ops]

Suggested change

	flyte-pod-webhook
	union-pod-webhook

or something that doesn't clash with the Flyte OSS one. Whenever a customer needs to run Union and Flyte OSS in the same namespace, this will make deployment fail

[EngHabu]

Why can't these be installed in separate namespaces?

[davidmirror-ops]

it can, but we have customers who only have permissions for a single namespace

[davidmirror-ops]

same here, this conflicts with Flyte OSS one

[EngHabu]

ditto, separate namespaces? the only thing that's cluster-wide is the MutatingWebhookConfiguration object for which I added "-org" in the name

[davidmirror-ops]

Suggested change

	name: union-pod-webhook

[davidmirror-ops]

I just tested this without overriding the default config for webhook certs and got this from a V2 execution that calls a secret

failed to execute node: failed at Node[a0]. RuntimeExecutionError: failed during plugin execution, caused by: failed to execute handle for plugin [container]: [InternalError] failed to create resource, caused by: Internal error occurred: failed calling webhook "flyte-pod-webhook.flyte.org": failed to call webhook: Post "https://flyte-pod-webhook.union.svc:443/mutate--v1-pod/secrets?timeout=30s": tls: failed to verify certificate: x509: certificate signed by unknown authority

[davidmirror-ops]

Not that testing is done but at least with the singleNamespace and low_privilege modes, the base components look and behave healthy