Self-hosted deployment docs (v2 variant structure) by ppiegaze · Pull Request #932 · unionai/unionai-docs

ppiegaze · 2026-04-22T15:17:23Z

Summary

Add self-hosted (intra-cluster) deployment guides under content/deployment/selfhosted/
Adapted from Self-hosted docs holding branch #843 to the current flyte/union variant structure
Includes AWS and GCP control plane/data plane guides, authentication, authorization, monitoring, image builder, and deployment glossary
Adds self-hosted section and link card to the deployment index page

Replaces #843 which was based on the old (flyte, byoc, selfmanaged) variant structure.

New pages

content/deployment/selfhosted/_index.md — Overview with architecture diagram
content/deployment/selfhosted/control-plane-aws.md — AWS control plane deployment
content/deployment/selfhosted/control-plane-gcp.md — GCP control plane deployment (Preview)
content/deployment/selfhosted/data-plane-aws.md — AWS data plane deployment
content/deployment/selfhosted/data-plane-gcp.md — GCP data plane deployment (Preview)
content/deployment/selfhosted/authentication.md — OIDC auth with OAuth apps
content/deployment/selfhosted/authorization.md — Authorization mode configuration
content/deployment/selfhosted/monitoring.md — Prometheus/Grafana monitoring
content/deployment/selfhosted/image-builder.md — Image builder registration
content/deployment/glossary.md — Deployment terminology

Test plan

make dist builds cleanly
New selfhosted pages appear in sidebar under Platform deployment
Pages only appear in union variant (not flyte)
Mermaid diagrams render correctly
Internal links resolve

🤖 Generated with Claude Code

Extracts self-hosted deployment content from peeter/selfhosted-docs (#843) and adapts it to the current main branch structure: - Place under content/deployment/selfhosted/ (matching byoc/, selfmanaged/) - Update variant tags from (-flyte -serverless -byoc +selfmanaged) to (-flyte +union) - Fix cross-references for new directory structure - Add self-hosted section and link card to deployment index - Include deployment glossary Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Copilot

Pull request overview

Adds a new self-hosted (intra-cluster) deployment documentation section under content/deployment/selfhosted/, aligned to the current flyte/union variant structure, and links it from the main Deployment index.

Changes:

Introduces self-hosted control plane + data plane deployment guides for AWS and GCP (preview).
Adds operational guides for self-hosted authn/authz, monitoring, and image builder registration.
Adds a deployment glossary and a new “Self-hosted deployment” entry on the deployment landing page.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
content/deployment/selfhosted/_index.md	Self-hosted landing page with architecture diagram and link-cards to guides
content/deployment/selfhosted/control-plane-aws.md	AWS control plane deployment steps
content/deployment/selfhosted/control-plane-gcp.md	GCP control plane deployment steps (preview)
content/deployment/selfhosted/data-plane-aws.md	AWS data plane deployment steps
content/deployment/selfhosted/data-plane-gcp.md	GCP data plane deployment steps (preview)
content/deployment/selfhosted/authentication.md	OIDC/OAuth2 setup for self-hosted deployments
content/deployment/selfhosted/authorization.md	Authorization modes and external authorizer contract
content/deployment/selfhosted/monitoring.md	Prometheus/Grafana/SLO/alerting guidance
content/deployment/selfhosted/image-builder.md	Image builder task registration instructions
content/deployment/glossary.md	Defines key deployment terms and models
content/deployment/_index.md	Adds a self-hosted section + card to the deployment index

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+{{< link-card target="./authentication" icon="lock" title="Authentication" >}}
+Configure OIDC/OAuth2 authentication for your deployment
+{{< /link-card >}}
+
+{{< link-card target="./authorization" icon="shield" title="Authorization" >}}
+Configure authorization mode (Noop, External, or Union built-in RBAC)
+{{< /link-card >}}
+
+{{< link-card target="./image-builder" icon="package" title="Image builder" >}}
+Register the image builder for automatic container image builds
+{{< /link-card >}}
+
+{{< /grid >}}


+: The orchestration layer that manages workflow execution state. Includes FlyteAdmin, scheduler, queue service, cache service, and supporting services. In self-managed deployments, the control plane is hosted by {{< key product_name >}}. In self-hosted deployments, you deploy it in the `union-cp` namespace.
+
+**Data plane**
+: The execution layer where your code and data reside. Includes the operator, propeller, and worker pods that run your tasks. Deployed in the `union` namespace.
+
+**Intra-cluster**
+: A deployment topology where the control plane and data plane run in the same Kubernetes cluster and communicate via internal DNS (e.g., `controlplane-nginx-controller.union-cp.svc.cluster.local`) rather than external networking.


+
+#### Install ScyllaDB CRDs (if using embedded ScyllaDB)
+
+```shell


+
+```shell


+Find the `appVersion` from your deployed controlplane Helm chart:
+
+```shell
+helm list -n <controlplane-namespace> -o json | jq '.[0].app_version'
+```
+
+Or check the chart directly:
+
+```shell
+grep appVersion charts/controlplane/Chart.yaml
+```


+## Step 3: Create Kubernetes secrets (control plane)
+
+The control plane needs secrets for the browser login app (App 1) and the service-to-service app (App 3):
+
+```shell
+# Secret for admin service (mounted at /etc/secrets/)
+# Note: "flyte-admin-secrets" is the default name expected by the Helm chart
+kubectl create secret generic flyte-admin-secrets \
+  --from-literal=client_secret='<BROWSER_LOGIN_CLIENT_SECRET>' \
+  -n <controlplane-namespace>
+
+# Secret for scheduler (mounted at /etc/secrets/)
+# Note: "flyte-secret-auth" is the default name expected by the Helm chart
+kubectl create secret generic flyte-secret-auth \
+  --from-literal=client_secret='<BROWSER_LOGIN_CLIENT_SECRET>' \
+  -n <controlplane-namespace>
+
+# Add service-to-service client secret to the controlplane secrets
+kubectl create secret generic <controlplane-secrets> \
+  --from-literal=pass.txt='<DB_PASSWORD>' \
+  --from-literal=client_secret='<SERVICE_TO_SERVICE_CLIENT_SECRET>' \
+  -n <controlplane-namespace> --dry-run=client -o yaml | kubectl apply -f -


* Update selfhosted auth docs for Entra ID and new auth block pattern Adapted from mike/selfhosted-auth-docs: adds Entra ID provider support, tabbed provider-specific instructions, identity type and subject claim requirements, and updated authorization server setup. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * Remove custom Entra ID browser scope from auth docs Browser login no longer requires the api://<app>/all custom scope. Standard OIDC scopes (profile, openid, offline_access) suffice — the IdP returns a plain ID token. Matches cloud PR #15558. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * Align Entra ID docs with entraid terraform module Source of truth: entraid authn module outputs + deployed identity-testing values. - Add identityTypeClaimsForApps to Entra YAML (was missing — critical for authz) - Add client_id to allowedAudience (module outputs both app URI + client_id) - Remove subjectClaimNames from Entra tab (Entra v2.0 always includes sub) - Add idpQueryParameter to userAuth - Clarify idtyp optional claim is configured on App 1 (resource server) only - Add admin consent warning for service app role assignments Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * Add selfhosted authorization bootstrap docs (#934) * Add Union authorization bootstrap docs for selfhosted deployments Adapted from mike/selfhosted-authz-bootstrap-docs: adds bootstrap procedure for initializing Union authorization mode with admin user and service account roles. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * Document bootstrap field schemas: adminUsers, projects, serviceAccounts - adminUsers: document object format (name + subject), not string list - projects: document pre-create project list field - serviceAccounts: clarify clientId = sub claim value from IdP Aligns with deployed identity-testing values and helm chart defaults. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * Fix bootstrap config field schemas in authorization docs - adminUsers: document object format {name, subject} (was string list) - projects: document pre-create project list field (was missing) - organization: use {{ .Values.global.UNION_ORG }} template reference - serviceAccounts: clarify clientId = resolved sub claim value Aligns with deployed identity-testing values and helm chart defaults. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> Co-authored-by: Michael Hotan <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> Co-authored-by: Michael Hotan <[email protected]>

cloudflare-workers-and-pages · 2026-04-24T22:06:14Z

Deploying docs with Cloudflare Pages

Latest commit:	`42cb445`
Status:	✅ Deploy successful!
Preview URL:	https://c0f83cc3.docs-dog.pages.dev
Branch Preview URL:	https://peeter-selfhosted-docs-v2.docs-dog.pages.dev

View logs

* Add self-hosted CI/CD integration guide under operations section (FAB-273) New operations/ section for post-deployment guides. CI/CD page documents how to create an OAuth app, encode credentials as FLYTE_API_KEY, and deploy from CI pipelines (GitHub Actions, Jenkins, GitLab CI) with IdP-specific tabs for Okta, Entra ID, and generic OIDC. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * Fix docs_home shortcode: add required v2 version parameter Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>

Copilot AI review requested due to automatic review settings April 22, 2026 15:17

ppiegaze requested review from EngHabu, cosmicBboy, kumare3 and samhita-alla as code owners April 22, 2026 15:17

ppiegaze mentioned this pull request Apr 22, 2026

Update selfhosted auth docs for Entra ID #933

Merged

2 tasks

Copilot started reviewing on behalf of ppiegaze April 22, 2026 15:17 View session

This was referenced Apr 22, 2026

Self-hosted docs holding branch #843

Closed

Update selfhosted auth docs: Entra ID support and new auth block pattern #925

Closed

Copilot AI reviewed Apr 22, 2026

View reviewed changes

ppiegaze added self-hosted Self-hosted docs. Do not merge do-not-merge PR is ready for review, but should not be merged just yet labels Apr 23, 2026

mhotan force-pushed the peeter/selfhosted-docs-v2 branch 2 times, most recently from 6b60395 to 4d6bd91 Compare April 29, 2026 04:12

mhotan and others added 2 commits April 29, 2026 14:13

Merge branch 'main' into peeter/selfhosted-docs-v2

0c20b60

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Self-hosted deployment docs (v2 variant structure)#932

Self-hosted deployment docs (v2 variant structure)#932
ppiegaze wants to merge 4 commits intomainfrom
peeter/selfhosted-docs-v2

ppiegaze commented Apr 22, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

cloudflare-workers-and-pages Bot commented Apr 24, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants


		#### Install ScyllaDB CRDs (if using embedded ScyllaDB)

		```shell

Conversation

ppiegaze commented Apr 22, 2026

Summary

New pages

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

cloudflare-workers-and-pages Bot commented Apr 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Deploying docs with Cloudflare Pages

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

cloudflare-workers-and-pages Bot commented Apr 24, 2026 •

edited

Loading