diff --git a/README.md b/README.md index 44ddf08..515df0f 100644 --- a/README.md +++ b/README.md @@ -57,8 +57,10 @@ applies the additions/updates with [Server-Side Apply], and deletes orphans. | `middlewares.addPrefix` | HTTPRoute filter `URLRewrite{path.ReplacePrefixMatch}` | | `middlewares.replacePath` | HTTPRoute filter `URLRewrite{path.ReplaceFullPath}` | | `middlewares.replacePathRegex` | not in core Gateway API – logged + skipped | -| `middlewares.plugin.badger` | pangolin's auth plugin – configure via Envoy Gateway `SecurityPolicy` instead | -| `tcp.*` / `udp.*` | not yet handled (planned: `TCPRoute` / `UDPRoute`) | +| `middlewares.plugin.badger` | when `CONFIG_BADGER_EXT_AUTH=true`: per-route Envoy Gateway `SecurityPolicy.extAuth`; otherwise logged + skipped | +| Pangolin dashboard host | optional static `HTTPRoute`s for Next.js (`:3002`) and API/WebSocket (`:3000`) when `CONFIG_PANGOLIN_DASHBOARD_HOST` is set | +| Gerbil UDP ports | optional `UDPRoute`s + UDP listeners when `CONFIG_GERBIL_UDP_ROUTE=true` | +| `tcp.*` / other `udp.*` dynamic config | logged + skipped | Unsupported rule constructs (`||` disjunction, `!` negation, `HostRegexp`, `Method`, `Headers`, …) cause the affected router to be **logged and skipped** @@ -81,6 +83,83 @@ direct `Service` `backendRef` — the controller does not synthesize a duplicate Pick the default unless you specifically want FQDN backends or the `Backend` CRD's other features (health checking via `BackendTrafficPolicy`, etc.). +## Badger external auth + +Gateway API cannot express Traefik plugins directly. When +`CONFIG_BADGER_EXT_AUTH=true`, routers that reference Pangolin's `badger` +plugin get a managed Envoy Gateway `SecurityPolicy` targeting the generated +`HTTPRoute`. + +The policy must point at an HTTP service that speaks Envoy's ext_authz protocol. +Pangolin's `/api/v1/badger/verify-session` endpoint expects Badger's JSON body, +so in today's Pangolin releases this normally means running a small shim service +that adapts Envoy's auth check request to Pangolin's badger verify-session API. +The controller defaults to a Service named `pangolin-badger-ext-authz` on port +`9002`; override that with the `CONFIG_BADGER_EXT_AUTH_*` settings. + +## Scoped reconcile + +Set `CONFIG_RECONCILE_ONLY` to a comma-separated allow-list to test a subset of +objects without touching the rest of the controller-owned state. The easiest +form is a hostname; the controller expands it to the matching `HTTPRoute`, its +badger `SecurityPolicy`, generated backend objects, and the shared +`ListenerSet`: + +```sh +CONFIG_RECONCILE_ONLY="9-chris.example.com" +``` + +Explicit object selectors still work as `Kind/name` or `Kind:name`, for example +`HTTPRoute/hr-9-chris-connect-router`. Use `Hostname/name` if you need to force +a hostname selector that does not contain a dot. + +Apply and GC both honor the scope. Objects outside the scope are left alone, and +selected objects that are no longer desired can still be garbage-collected. + +## Migration config changes + +For a dual-stack rollout, keep Traefik active and add the Gateway API pieces in +small steps: + +1. Deploy a badger ext_auth shim Service, then set: + + ```sh + CONFIG_BADGER_EXT_AUTH=true + CONFIG_BADGER_EXT_AUTH_BACKEND_NAME=pangolin-badger-ext-authz + CONFIG_BADGER_EXT_AUTH_BACKEND_PORT=9002 + ``` + +2. Start with a narrow reconcile scope: + + ```sh + CONFIG_RECONCILE_ONLY="9-chris.example.com" + ``` + +3. To serve Pangolin itself through Envoy Gateway, expose the Pangolin Service + ports for API/WebSocket (`3000`) and Next.js (`3002`), then set: + + ```sh + CONFIG_PANGOLIN_DASHBOARD_HOST=pangolin.example.com + ``` + +4. To move Gerbil UDP traffic through Envoy Gateway, expose UDP ports on the + Gerbil Service and enable: + + ```sh + CONFIG_GERBIL_UDP_ROUTE=true + CONFIG_GERBIL_UDP_PORTS=51820,21820 + ``` + +Remove `CONFIG_RECONCILE_ONLY` only after the selected route, policy, dashboard, +and UDP paths have been validated against Envoy Gateway. + +For the Gerbil public-IP migration pattern, including direct old-IP traffic plus +a separate Envoy Gateway IP, see +[docs/gerbil-gateway-migration.md](docs/gerbil-gateway-migration.md). +For the shared-IP path where only HTTP/HTTPS moves from Traefik to Envoy while +Gerbil keeps the WireGuard UDP ports, see +[docs/shared-ip-http-envoy-migration.md](docs/shared-ip-http-envoy-migration.md). + ## Certificate handling with cert-manager There are two ways the controller fills in `tls.certificateRefs` on its listeners: @@ -181,6 +260,25 @@ upstream Go controller (`CONFIG_*`) where the concepts overlap. | `CONFIG_BACKEND_KIND` | `service` | `service` (default) or `envoy-backend`. See [Backend strategy](#backend-strategy) | | `CONFIG_HTTPROUTE_ANNOTATIONS` | _(unset)_ | `k=v,k=v` annotations stamped on every HTTPRoute. Typical: `cert-manager.io/cluster-issuer=letsencrypt-prod` | | `CONFIG_LISTENERSET_ANNOTATIONS` | _(unset)_ | `k=v,k=v` annotations stamped on the ListenerSet | +| `CONFIG_BADGER_EXT_AUTH` | `false` | Emit Envoy Gateway `SecurityPolicy.extAuth` for routes that use badger | +| `CONFIG_BADGER_EXT_AUTH_BACKEND_NAME` | `pangolin-badger-ext-authz` | Service name of the Envoy ext_auth-compatible badger shim | +| `CONFIG_BADGER_EXT_AUTH_BACKEND_NAMESPACE` | controller namespace | Namespace of the ext_auth backend Service | +| `CONFIG_BADGER_EXT_AUTH_BACKEND_PORT` | `9002` | Service port for the ext_auth backend | +| `CONFIG_BADGER_EXT_AUTH_PATH` | _(unset)_ | Optional fixed auth check path | +| `CONFIG_BADGER_EXT_AUTH_HEADERS_TO_EXTAUTH` | `authorization,cookie,x-forwarded-for,x-forwarded-host,x-forwarded-proto,x-real-ip,p-access-token-id,p-access-token` | Headers forwarded to the auth service | +| `CONFIG_BADGER_EXT_AUTH_HEADERS_TO_BACKEND` | `remote-user,remote-email,remote-name,remote-role` | Headers copied from auth response to upstream backends | +| `CONFIG_BADGER_EXT_AUTH_FAIL_OPEN` | `false` | Allow traffic if the auth service is unavailable | +| `CONFIG_RECONCILE_ONLY` | _(unset)_ | Optional comma-separated allow-list of hostnames or `Kind/name` objects to apply/GC | +| `CONFIG_PANGOLIN_DASHBOARD_HOST` | _(unset)_ | When set, emit static dashboard HTTPRoutes for this hostname | +| `CONFIG_PANGOLIN_SERVICE_NAME` | `pangolin` | Service that backs the dashboard/API routes | +| `CONFIG_PANGOLIN_SERVICE_NAMESPACE`| controller namespace | Namespace of the Pangolin Service | +| `CONFIG_PANGOLIN_API_PORT` | `3000` | Pangolin API/WebSocket Service port | +| `CONFIG_PANGOLIN_NEXT_PORT` | `3002` | Pangolin Next.js Service port | +| `CONFIG_PANGOLIN_REDIRECT_HTTP_TO_HTTPS` | `true` | Emit an HTTP-to-HTTPS redirect route when HTTPS listeners are configured | +| `CONFIG_GERBIL_UDP_ROUTE` | `false` | Emit UDP listeners and UDPRoutes for Gerbil | +| `CONFIG_GERBIL_SERVICE_NAME` | `gerbil` | Service that backs Gerbil UDP traffic | +| `CONFIG_GERBIL_SERVICE_NAMESPACE` | controller namespace | Namespace of the Gerbil Service | +| `CONFIG_GERBIL_UDP_PORTS` | `51820,21820` | UDP ports to expose through the ListenerSet | | `CONFIG_FIELD_MANAGER` | `pangolin-gateway-controller` | Server-Side Apply field manager | | `CONFIG_MANAGED_LABEL_KEY` | `app.kubernetes.io/managed-by` | Used for GC selector | | `CONFIG_MANAGED_LABEL_VALUE` | `pangolin-gateway-controller` | Used for GC selector | diff --git a/deploy/README.md b/deploy/README.md index b3633c2..1dc1d0d 100644 --- a/deploy/README.md +++ b/deploy/README.md @@ -10,14 +10,15 @@ and as a component from sibling infrastructure repos. | File | Resource | Notes | |---|---|---| | `namespace.yaml` | `Namespace/pangolin-system` | Drop this if the namespace is created elsewhere — see [Embedding from another kustomization](#embedding-from-another-kustomization). | -| `rbac.yaml` | `ServiceAccount`, `ClusterRole`, `ClusterRoleBinding` | Cluster-wide read on Gateways; write on HTTPRoute / ListenerSet / Service / EndpointSlice / Envoy Gateway `Backend`. | +| `rbac.yaml` | `ServiceAccount`, `ClusterRole`, `ClusterRoleBinding` | Cluster-wide read on Gateways; write on HTTPRoute / UDPRoute / ListenerSet / Service / EndpointSlice / Envoy Gateway `Backend` and `SecurityPolicy`. | | `deployment.yaml` | `Deployment/pangolin-gateway-controller` | Single replica, read-only rootfs, all caps dropped. | The CRDs are **not** installed here. You need: - Gateway API v1.5+ **experimental** channel (for `ListenerSet`): `kubectl apply -k https://github.com/kubernetes-sigs/gateway-api/config/crd/experimental?ref=v1.5.1` -- Envoy Gateway, if you set `CONFIG_BACKEND_KIND=envoy-backend`: +- Envoy Gateway, if you set `CONFIG_BACKEND_KIND=envoy-backend` or + `CONFIG_BADGER_EXT_AUTH=true`: see . A `Gateway` object the controller can attach its `ListenerSet` to must exist @@ -72,6 +73,13 @@ that file for authoritative defaults. | `CONFIG_TLS_SECRET_TEMPLATE` | _(unset)_ | Required when HTTPS listeners are on. Supports `{hostname}` and `{hostname-dashed}` placeholders. | | `CONFIG_HTTPROUTE_ANNOTATIONS` | _(empty)_ | `k=v,k=v` annotations stamped onto every `HTTPRoute`. Typical use: cert-manager cluster-issuer. | | `CONFIG_LISTENERSET_ANNOTATIONS` | _(empty)_ | Same for `ListenerSet`. | +| `CONFIG_BADGER_EXT_AUTH` | `false` | Emit Envoy Gateway `SecurityPolicy.extAuth` for routes that use Pangolin's badger plugin. Requires an Envoy ext_auth-compatible shim service. | +| `CONFIG_BADGER_EXT_AUTH_BACKEND_NAME` | `pangolin-badger-ext-authz` | Auth shim Service name. | +| `CONFIG_BADGER_EXT_AUTH_BACKEND_PORT` | `9002` | Auth shim Service port. | +| `CONFIG_RECONCILE_ONLY` | _(empty)_ | Optional comma-separated allow-list of hostnames or object selectors such as `9-chris.example.com` or `HTTPRoute/hr-9-chris-connect-router`. | +| `CONFIG_PANGOLIN_DASHBOARD_HOST` | _(empty)_ | Emit static dashboard/API/WebSocket HTTPRoutes for this host. | +| `CONFIG_GERBIL_UDP_ROUTE` | `false` | Emit Gerbil UDP listeners and UDPRoutes; requires UDPRoute support in your Gateway implementation. | +| `CONFIG_GERBIL_UDP_PORTS` | `51820,21820` | UDP ports to route to the Gerbil Service. | | `CONFIG_FIELD_MANAGER` | `pangolin-gateway-controller` | SSA field manager. Change only if you need to migrate ownership. | | `RUST_LOG` | `info` | Standard `tracing_subscriber::EnvFilter`. | @@ -98,6 +106,80 @@ annotation env vars to your cluster issuer: cert-manager will see the annotated `ListenerSet`, mint a Certificate per listener hostname, and write the Secret matching `CONFIG_TLS_SECRET_TEMPLATE`. +## Badger ext_authz (optional) + +The controller can close the badger gap from the Traefik plugin side by emitting +Envoy Gateway `SecurityPolicy` objects, but it does not make Pangolin's badger +endpoint speak Envoy's auth protocol by itself. Run a shim Service that accepts +Envoy HTTP ext_auth checks and calls +`http://pangolin:3001/api/v1/badger/verify-session`, then enable: + +```yaml +- name: CONFIG_BADGER_EXT_AUTH + value: "true" +- name: CONFIG_BADGER_EXT_AUTH_BACKEND_NAME + value: "pangolin-badger-ext-authz" +- name: CONFIG_BADGER_EXT_AUTH_BACKEND_PORT + value: "9002" +``` + +During migration, combine this with `CONFIG_RECONCILE_ONLY` to apply a single +hostname first. Hostname scopes expand to the matching `HTTPRoute`, badger +`SecurityPolicy`, generated backend objects, and the shared `ListenerSet`. + +## Configuration changes for migration + +When enabling the new Gateway API path in an existing dual-stack deployment, +change configuration in this order: + +1. Keep Traefik serving production traffic. +2. Add or deploy a Service that fronts your badger ext_auth shim. The Service + name and port must match `CONFIG_BADGER_EXT_AUTH_BACKEND_NAME` and + `CONFIG_BADGER_EXT_AUTH_BACKEND_PORT`. +3. Make sure the Pangolin Service exposes the API/WebSocket port (`3000` by + default) and the Next.js port (`3002` by default) if you want + `CONFIG_PANGOLIN_DASHBOARD_HOST`. +4. Make sure the Gerbil Service exposes UDP `51820` and `21820`, or override + `CONFIG_GERBIL_UDP_PORTS`. +5. Patch the controller Deployment env vars. + +Example kustomize patch: + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: pangolin-gateway-controller + namespace: pangolin-system +spec: + template: + spec: + containers: + - name: controller + env: + - name: CONFIG_BADGER_EXT_AUTH + value: "true" + - name: CONFIG_BADGER_EXT_AUTH_BACKEND_NAME + value: "pangolin-badger-ext-authz" + - name: CONFIG_BADGER_EXT_AUTH_BACKEND_PORT + value: "9002" + - name: CONFIG_RECONCILE_ONLY + value: "9-chris.example.com" + - name: CONFIG_PANGOLIN_DASHBOARD_HOST + value: "pangolin.example.com" + - name: CONFIG_GERBIL_UDP_ROUTE + value: "true" +``` + +For the first test, keep `CONFIG_RECONCILE_ONLY` narrow. Remove it only after the +selected hostname's `HTTPRoute` and `SecurityPolicy` are accepted by Envoy +Gateway and the auth shim returns the expected allow/redirect/deny responses. +For Gerbil's old-IP plus Envoy-IP migration pattern, see +[`docs/gerbil-gateway-migration.md`](../docs/gerbil-gateway-migration.md). +For a shared public IP where only TCP `80`/`443` moves from Traefik to Envoy, +see +[`docs/shared-ip-http-envoy-migration.md`](../docs/shared-ip-http-envoy-migration.md). + ## Embedding from another kustomization The deploy folder is a working stand-alone overlay. To embed it inside an diff --git a/deploy/deployment.yaml b/deploy/deployment.yaml index 8f99d08..f0555f8 100644 --- a/deploy/deployment.yaml +++ b/deploy/deployment.yaml @@ -54,6 +54,23 @@ spec: # value: "cert-manager.io/cluster-issuer=letsencrypt-prod" # - name: CONFIG_LISTENERSET_ANNOTATIONS # value: "cert-manager.io/cluster-issuer=letsencrypt-prod" + # Badger ext_authz (optional). Point this at an Envoy ext_auth-compatible + # shim for pangolin's /api/v1/badger/verify-session protocol. + # - name: CONFIG_BADGER_EXT_AUTH + # value: "true" + # - name: CONFIG_BADGER_EXT_AUTH_BACKEND_NAME + # value: "pangolin-badger-ext-authz" + # - name: CONFIG_BADGER_EXT_AUTH_BACKEND_PORT + # value: "9002" + # Test-mode reconcile scope (optional), e.g. only one hostname. + # - name: CONFIG_RECONCILE_ONLY + # value: "9-chris.example.com" + # Pangolin dashboard through Gateway API (optional). + # - name: CONFIG_PANGOLIN_DASHBOARD_HOST + # value: "pangolin.example.com" + # Gerbil UDP through Gateway API (optional; requires UDPRoute support). + # - name: CONFIG_GERBIL_UDP_ROUTE + # value: "true" - name: RUST_LOG value: "info,pangolin_gateway_controller=info" resources: diff --git a/deploy/rbac.yaml b/deploy/rbac.yaml index 05876bf..1a9e806 100644 --- a/deploy/rbac.yaml +++ b/deploy/rbac.yaml @@ -11,10 +11,10 @@ metadata: rules: # Gateway API objects we own. - apiGroups: ["gateway.networking.k8s.io"] - resources: ["httproutes", "listenersets"] + resources: ["httproutes", "listenersets", "udproutes"] verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] - apiGroups: ["gateway.networking.k8s.io"] - resources: ["httproutes/status", "listenersets/status"] + resources: ["httproutes/status", "listenersets/status", "udproutes/status"] verbs: ["get"] # Core resources we synthesize for IP-backed pangolin targets. - apiGroups: [""] @@ -27,10 +27,11 @@ rules: - apiGroups: ["gateway.networking.k8s.io"] resources: ["gateways", "gatewayclasses"] verbs: ["get", "list", "watch"] - # Envoy Gateway's Backend CRD. Only used when CONFIG_BACKEND_KIND=envoy-backend. - # Safe to leave granted even when unused — RBAC checks happen lazily. + # Envoy Gateway CRDs. Backend is used when CONFIG_BACKEND_KIND=envoy-backend; + # SecurityPolicy is used when CONFIG_BADGER_EXT_AUTH=true. + # Safe to leave granted even when unused — checks happen lazily. - apiGroups: ["gateway.envoyproxy.io"] - resources: ["backends"] + resources: ["backends", "securitypolicies"] verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] --- apiVersion: rbac.authorization.k8s.io/v1 diff --git a/docs/gerbil-gateway-migration.md b/docs/gerbil-gateway-migration.md new file mode 100644 index 0000000..410e677 --- /dev/null +++ b/docs/gerbil-gateway-migration.md @@ -0,0 +1,237 @@ +# Gerbil Gateway API Migration + +This is the safe rollout shape for moving Gerbil L4 traffic toward Envoy +Gateway without taking the old public address away from the existing Gerbil +`LoadBalancer` Service. + +For the shared-IP variant where Gerbil keeps UDP and only HTTP/HTTPS moves from +Traefik to Envoy, see +[shared-ip-http-envoy-migration.md](shared-ip-http-envoy-migration.md). + +## Target Shape + +```mermaid +flowchart LR + internet["Internet clients"] + oldip["old external IP\nGerbil LoadBalancer Service"] + envoyip["new external IP\nEnvoy Gateway Service"] + gateway["Gateway API\nUDPRoute / TCPRoute"] + gerbil["Gerbil pods"] + + internet --> oldip --> gerbil + internet --> envoyip --> gateway --> gerbil +``` + +Yes, Gerbil can receive traffic from both paths at the same time. The important +constraint is that the same external IP and port cannot normally be owned by the +old Gerbil `LoadBalancer` Service and the Envoy Gateway `LoadBalancer` Service +simultaneously. Start with two public IPs, then move the old IP to Envoy only +when Envoy is the only intended public entrypoint for those ports. + +## Prerequisites + +- Gateway API experimental CRDs are installed, because `UDPRoute` and `TCPRoute` + are `gateway.networking.k8s.io/v1alpha2`. +- Envoy Gateway supports the L4 protocols and ports you want to expose. +- Gerbil has a stable in-cluster backend Service that Envoy can route to. +- The old direct `LoadBalancer` Service keeps the old external IP during the + test phase. + +## Services + +Keep the old Service that owns the old IP. Use your provider's static-IP +annotation if it does not honor `loadBalancerIP`. + +```yaml +apiVersion: v1 +kind: Service +metadata: + name: gerbil-public-old + namespace: pangolin-system +spec: + type: LoadBalancer + loadBalancerIP: OLD_PUBLIC_IP + externalTrafficPolicy: Local + selector: + app.kubernetes.io/name: gerbil + ports: + - name: wg-51820 + protocol: UDP + port: 51820 + targetPort: 51820 + - name: wg-21820 + protocol: UDP + port: 21820 + targetPort: 21820 + - name: tcp-443 + protocol: TCP + port: 443 + targetPort: 443 +``` + +Add or reuse a normal in-cluster Service for Envoy's route backend: + +```yaml +apiVersion: v1 +kind: Service +metadata: + name: gerbil + namespace: pangolin-system +spec: + type: ClusterIP + selector: + app.kubernetes.io/name: gerbil + ports: + - name: wg-51820 + protocol: UDP + port: 51820 + targetPort: 51820 + - name: wg-21820 + protocol: UDP + port: 21820 + targetPort: 21820 + - name: tcp-443 + protocol: TCP + port: 443 + targetPort: 443 +``` + +## Separate Envoy Gateway IP + +Create a Gateway whose Envoy data plane gets its own external IP first. The +exact static-IP field depends on your Envoy Gateway and cloud provider setup; +the example shows only the Gateway API part. + +```yaml +apiVersion: gateway.networking.k8s.io/v1 +kind: Gateway +metadata: + name: gerbil-eg + namespace: envoy-gateway-system +spec: + gatewayClassName: eg + listeners: + - name: gerbil-udp-51820 + protocol: UDP + port: 51820 + allowedRoutes: + namespaces: + from: All + - name: gerbil-udp-21820 + protocol: UDP + port: 21820 + allowedRoutes: + namespaces: + from: All + - name: gerbil-tcp-443 + protocol: TCP + port: 443 + allowedRoutes: + namespaces: + from: All +``` + +## UDPRoute And TCPRoute + +Attach routes to the new Gateway and point them at the Gerbil backend Service. + +```yaml +apiVersion: gateway.networking.k8s.io/v1alpha2 +kind: UDPRoute +metadata: + name: gerbil-udp-51820 + namespace: pangolin-system +spec: + parentRefs: + - group: gateway.networking.k8s.io + kind: Gateway + name: gerbil-eg + namespace: envoy-gateway-system + sectionName: gerbil-udp-51820 + rules: + - backendRefs: + - group: "" + kind: Service + name: gerbil + port: 51820 +--- +apiVersion: gateway.networking.k8s.io/v1alpha2 +kind: UDPRoute +metadata: + name: gerbil-udp-21820 + namespace: pangolin-system +spec: + parentRefs: + - group: gateway.networking.k8s.io + kind: Gateway + name: gerbil-eg + namespace: envoy-gateway-system + sectionName: gerbil-udp-21820 + rules: + - backendRefs: + - group: "" + kind: Service + name: gerbil + port: 21820 +--- +apiVersion: gateway.networking.k8s.io/v1alpha2 +kind: TCPRoute +metadata: + name: gerbil-tcp-443 + namespace: pangolin-system +spec: + parentRefs: + - group: gateway.networking.k8s.io + kind: Gateway + name: gerbil-eg + namespace: envoy-gateway-system + sectionName: gerbil-tcp-443 + rules: + - backendRefs: + - group: "" + kind: Service + name: gerbil + port: 443 +``` + +The controller can emit Gerbil `UDPRoute`s for its managed `ListenerSet` with: + +```yaml +- name: CONFIG_GERBIL_UDP_ROUTE + value: "true" +- name: CONFIG_GERBIL_UDP_PORTS + value: "51820,21820" +- name: CONFIG_GERBIL_SERVICE_NAME + value: "gerbil" +``` + +Use either the controller-managed UDP path or the manual Gateway above for the +same port, not both, unless they attach to different Envoy Gateways/IPs. +`TCPRoute` is currently a manual manifest. + +## Validation + +Check that both entrypoints exist: + +```sh +kubectl -n pangolin-system get svc gerbil-public-old gerbil +kubectl -n envoy-gateway-system get gateway gerbil-eg +kubectl -n pangolin-system get udproute,tcproute +``` + +Then test clients against the new Envoy external IP while existing clients keep +using the old IP. Watch Gerbil logs and Envoy Gateway route status conditions. + +## Cutover + +When the Envoy path is validated: + +1. Remove or pause the old Gerbil `LoadBalancer` Service so it releases the old + IP. +2. Assign the old static IP to the Envoy Gateway data-plane Service using your + cloud provider's static-IP mechanism. +3. Confirm `UDPRoute` and `TCPRoute` are accepted and the old IP now reaches + Gerbil through Envoy. +4. Delete the temporary Envoy IP after traffic has drained. + +Do not move the old IP while the direct Gerbil `LoadBalancer` still owns it. diff --git a/docs/shared-ip-http-envoy-migration.md b/docs/shared-ip-http-envoy-migration.md new file mode 100644 index 0000000..d109f1a --- /dev/null +++ b/docs/shared-ip-http-envoy-migration.md @@ -0,0 +1,284 @@ +# Shared IP HTTP/HTTPS Envoy Migration + +This path keeps one public IP, leaves Gerbil responsible for WireGuard traffic, +and moves only web traffic from Traefik to Envoy Gateway. + +## Target Shape + +```mermaid +flowchart LR + internet["Internet clients"] + ip["same public IP"] + udp["UDP 51820/21820"] + tcp["TCP 80/443"] + gerbil["Gerbil"] + envoy["Envoy Gateway"] + routes["Gateway API HTTPRoutes"] + overlay["Gerbil / overlay reachability"] + backends["Pangolin resources"] + + internet --> ip + ip --> udp --> gerbil + ip --> tcp --> envoy --> routes --> overlay --> backends +``` + +This is "Envoy replaces Traefik" for HTTP/HTTPS only. Gerbil remains the +WireGuard/tunnel component and the old public endpoint for UDP. For tunneled +resources, Envoy must have the same backend reachability that Traefik had, +which may mean reaching resource networks through Gerbil or another cluster +route to the Gerbil-managed overlay. + +## What The Upstream Helm Chart Does + +The upstream chart supports controller mode, standalone mode, and single or +multi workload topology. In the chart README, controller mode is the +recommended production path and expects a Traefik controller; standalone mode +runs chart-managed Traefik; single mode puts Pangolin plus optional Gerbil and +either controller or standalone Traefik into one shared Pod. + +Relevant chart behavior: + +- `gerbil.service` is value-driven and selects Gerbil pods in multi mode, or + the shared single Pod in single mode. +- Default Gerbil service ports are `wg1` UDP `51820`, `wg2` UDP `21820`, and + `internal-api` TCP `3004`. +- Standalone Traefik listens on container ports `8000` and `8443`; its Service + exposes public TCP `80` and `443` to those container ports. +- In single+standalone mode, one Service can target ports from both Gerbil and + Traefik because all containers live in the same Pod network namespace. + +Important consequence: the chart does not configure Gerbil as the HTTP/HTTPS +frontend that forwards port 80/443 to Traefik. Gerbil is not a drop-in L7 +replacement for a LoadBalancer or Gateway. With standard Envoy Gateway, Envoy +data-plane pods are separate from the Pangolin/Gerbil pods, so a single +Kubernetes Service cannot select Gerbil for UDP and Envoy for TCP unless your +load balancer supports sharing one IP across multiple Services, or you put an +external L4 load balancer in front. + +Sources checked: + +- [Pangolin chart README](https://github.com/fosrl/helm-charts/blob/main/charts/pangolin/README.md) +- [Pangolin chart values](https://github.com/fosrl/helm-charts/blob/main/charts/pangolin/values.yaml) +- [Gerbil Service template](https://github.com/fosrl/helm-charts/blob/main/charts/pangolin/templates/service-gerbil.yaml) +- [Standalone Traefik Service template](https://github.com/fosrl/helm-charts/blob/main/charts/pangolin/templates/service-traefik.yaml) +- [Single Pod template](https://github.com/fosrl/helm-charts/blob/main/charts/pangolin/templates/deployment-single.yaml) + +## Recommended Shared-IP Pattern + +Use one public IP with two L4 owners: + +- Gerbil public UDP Service owns UDP `51820` and `21820`. +- Envoy Gateway data-plane Service owns TCP `80` and `443`. + +This requires one of: + +- a load-balancer implementation that supports the same IP on multiple Services + when their ports do not overlap, such as some MetalLB or Cilium LB IPAM + setups; +- a cloud/provider feature that can bind different port mappings for the same + IP to different Kubernetes Services; +- an external L4 load balancer that sends UDP ports to Gerbil and TCP ports to + Envoy. + +If your provider cannot share one IP across Services, Kubernetes cannot express +"UDP to Service A, TCP to Service B, same external IP" with ordinary Services. +Use the separate-IP migration first, or do a cutover where Envoy becomes the +only owner of the public IP. + +## Keep The Chart Gerbil Service Internal + +For a clean production setup, keep the chart-managed Gerbil Service as +`ClusterIP` so Gerbil's internal API stays internal: + +```yaml +gerbil: + service: + enabled: true + type: ClusterIP +``` + +Then create a separate public UDP Service outside the chart. Copy the selector +from the chart-rendered Gerbil Service: + +```sh +kubectl -n pangolin-system get svc -pangolin-gerbil -o yaml +``` + +Use the same selector in a UDP-only public Service: + +```yaml +apiVersion: v1 +kind: Service +metadata: + name: gerbil-public-udp + namespace: pangolin-system + annotations: + # Provider-specific static/shared IP annotations go here. + example.com/shared-ip-group: pangolin-public +spec: + type: LoadBalancer + loadBalancerIP: OLD_PUBLIC_IP + externalTrafficPolicy: Local + selector: + # Copy from the chart-rendered -pangolin-gerbil Service. + # These are examples; single mode uses the shared single-Pod selector. + app.kubernetes.io/instance: RELEASE_NAME + app.kubernetes.io/component: gerbil + ports: + - name: wg1 + protocol: UDP + port: 51820 + targetPort: wg1 + - name: wg2 + protocol: UDP + port: 21820 + targetPort: wg2 +``` + +Avoid exposing Gerbil `internal-api` TCP `3004` on the public LoadBalancer. + +## Put Envoy On TCP 80/443 Of The Same IP + +Configure the Envoy Gateway data-plane Service with the same static IP and the +same provider-specific shared-IP group, but only for TCP `80` and `443`. + +The exact object is Envoy Gateway installation dependent. In many clusters, +Envoy Gateway creates a Service in the Envoy namespace for the data-plane +Deployment. Patch or configure that Service through your Envoy Gateway +installation mechanism rather than hand-editing generated resources when your +operator supports a first-class value. + +Schematic only: + +```yaml +apiVersion: v1 +kind: Service +metadata: + name: envoy-gateway-data-plane + namespace: envoy-gateway-system + annotations: + # Must match the Gerbil public UDP Service for providers that require a + # shared-IP grouping key. + example.com/shared-ip-group: pangolin-public +spec: + type: LoadBalancer + loadBalancerIP: OLD_PUBLIC_IP + ports: + - name: http + protocol: TCP + port: 80 + - name: https + protocol: TCP + port: 443 +``` + +Make sure the final public IP ownership is non-overlapping: + +```sh +kubectl -n pangolin-system get svc gerbil-public-udp +kubectl -n envoy-gateway-system get svc +``` + +You should see the same external IP on both Services, with UDP ports only on +Gerbil and TCP web ports only on Envoy. + +## Gateway API Setup + +Create or keep a parent Gateway that allows the controller's `ListenerSet` to +attach: + +```yaml +apiVersion: gateway.networking.k8s.io/v1 +kind: Gateway +metadata: + name: eg + namespace: envoy-gateway-system +spec: + gatewayClassName: eg + allowedListeners: + namespaces: + from: All + listeners: + - name: placeholder + protocol: HTTP + port: 8080 +``` + +Then configure the Pangolin Gateway API controller for Envoy: + +```yaml +- name: CONFIG_PARENT_GATEWAY + value: "eg" +- name: CONFIG_PARENT_GATEWAY_NAMESPACE + value: "envoy-gateway-system" +- name: CONFIG_ENABLE_HTTPS_LISTENERS + value: "true" +- name: CONFIG_TLS_SECRET_TEMPLATE + value: "{hostname-dashed}-tls" +- name: CONFIG_BADGER_EXT_AUTH + value: "true" +- name: CONFIG_BADGER_EXT_AUTH_BACKEND_NAME + value: "pangolin-badger-ext-authz" +- name: CONFIG_RECONCILE_ONLY + value: "first-test-resource.example.com" +``` + +Once the selected hostname works through Envoy, remove +`CONFIG_RECONCILE_ONLY`. + +## Pangolin Dashboard + +If the Pangolin dashboard should also move from Traefik to Envoy, disable or +ignore the Traefik dashboard `IngressRoute` and enable the controller's Gateway +API dashboard routes: + +```yaml +- name: CONFIG_PANGOLIN_DASHBOARD_HOST + value: "pangolin.example.com" +``` + +The Pangolin Service must expose: + +- API/WebSocket port `3000` via `CONFIG_PANGOLIN_API_PORT` +- Next.js UI port `3002` via `CONFIG_PANGOLIN_NEXT_PORT` + +## Critical Reachability Check + +Replacing Traefik with Envoy is only safe if Envoy can reach the same resource +backends Traefik reached. + +The upstream chart's single+standalone mode can make Traefik and Gerbil share a +Pod network namespace. Standard Envoy Gateway data-plane pods do not share that +namespace. If your current Traefik reaches tunneled resources only because it is +co-located with Gerbil, Envoy may not have equivalent reachability by default. + +Before moving production traffic: + +```sh +kubectl -n get httproute +kubectl -n get securitypolicy +kubectl -n get svc,endpointslice +kubectl -n envoy-gateway-system logs deploy/ +``` + +Then test a single hostname through the shared IP. Confirm: + +- TLS terminates at Envoy. +- Badger auth redirects or allows as expected. +- The selected route can reach its backend. +- Gerbil UDP tunnels still connect through the same public IP. + +## Cutover Checklist + +1. Keep Traefik serving production HTTP/HTTPS. +2. Add Envoy on the shared IP for TCP `80` and `443`, if your LB supports that. +3. Keep Gerbil UDP on the same IP and do not expose `internal-api` publicly. +4. Scope the controller with `CONFIG_RECONCILE_ONLY=`. +5. Validate one protected hostname and the Pangolin dashboard. +6. Move DNS/traffic policy from Traefik to Envoy for HTTP/HTTPS. +7. Remove Traefik's public Service or stop exposing ports `80` and `443` from + Traefik. +8. Remove `CONFIG_RECONCILE_ONLY`. + +If the load balancer cannot share one IP across Services, stop here and use the +separate-IP guide until you are ready for a full public-IP cutover. diff --git a/src/config.rs b/src/config.rs index 2c76f99..c3f8f50 100644 --- a/src/config.rs +++ b/src/config.rs @@ -50,10 +50,228 @@ pub struct Config { /// implementations that watch the ListenerSet directly. pub listenerset_annotations: BTreeMap, + /// Optional Envoy Gateway external auth wiring for pangolin's badger plugin. + pub badger_ext_auth: Option, + + /// Optional static routes for serving the Pangolin dashboard through the same + /// ListenerSet as managed resources. + pub pangolin_dashboard: Option, + + /// Optional UDP routing for Gerbil's WireGuard-facing ports. + pub gerbil_udp: Option, + + /// Optional allow-list used for migration testing. Empty means reconcile all + /// desired objects. + pub reconcile_scope: ReconcileScope, + pub read_only: bool, pub log_traefik_config: bool, } +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct BadgerExtAuthConfig { + pub backend_name: String, + pub backend_namespace: Option, + pub backend_port: i32, + pub path: Option, + pub headers_to_ext_auth: Vec, + pub headers_to_backend: Vec, + pub fail_open: bool, +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct PangolinDashboardConfig { + pub hostname: String, + pub service_name: String, + pub service_namespace: Option, + pub api_port: i32, + pub next_port: i32, + pub redirect_http_to_https: bool, +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct GerbilUdpConfig { + pub service_name: String, + pub service_namespace: Option, + pub ports: Vec, +} + +#[derive(Debug, Clone, Default, PartialEq, Eq)] +pub struct ReconcileScope { + selectors: Vec, +} + +#[derive(Debug, Clone, PartialEq, Eq)] +enum ReconcileSelector { + Object { + kind: Option, + name: String, + }, + Hostname(String), + ObjectNameOrHostname(String), +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)] +pub enum ReconcileKind { + HttpRoute, + ListenerSet, + Service, + EndpointSlice, + Backend, + SecurityPolicy, + UDPRoute, +} + +impl ReconcileKind { + pub fn parse(s: &str) -> Result { + match normalize_kind(s).as_str() { + "httproute" | "httproutes" | "hr" => Ok(Self::HttpRoute), + "listenerset" | "listenersets" | "ls" => Ok(Self::ListenerSet), + "service" | "services" | "svc" => Ok(Self::Service), + "endpointslice" | "endpointslices" | "eps" => Ok(Self::EndpointSlice), + "backend" | "backends" | "envoybackend" | "envoybackends" | "be" => Ok(Self::Backend), + "securitypolicy" | "securitypolicies" | "sp" => Ok(Self::SecurityPolicy), + "udproute" | "udproutes" | "udp" => Ok(Self::UDPRoute), + other => bail!( + "invalid reconcile object kind {other:?}; expected HTTPRoute, ListenerSet, \ + Service, EndpointSlice, Backend, SecurityPolicy, or UDPRoute" + ), + } + } + + pub fn as_str(self) -> &'static str { + match self { + Self::HttpRoute => "HTTPRoute", + Self::ListenerSet => "ListenerSet", + Self::Service => "Service", + Self::EndpointSlice => "EndpointSlice", + Self::Backend => "Backend", + Self::SecurityPolicy => "SecurityPolicy", + Self::UDPRoute => "UDPRoute", + } + } +} + +impl ReconcileScope { + pub fn parse(raw: &str) -> Result { + let mut selectors = Vec::new(); + for entry in raw.split(',') { + let entry = entry.trim(); + if entry.is_empty() { + continue; + } + + let selector = if let Some((kind, name)) = entry.split_once('/') { + let kind = kind.trim(); + let name = name.trim(); + if name.is_empty() { + bail!("CONFIG_RECONCILE_ONLY entry {entry:?} has an empty selector value"); + } + if is_hostname_selector_kind(kind) { + ReconcileSelector::Hostname(normalize_hostname_selector(name)) + } else { + ReconcileSelector::Object { + kind: Some(ReconcileKind::parse(kind)?), + name: name.to_string(), + } + } + } else if let Some((kind, name)) = entry.split_once(':') { + let kind = kind.trim(); + let name = name.trim(); + if name.is_empty() { + bail!("CONFIG_RECONCILE_ONLY entry {entry:?} has an empty selector value"); + } + if is_hostname_selector_kind(kind) { + ReconcileSelector::Hostname(normalize_hostname_selector(name)) + } else { + ReconcileSelector::Object { + kind: Some(ReconcileKind::parse(kind)?), + name: name.to_string(), + } + } + } else { + ReconcileSelector::ObjectNameOrHostname(entry.to_string()) + }; + + selectors.push(selector); + } + Ok(Self { selectors }) + } + + pub fn all() -> Self { + Self::default() + } + + pub fn is_all(&self) -> bool { + self.selectors.is_empty() + } + + pub fn includes(&self, kind: ReconcileKind, name: &str) -> bool { + self.is_all() + || self.selectors.iter().any(|selector| { + matches!( + selector, + ReconcileSelector::Object { + kind: selected, + name: selected_name, + } if selected.is_none_or(|selected| selected == kind) && selected_name == name + ) || matches!( + selector, + ReconcileSelector::ObjectNameOrHostname(selected_name) + if selected_name == name + ) + }) + } + + pub fn affects_kind(&self, kind: ReconcileKind) -> bool { + self.is_all() + || self.selectors.iter().any(|selector| match selector { + ReconcileSelector::Object { kind: selected, .. } => { + selected.is_none_or(|selected| selected == kind) + } + ReconcileSelector::ObjectNameOrHostname(_) => true, + ReconcileSelector::Hostname(_) => false, + }) + } + + pub fn hostname_candidates(&self) -> Vec { + self.selectors + .iter() + .filter_map(|selector| match selector { + ReconcileSelector::Hostname(hostname) => { + Some(normalize_hostname_selector(hostname)) + } + ReconcileSelector::ObjectNameOrHostname(hostname) => { + looks_like_hostname_selector(hostname) + .then(|| normalize_hostname_selector(hostname)) + } + ReconcileSelector::Object { .. } => None, + }) + .collect() + } + + pub fn with_expanded_objects(&self, objects: I) -> Self + where + I: IntoIterator, + { + let mut selectors: Vec<_> = self + .selectors + .iter() + .filter(|selector| !matches!(selector, ReconcileSelector::Hostname(_))) + .cloned() + .collect(); + selectors.extend( + objects + .into_iter() + .map(|(kind, name)| ReconcileSelector::Object { + kind: Some(kind), + name, + }), + ); + Self { selectors } + } +} + /// Backend object kind used for IP / FQDN pangolin targets. /// /// Cluster-DNS pass-through (`..svc[.cluster.local]`) is independent of @@ -100,6 +318,15 @@ impl Config { ); } + let namespace = optional_env("CONFIG_NAMESPACE").unwrap_or_else(|| "default".into()); + let badger_ext_auth = parse_badger_ext_auth(&namespace)?; + let pangolin_dashboard = parse_pangolin_dashboard(&namespace)?; + let gerbil_udp = parse_gerbil_udp(&namespace)?; + let reconcile_scope = optional_env("CONFIG_RECONCILE_ONLY") + .map(|raw| ReconcileScope::parse(&raw)) + .transpose()? + .unwrap_or_else(ReconcileScope::all); + Ok(Self { pangolin_endpoint, auth_header: optional_env("CONFIG_AUTH_HEADER"), @@ -110,7 +337,7 @@ impl Config { tls_skip_verify, ca_file: optional_env("CONFIG_CA_FILE"), - namespace: optional_env("CONFIG_NAMESPACE").unwrap_or_else(|| "default".into()), + namespace, parent_gateway: required_env("CONFIG_PARENT_GATEWAY")?, parent_gateway_namespace: optional_env("CONFIG_PARENT_GATEWAY_NAMESPACE"), listener_set_name: optional_env("CONFIG_LISTENERSET_NAME") @@ -144,6 +371,11 @@ impl Config { httproute_annotations: parse_kv_env("CONFIG_HTTPROUTE_ANNOTATIONS")?, listenerset_annotations: parse_kv_env("CONFIG_LISTENERSET_ANNOTATIONS")?, + badger_ext_auth, + pangolin_dashboard, + gerbil_udp, + reconcile_scope, + read_only: bool_env("CONFIG_READ_ONLY", false)?, log_traefik_config: bool_env("CONFIG_LOG_TRAEFIK_CONFIG", false)?, }) @@ -209,6 +441,133 @@ fn i32_env(key: &str, default: i32) -> Result { } } +fn parse_badger_ext_auth(namespace: &str) -> Result> { + if !bool_env("CONFIG_BADGER_EXT_AUTH", false)? { + return Ok(None); + } + + Ok(Some(BadgerExtAuthConfig { + backend_name: optional_env("CONFIG_BADGER_EXT_AUTH_BACKEND_NAME") + .unwrap_or_else(|| "pangolin-badger-ext-authz".into()), + backend_namespace: optional_env("CONFIG_BADGER_EXT_AUTH_BACKEND_NAMESPACE") + .or_else(|| Some(namespace.to_string())), + backend_port: i32_env("CONFIG_BADGER_EXT_AUTH_BACKEND_PORT", 9002)?, + path: optional_env("CONFIG_BADGER_EXT_AUTH_PATH"), + headers_to_ext_auth: csv_env( + "CONFIG_BADGER_EXT_AUTH_HEADERS_TO_EXTAUTH", + &[ + "authorization", + "cookie", + "x-forwarded-for", + "x-forwarded-host", + "x-forwarded-proto", + "x-real-ip", + "p-access-token-id", + "p-access-token", + ], + )?, + headers_to_backend: csv_env( + "CONFIG_BADGER_EXT_AUTH_HEADERS_TO_BACKEND", + &["remote-user", "remote-email", "remote-name", "remote-role"], + )?, + fail_open: bool_env("CONFIG_BADGER_EXT_AUTH_FAIL_OPEN", false)?, + })) +} + +fn parse_pangolin_dashboard(namespace: &str) -> Result> { + let Some(hostname) = optional_env("CONFIG_PANGOLIN_DASHBOARD_HOST") else { + return Ok(None); + }; + + Ok(Some(PangolinDashboardConfig { + hostname, + service_name: optional_env("CONFIG_PANGOLIN_SERVICE_NAME") + .unwrap_or_else(|| "pangolin".into()), + service_namespace: optional_env("CONFIG_PANGOLIN_SERVICE_NAMESPACE") + .or_else(|| Some(namespace.to_string())), + api_port: i32_env("CONFIG_PANGOLIN_API_PORT", 3000)?, + next_port: i32_env("CONFIG_PANGOLIN_NEXT_PORT", 3002)?, + redirect_http_to_https: bool_env("CONFIG_PANGOLIN_REDIRECT_HTTP_TO_HTTPS", true)?, + })) +} + +fn parse_gerbil_udp(namespace: &str) -> Result> { + if !bool_env("CONFIG_GERBIL_UDP_ROUTE", false)? { + return Ok(None); + } + + Ok(Some(GerbilUdpConfig { + service_name: optional_env("CONFIG_GERBIL_SERVICE_NAME").unwrap_or_else(|| "gerbil".into()), + service_namespace: optional_env("CONFIG_GERBIL_SERVICE_NAMESPACE") + .or_else(|| Some(namespace.to_string())), + ports: i32_csv_env("CONFIG_GERBIL_UDP_PORTS", &[51820, 21820])?, + })) +} + +fn csv_env(key: &str, default: &[&str]) -> Result> { + match optional_env(key) { + None => Ok(default.iter().map(|s| (*s).to_string()).collect()), + Some(raw) => { + let values = raw + .split(',') + .map(str::trim) + .filter(|s| !s.is_empty()) + .map(ToString::to_string) + .collect::>(); + if values.is_empty() { + bail!("{key} must contain at least one value when set"); + } + Ok(values) + } + } +} + +fn i32_csv_env(key: &str, default: &[i32]) -> Result> { + match optional_env(key) { + None => Ok(default.to_vec()), + Some(raw) => { + let mut out = Vec::new(); + for entry in raw.split(',') { + let entry = entry.trim(); + if entry.is_empty() { + continue; + } + out.push( + entry + .parse() + .with_context(|| format!("invalid i32 in {key}: {entry:?}"))?, + ); + } + if out.is_empty() { + bail!("{key} must contain at least one port when set"); + } + Ok(out) + } + } +} + +fn normalize_kind(s: &str) -> String { + s.chars() + .filter(|c| *c != '-' && *c != '_' && !c.is_whitespace()) + .collect::() + .to_ascii_lowercase() +} + +fn is_hostname_selector_kind(kind: &str) -> bool { + matches!( + normalize_kind(kind).as_str(), + "host" | "hosts" | "hostname" | "hostnames" + ) +} + +fn normalize_hostname_selector(hostname: &str) -> String { + hostname.trim().trim_end_matches('.').to_ascii_lowercase() +} + +fn looks_like_hostname_selector(hostname: &str) -> bool { + hostname.contains('.') +} + /// Parse a `key=value,key=value,...` list. Empty entries are skipped. fn parse_kv_env(key: &str) -> Result> { let Some(raw) = optional_env(key) else { diff --git a/src/envoy_gateway.rs b/src/envoy_gateway.rs index fe23d39..0fb99b0 100644 --- a/src/envoy_gateway.rs +++ b/src/envoy_gateway.rs @@ -2,8 +2,8 @@ //! //! Envoy Gateway publishes its CRDs at `gateway.envoyproxy.io/v1alpha1`. The //! `gateway-api` crate doesn't include these (it tracks upstream Gateway API -//! only), so we define just enough of the `Backend` CRD to emit one when the -//! controller is configured for the `envoy-backend` strategy. +//! only), so we define just enough of the `Backend` and `SecurityPolicy` CRDs +//! for the controller features we emit. //! //! Reference: . @@ -46,3 +46,77 @@ pub struct BackendFqdn { pub hostname: String, pub port: i32, } + +/// `gateway.envoyproxy.io/v1alpha1/SecurityPolicy` — used to attach Envoy +/// external authorization to HTTPRoutes that carry pangolin's badger middleware. +#[derive(CustomResource, Serialize, Deserialize, Clone, Debug, JsonSchema, Default, PartialEq)] +#[kube( + group = "gateway.envoyproxy.io", + version = "v1alpha1", + kind = "SecurityPolicy", + plural = "securitypolicies", + namespaced +)] +pub struct SecurityPolicySpec { + #[serde( + default, + skip_serializing_if = "Option::is_none", + rename = "targetRefs" + )] + pub target_refs: Option>, + #[serde(default, skip_serializing_if = "Option::is_none", rename = "extAuth")] + pub ext_auth: Option, +} + +#[derive(Serialize, Deserialize, Clone, Debug, JsonSchema, Default, PartialEq)] +pub struct SecurityPolicyTargetRef { + pub group: String, + pub kind: String, + pub name: String, + #[serde( + default, + skip_serializing_if = "Option::is_none", + rename = "sectionName" + )] + pub section_name: Option, +} + +#[derive(Serialize, Deserialize, Clone, Debug, JsonSchema, Default, PartialEq)] +pub struct SecurityPolicyExtAuth { + #[serde(default, skip_serializing_if = "Option::is_none")] + pub http: Option, + #[serde( + default, + skip_serializing_if = "Option::is_none", + rename = "headersToExtAuth" + )] + pub headers_to_ext_auth: Option>, + #[serde(default, skip_serializing_if = "Option::is_none", rename = "failOpen")] + pub fail_open: Option, +} + +#[derive(Serialize, Deserialize, Clone, Debug, JsonSchema, Default, PartialEq)] +pub struct HttpExtAuthService { + #[serde(rename = "backendRefs")] + pub backend_refs: Vec, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub path: Option, + #[serde( + default, + skip_serializing_if = "Option::is_none", + rename = "headersToBackend" + )] + pub headers_to_backend: Option>, +} + +#[derive(Serialize, Deserialize, Clone, Debug, JsonSchema, Default, PartialEq)] +pub struct SecurityPolicyBackendRef { + #[serde(default, skip_serializing_if = "Option::is_none")] + pub group: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub kind: Option, + pub name: String, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub namespace: Option, + pub port: i32, +} diff --git a/src/gc.rs b/src/gc.rs index d501aab..51481af 100644 --- a/src/gc.rs +++ b/src/gc.rs @@ -10,12 +10,22 @@ use serde::Serialize; use serde::de::DeserializeOwned; use tracing::{info, warn}; -use crate::config::Config; +use crate::config::{Config, ReconcileKind, ReconcileScope}; -pub async fn sweep(api: &Api, cfg: &Config, desired_names: &BTreeSet) -> Result<()> +pub async fn sweep( + api: &Api, + cfg: &Config, + scope: &ReconcileScope, + desired_names: &BTreeSet, +) -> Result<()> where T: Resource + Clone + Serialize + DeserializeOwned + std::fmt::Debug, { + let kind = ReconcileKind::parse(&T::kind(&()))?; + if !scope.affects_kind(kind) { + return Ok(()); + } + let lp = ListParams::default().labels(&cfg.managed_selector()); let existing = api .list(&lp) @@ -29,6 +39,9 @@ where if desired_names.contains(&name) { continue; } + if !scope.includes(kind, &name) { + continue; + } if cfg.read_only { info!(kind = %T::kind(&()), name = %name, "read-only mode: would delete orphan"); continue; diff --git a/src/pangolin/types.rs b/src/pangolin/types.rs index 6b22d6c..07655e1 100644 --- a/src/pangolin/types.rs +++ b/src/pangolin/types.rs @@ -22,7 +22,7 @@ pub struct TraefikDynamicConfig { /// Opaque TCP block. Reserved for future TCPRoute support. #[serde(default)] pub tcp: Option, - /// Opaque UDP block. Reserved for future UDPRoute support. + /// Opaque UDP block. Reserved for future dynamic UDPRoute support. #[serde(default)] pub udp: Option, } diff --git a/src/reconcile.rs b/src/reconcile.rs index afda220..56bd736 100644 --- a/src/reconcile.rs +++ b/src/reconcile.rs @@ -3,18 +3,19 @@ use std::collections::BTreeSet; use std::time::Duration; -use anyhow::Result; +use anyhow::{Context, Result}; use gateway_api::apis::experimental::httproutes::HTTPRoute; use gateway_api::apis::experimental::listenersets::ListenerSet; +use gateway_api::apis::experimental::udproutes::UDPRoute; use k8s_openapi::api::core::v1::Service; use k8s_openapi::api::discovery::v1::EndpointSlice; -use kube::Api; +use kube::{Api, Resource}; use tokio_util::sync::CancellationToken; use tracing::{error, info, warn}; use crate::apply::ssa_apply; -use crate::config::{BackendKind, Config}; -use crate::envoy_gateway::Backend as EnvoyBackend; +use crate::config::{BackendKind, Config, ReconcileKind, ReconcileScope}; +use crate::envoy_gateway::{Backend as EnvoyBackend, SecurityPolicy}; use crate::gc; use crate::pangolin::{Client as PangClient, FetchOutcome}; use crate::transform::{Desired, build_desired}; @@ -96,29 +97,73 @@ pub async fn run_loop( } async fn reconcile_once(cfg: &Config, kube_client: &kube::Client, desired: &Desired) -> Result<()> { + let reconcile_scope = expand_reconcile_scope(cfg, desired); let ns = cfg.namespace.as_str(); let route_api: Api = Api::namespaced(kube_client.clone(), ns); let ls_api: Api = Api::namespaced(kube_client.clone(), ns); let svc_api: Api = Api::namespaced(kube_client.clone(), ns); let eps_api: Api = Api::namespaced(kube_client.clone(), ns); let be_api: Api = Api::namespaced(kube_client.clone(), ns); + let sp_api: Api = Api::namespaced(kube_client.clone(), ns); + let udp_api: Api = Api::namespaced(kube_client.clone(), ns); // Apply backends first so HTTPRoute backendRefs resolve immediately. for svc in desired.services.values() { - ssa_apply(&svc_api, cfg, svc).await?; + ssa_apply_scoped(&svc_api, cfg, &reconcile_scope, ReconcileKind::Service, svc).await?; } for eps in desired.endpoint_slices.values() { - ssa_apply(&eps_api, cfg, eps).await?; + ssa_apply_scoped( + &eps_api, + cfg, + &reconcile_scope, + ReconcileKind::EndpointSlice, + eps, + ) + .await?; } for be in desired.envoy_backends.values() { - ssa_apply(&be_api, cfg, be).await?; + ssa_apply_scoped(&be_api, cfg, &reconcile_scope, ReconcileKind::Backend, be).await?; } // Then listener set so the parent for routes exists. for ls in desired.listener_sets.values() { - ssa_apply(&ls_api, cfg, ls).await?; + ssa_apply_scoped( + &ls_api, + cfg, + &reconcile_scope, + ReconcileKind::ListenerSet, + ls, + ) + .await?; + } + for udp in desired.udp_routes.values() { + ssa_apply_scoped( + &udp_api, + cfg, + &reconcile_scope, + ReconcileKind::UDPRoute, + udp, + ) + .await?; } for route in desired.http_routes.values() { - ssa_apply(&route_api, cfg, route).await?; + ssa_apply_scoped( + &route_api, + cfg, + &reconcile_scope, + ReconcileKind::HttpRoute, + route, + ) + .await?; + } + for policy in desired.security_policies.values() { + ssa_apply_scoped( + &sp_api, + cfg, + &reconcile_scope, + ReconcileKind::SecurityPolicy, + policy, + ) + .await?; } // GC anything we own that's no longer wanted. @@ -127,29 +172,202 @@ async fn reconcile_once(cfg: &Config, kube_client: &kube::Client, desired: &Desi let svc_names: BTreeSet = desired.services.keys().cloned().collect(); let eps_names: BTreeSet = desired.endpoint_slices.keys().cloned().collect(); let be_names: BTreeSet = desired.envoy_backends.keys().cloned().collect(); + let sp_names: BTreeSet = desired.security_policies.keys().cloned().collect(); + let udp_names: BTreeSet = desired.udp_routes.keys().cloned().collect(); - if let Err(e) = gc::sweep(&route_api, cfg, &route_names).await { + if let Err(e) = gc::sweep(&route_api, cfg, &reconcile_scope, &route_names).await { warn!(error = ?e, "GC HTTPRoute failed"); } - if let Err(e) = gc::sweep(&ls_api, cfg, &ls_names).await { + if let Err(e) = gc::sweep(&ls_api, cfg, &reconcile_scope, &ls_names).await { warn!(error = ?e, "GC ListenerSet failed"); } - if let Err(e) = gc::sweep(&eps_api, cfg, &eps_names).await { + if let Err(e) = gc::sweep(&eps_api, cfg, &reconcile_scope, &eps_names).await { warn!(error = ?e, "GC EndpointSlice failed"); } - if let Err(e) = gc::sweep(&svc_api, cfg, &svc_names).await { + if let Err(e) = gc::sweep(&svc_api, cfg, &reconcile_scope, &svc_names).await { warn!(error = ?e, "GC Service failed"); } // Only sweep Envoy Backends when the controller is in that mode; otherwise // the Backend CRD may not be installed and the list call would 404. if cfg.backend_kind == BackendKind::EnvoyBackend - && let Err(e) = gc::sweep(&be_api, cfg, &be_names).await + && let Err(e) = gc::sweep(&be_api, cfg, &reconcile_scope, &be_names).await { warn!(error = ?e, "GC Envoy Backend failed"); } + if cfg.badger_ext_auth.is_some() + && let Err(e) = gc::sweep(&sp_api, cfg, &reconcile_scope, &sp_names).await + { + warn!(error = ?e, "GC SecurityPolicy failed"); + } + if cfg.gerbil_udp.is_some() + && let Err(e) = gc::sweep(&udp_api, cfg, &reconcile_scope, &udp_names).await + { + warn!(error = ?e, "GC UDPRoute failed"); + } Ok(()) } +fn expand_reconcile_scope(cfg: &Config, desired: &Desired) -> ReconcileScope { + if cfg.reconcile_scope.is_all() { + return cfg.reconcile_scope.clone(); + } + + let hostname_selectors = cfg.reconcile_scope.hostname_candidates(); + if hostname_selectors.is_empty() { + return cfg.reconcile_scope.clone(); + } + + let mut expanded = BTreeSet::new(); + let mut matched_any_hostname = false; + for (route_name, route) in &desired.http_routes { + if !route_matches_hostname(route, &hostname_selectors) { + continue; + } + matched_any_hostname = true; + expanded.insert((ReconcileKind::HttpRoute, route_name.clone())); + expand_route_dependencies(route_name, route, desired, &mut expanded); + } + + if matched_any_hostname { + expanded.extend( + desired + .listener_sets + .keys() + .cloned() + .map(|name| (ReconcileKind::ListenerSet, name)), + ); + } else { + warn!( + selectors = ?hostname_selectors, + "CONFIG_RECONCILE_ONLY hostname selectors did not match any desired HTTPRoute" + ); + } + + cfg.reconcile_scope.with_expanded_objects(expanded) +} + +fn route_matches_hostname(route: &HTTPRoute, hostname_selectors: &[String]) -> bool { + let Some(hostnames) = &route.spec.hostnames else { + return false; + }; + hostnames.iter().any(|route_hostname| { + hostname_selectors + .iter() + .any(|selector| hostname_matches(selector, route_hostname)) + }) +} + +fn hostname_matches(selector: &str, route_hostname: &str) -> bool { + let selector = normalize_hostname(selector); + let route_hostname = normalize_hostname(route_hostname); + selector == route_hostname + || route_hostname.strip_prefix("*.").is_some_and(|suffix| { + selector.len() > suffix.len() + && selector.ends_with(suffix) + && selector.as_bytes()[selector.len() - suffix.len() - 1] == b'.' + }) +} + +fn normalize_hostname(hostname: &str) -> String { + hostname.trim().trim_end_matches('.').to_ascii_lowercase() +} + +fn expand_route_dependencies( + route_name: &str, + route: &HTTPRoute, + desired: &Desired, + expanded: &mut BTreeSet<(ReconcileKind, String)>, +) { + for policy_name in security_policies_for_route(route_name, desired) { + expanded.insert((ReconcileKind::SecurityPolicy, policy_name)); + } + + let Some(rules) = &route.spec.rules else { + return; + }; + for rule in rules { + let Some(backend_refs) = &rule.backend_refs else { + continue; + }; + for backend_ref in backend_refs { + let name = backend_ref.name.clone(); + let kind = backend_ref.kind.as_deref().unwrap_or("Service"); + let group = backend_ref.group.as_deref().unwrap_or(""); + if group == "gateway.envoyproxy.io" + && kind == "Backend" + && desired.envoy_backends.contains_key(&name) + { + expanded.insert((ReconcileKind::Backend, name)); + } else if group.is_empty() && kind == "Service" && desired.services.contains_key(&name) + { + expanded.insert((ReconcileKind::Service, name.clone())); + for endpoint_slice_name in endpoint_slices_for_service(&name, desired) { + expanded.insert((ReconcileKind::EndpointSlice, endpoint_slice_name)); + } + } + } + } +} + +fn security_policies_for_route(route_name: &str, desired: &Desired) -> Vec { + desired + .security_policies + .iter() + .filter_map(|(policy_name, policy)| { + let matches = policy.spec.target_refs.as_ref().is_some_and(|target_refs| { + target_refs.iter().any(|target_ref| { + target_ref.group == "gateway.networking.k8s.io" + && target_ref.kind == "HTTPRoute" + && target_ref.name == route_name + }) + }); + matches.then(|| policy_name.clone()) + }) + .collect() +} + +fn endpoint_slices_for_service(service_name: &str, desired: &Desired) -> Vec { + desired + .endpoint_slices + .iter() + .filter_map(|(slice_name, slice)| { + let matches = slice + .metadata + .labels + .as_ref() + .and_then(|labels| labels.get("kubernetes.io/service-name")) + .is_some_and(|name| name == service_name); + matches.then(|| slice_name.clone()) + }) + .collect() +} + +async fn ssa_apply_scoped( + api: &Api, + cfg: &Config, + scope: &ReconcileScope, + kind: ReconcileKind, + obj: &T, +) -> Result<()> +where + T: Resource + + Clone + + serde::Serialize + + serde::de::DeserializeOwned + + std::fmt::Debug, +{ + let name = obj + .meta() + .name + .as_deref() + .context("object has no metadata.name")?; + if !scope.includes(kind, name) { + info!(kind = %kind.as_str(), name = %name, "reconcile scope: skipping apply"); + return Ok(()); + } + ssa_apply(api, cfg, obj).await +} + /// Sleep for `dur`, returning true if shutdown was requested before the timer expired. async fn wait_or_shutdown(dur: Duration, shutdown: &CancellationToken) -> bool { tokio::select! { @@ -157,3 +375,104 @@ async fn wait_or_shutdown(dur: Duration, shutdown: &CancellationToken) -> bool { _ = shutdown.cancelled() => true, } } + +#[cfg(test)] +mod tests { + use std::collections::BTreeMap; + + use serde_json::json; + use url::Url; + + use super::*; + use crate::config::{BadgerExtAuthConfig, PangolinDashboardConfig}; + use crate::pangolin::TraefikDynamicConfig; + + fn test_config(reconcile_scope: ReconcileScope) -> Config { + Config { + pangolin_endpoint: Url::parse("http://pangolin.local/api/v1/traefik-config").unwrap(), + auth_header: None, + fetch_timeout: Duration::from_secs(10), + poll_interval: Duration::from_secs(30), + max_backoff: Duration::from_secs(60), + max_response_body_bytes: 1 << 20, + tls_skip_verify: false, + ca_file: None, + namespace: "gateway".into(), + parent_gateway: "eg".into(), + parent_gateway_namespace: Some("gateway".into()), + listener_set_name: "pangolin".into(), + http_port: 80, + https_port: 443, + enable_https_listeners: true, + backend_kind: BackendKind::Service, + tls_secret_template: Some("{hostname-dashed}-tls".into()), + tls_secret_namespace: None, + field_manager: "pangolin-gateway-controller".into(), + managed_label_key: "app.kubernetes.io/managed-by".into(), + managed_label_value: "pangolin-gateway-controller".into(), + instance_label_key: "pangolin.envisia.de/instance".into(), + instance_label_value: "default".into(), + managed_annotation_key: "pangolin.envisia.de/source".into(), + managed_annotation_value: "pangolin-gateway-controller".into(), + httproute_annotations: BTreeMap::new(), + listenerset_annotations: BTreeMap::new(), + badger_ext_auth: Some(BadgerExtAuthConfig { + backend_name: "badger-shim".into(), + backend_namespace: Some("gateway".into()), + backend_port: 9002, + path: None, + headers_to_ext_auth: vec!["cookie".into()], + headers_to_backend: vec!["remote-user".into()], + fail_open: false, + }), + pangolin_dashboard: None::, + gerbil_udp: None, + reconcile_scope, + read_only: false, + log_traefik_config: false, + } + } + + #[test] + fn hostname_scope_expands_to_route_policy_backend_and_listener_set() { + let cfg = test_config(ReconcileScope::parse("protected.example.com").unwrap()); + let dyn_config: TraefikDynamicConfig = serde_json::from_value(json!({ + "http": { + "routers": { + "protected": { + "rule": "Host(`protected.example.com`)", + "service": "protected-service", + "middlewares": ["badger"] + }, + "other": { + "rule": "Host(`other.example.com`)", + "service": "other-service" + } + }, + "services": { + "protected-service": { + "loadBalancer": { "servers": [{"url": "http://10.0.0.7:8080"}] } + }, + "other-service": { + "loadBalancer": { "servers": [{"url": "http://10.0.0.8:8080"}] } + } + }, + "middlewares": { + "badger": { "plugin": { "badger": { "disableForwardAuth": true } } } + } + } + })) + .unwrap(); + + let desired = build_desired(&cfg, &dyn_config); + let scope = expand_reconcile_scope(&cfg, &desired); + + assert!(scope.includes(ReconcileKind::HttpRoute, "hr-protected")); + assert!(!scope.includes(ReconcileKind::HttpRoute, "hr-other")); + assert!(scope.includes(ReconcileKind::SecurityPolicy, "sp-protected")); + assert!(scope.includes(ReconcileKind::Service, "svc-protected-service")); + assert!(!scope.includes(ReconcileKind::Service, "svc-other-service")); + assert!(scope.includes(ReconcileKind::EndpointSlice, "eps-protected-service")); + assert!(scope.includes(ReconcileKind::ListenerSet, "pangolin")); + } +} diff --git a/src/transform/dashboard.rs b/src/transform/dashboard.rs new file mode 100644 index 0000000..d8ac4c5 --- /dev/null +++ b/src/transform/dashboard.rs @@ -0,0 +1,162 @@ +//! Optional static Gateway API routes for Pangolin's own dashboard/API. + +use gateway_api::apis::experimental::httproutes::{ + HTTPRoute, HttpRouteParentRefs, HttpRouteRules, HttpRouteRulesBackendRefs, + HttpRouteRulesFilters, HttpRouteRulesFiltersRequestRedirect, + HttpRouteRulesFiltersRequestRedirectScheme, HttpRouteRulesFiltersType, HttpRouteRulesMatches, + HttpRouteRulesMatchesHeaders, HttpRouteRulesMatchesHeadersType, HttpRouteRulesMatchesPath, + HttpRouteRulesMatchesPathType, HttpRouteSpec, +}; + +use crate::apply::{managed_metadata_with, owner_labels}; +use crate::config::{Config, PangolinDashboardConfig}; +use crate::transform::Desired; +use crate::transform::naming::{dns_label, prefixed_label}; +use crate::transform::route::RouteIndex; + +pub fn build_dashboard_routes(cfg: &Config, routes: &mut RouteIndex, desired: &mut Desired) { + let Some(dashboard) = cfg.pangolin_dashboard.as_ref() else { + return; + }; + + let tls_enabled = cfg.enable_https_listeners && cfg.tls_secret_template.is_some(); + routes.hostnames.insert(dashboard.hostname.clone()); + if tls_enabled { + routes.https_hosts.insert(dashboard.hostname.clone()); + } + + if dashboard.redirect_http_to_https && tls_enabled { + insert_route( + cfg, + desired, + "pangolin-dashboard-redirect", + dashboard, + Some(vec![HttpRouteRulesFilters { + r#type: HttpRouteRulesFiltersType::RequestRedirect, + request_redirect: Some(HttpRouteRulesFiltersRequestRedirect { + scheme: Some(HttpRouteRulesFiltersRequestRedirectScheme::Https), + ..Default::default() + }), + ..Default::default() + }]), + None, + None, + Some(dns_label(&format!("http-{}", dashboard.hostname))), + ); + } + + let backend_section = if tls_enabled { + Some(dns_label(&format!("https-{}", dashboard.hostname))) + } else { + Some(dns_label(&format!("http-{}", dashboard.hostname))) + }; + + insert_route( + cfg, + desired, + "pangolin-dashboard-ws", + dashboard, + None, + Some(path_and_header_match("/api/v1/ws", "upgrade", "websocket")), + Some(dashboard.api_port), + backend_section.clone(), + ); + + insert_route( + cfg, + desired, + "pangolin-dashboard-api", + dashboard, + None, + Some(path_match("/api/v1")), + Some(dashboard.api_port), + backend_section.clone(), + ); + + insert_route( + cfg, + desired, + "pangolin-dashboard-web", + dashboard, + None, + Some(path_match("/")), + Some(dashboard.next_port), + backend_section, + ); +} + +fn insert_route( + cfg: &Config, + desired: &mut Desired, + source_name: &str, + dashboard: &PangolinDashboardConfig, + filters: Option>, + matches: Option>, + backend_port: Option, + parent_section: Option, +) { + let route_name = prefixed_label("hr", source_name); + let labels = owner_labels(cfg, &route_name); + + let backend_refs = backend_port.map(|port| { + vec![HttpRouteRulesBackendRefs { + group: Some(String::new()), + kind: Some("Service".into()), + name: dashboard.service_name.clone(), + namespace: dashboard.service_namespace.clone(), + port: Some(port), + weight: Some(1), + ..Default::default() + }] + }); + + let route = HTTPRoute { + metadata: managed_metadata_with(cfg, &route_name, labels, &cfg.httproute_annotations), + spec: HttpRouteSpec { + parent_refs: Some(vec![HttpRouteParentRefs { + group: Some("gateway.networking.k8s.io".into()), + kind: Some("ListenerSet".into()), + name: cfg.listener_set_name.clone(), + namespace: Some(cfg.namespace.clone()), + section_name: parent_section, + ..Default::default() + }]), + hostnames: Some(vec![dashboard.hostname.clone()]), + rules: Some(vec![HttpRouteRules { + matches, + filters, + backend_refs, + ..Default::default() + }]), + ..Default::default() + }, + status: None, + }; + + desired.http_routes.insert(route_name, route); +} + +fn path_match(path: &str) -> Vec { + vec![HttpRouteRulesMatches { + path: Some(HttpRouteRulesMatchesPath { + r#type: Some(HttpRouteRulesMatchesPathType::PathPrefix), + value: Some(path.to_string()), + }), + ..Default::default() + }] +} + +fn path_and_header_match(path: &str, header: &str, value: &str) -> Vec { + vec![HttpRouteRulesMatches { + path: Some(HttpRouteRulesMatchesPath { + r#type: Some(HttpRouteRulesMatchesPathType::PathPrefix), + value: Some(path.to_string()), + }), + headers: Some(vec![HttpRouteRulesMatchesHeaders { + name: header.to_string(), + r#type: Some(HttpRouteRulesMatchesHeadersType::Exact), + value: value.to_string(), + }]), + ..Default::default() + }] +} diff --git a/src/transform/gerbil.rs b/src/transform/gerbil.rs new file mode 100644 index 0000000..6f67c07 --- /dev/null +++ b/src/transform/gerbil.rs @@ -0,0 +1,52 @@ +//! Optional Gateway API UDPRoutes for Gerbil's WireGuard-facing ports. + +use gateway_api::apis::experimental::udproutes::{ + UDPRoute, UdpRouteParentRefs, UdpRouteRules, UdpRouteRulesBackendRefs, UdpRouteSpec, +}; + +use crate::apply::{managed_metadata, owner_labels}; +use crate::config::Config; +use crate::transform::Desired; +use crate::transform::naming::prefixed_label; + +pub fn build_udp_routes(cfg: &Config, desired: &mut Desired) { + let Some(gerbil) = cfg.gerbil_udp.as_ref() else { + return; + }; + + for port in &gerbil.ports { + let source_name = format!("gerbil-udp-{port}"); + let route_name = prefixed_label("udpr", &source_name); + let labels = owner_labels(cfg, &route_name); + let listener_name = prefixed_label("udp", &source_name); + + let route = UDPRoute { + metadata: managed_metadata(cfg, &route_name, labels), + spec: UdpRouteSpec { + parent_refs: Some(vec![UdpRouteParentRefs { + group: Some("gateway.networking.k8s.io".into()), + kind: Some("ListenerSet".into()), + name: cfg.listener_set_name.clone(), + namespace: Some(cfg.namespace.clone()), + section_name: Some(listener_name), + ..Default::default() + }]), + rules: vec![UdpRouteRules { + backend_refs: vec![UdpRouteRulesBackendRefs { + group: Some(String::new()), + kind: Some("Service".into()), + name: gerbil.service_name.clone(), + namespace: gerbil.service_namespace.clone(), + port: Some(*port), + weight: Some(1), + }], + name: Some("default".into()), + }], + ..Default::default() + }, + status: None, + }; + + desired.udp_routes.insert(route_name, route); + } +} diff --git a/src/transform/listener.rs b/src/transform/listener.rs index 4457ddc..939aad8 100644 --- a/src/transform/listener.rs +++ b/src/transform/listener.rs @@ -51,6 +51,22 @@ pub fn build_listener_set(cfg: &Config, routes: &RouteIndex, desired: &mut Desir } } + if let Some(gerbil) = cfg.gerbil_udp.as_ref() { + for port in &gerbil.ports { + listeners.push(ListenerSetListeners { + name: crate::transform::naming::prefixed_label( + "udp", + &format!("gerbil-udp-{port}"), + ), + hostname: None, + port: *port, + protocol: "UDP".into(), + tls: None, + allowed_routes: None, + }); + } + } + let name = dns_label(&cfg.listener_set_name); let labels = owner_labels(cfg, &name); diff --git a/src/transform/middleware.rs b/src/transform/middleware.rs index 7cf47e3..de64452 100644 --- a/src/transform/middleware.rs +++ b/src/transform/middleware.rs @@ -24,6 +24,7 @@ pub fn build_filters( router: &str, middleware_names: &[String], middlewares: &BTreeMap, + badger_handled_by_ext_auth: bool, ) -> Vec { let mut filters = Vec::new(); for name in middleware_names { @@ -31,14 +32,29 @@ pub fn build_filters( warn!(router, middleware = %name, "router references missing middleware"); continue; }; - if let Some(filter) = translate(router, name, mw) { + if let Some(filter) = translate(router, name, mw, badger_handled_by_ext_auth) { filters.push(filter); } } filters } -fn translate(router: &str, name: &str, mw: &Middleware) -> Option { +pub fn references_badger( + middleware_names: &[String], + middlewares: &BTreeMap, +) -> bool { + middleware_names + .iter() + .filter_map(|name| middlewares.get(name)) + .any(is_badger_plugin) +} + +fn translate( + router: &str, + name: &str, + mw: &Middleware, + badger_handled_by_ext_auth: bool, +) -> Option { let obj = mw.as_object()?; // Pangolin emits one top-level key per middleware kind. let (kind, body) = obj.iter().next()?; @@ -53,7 +69,9 @@ fn translate(router: &str, name: &str, mw: &Middleware) -> Option translate_strip_prefix(body), "plugin" => { - warn!(router, middleware = %name, "plugin middlewares (e.g. badger) must be configured via Envoy Gateway policies; skipping"); + if !(badger_handled_by_ext_auth && plugin_contains_badger(body)) { + warn!(router, middleware = %name, "plugin middlewares (e.g. badger) must be configured via Envoy Gateway policies; skipping"); + } None } other => { @@ -63,6 +81,17 @@ fn translate(router: &str, name: &str, mw: &Middleware) -> Option bool { + mw.as_object() + .and_then(|obj| obj.get("plugin")) + .is_some_and(plugin_contains_badger) +} + +fn plugin_contains_badger(body: &Value) -> bool { + body.as_object() + .is_some_and(|plugins| plugins.contains_key("badger")) +} + fn translate_redirect_scheme(body: &Value) -> Option { let scheme = body.get("scheme")?.as_str()?; let port = body diff --git a/src/transform/mod.rs b/src/transform/mod.rs index 5ca8c98..8acf9d6 100644 --- a/src/transform/mod.rs +++ b/src/transform/mod.rs @@ -1,6 +1,8 @@ //! Transform pangolin's Traefik dynamic config into a desired set of Gateway API objects. pub mod backend; +pub mod dashboard; +pub mod gerbil; pub mod listener; pub mod middleware; pub mod naming; @@ -14,10 +16,11 @@ use k8s_openapi::api::discovery::v1::EndpointSlice; use tracing::warn; use crate::config::Config; -use crate::envoy_gateway::Backend; +use crate::envoy_gateway::{Backend, SecurityPolicy}; use crate::pangolin::TraefikDynamicConfig; use gateway_api::apis::experimental::httproutes::HTTPRoute; use gateway_api::apis::experimental::listenersets::ListenerSet; +use gateway_api::apis::experimental::udproutes::UDPRoute; /// Everything the controller wants to exist in the cluster, keyed by resource name. /// Names are guaranteed DNS-label safe. @@ -29,6 +32,10 @@ pub struct Desired { pub endpoint_slices: BTreeMap, /// Populated only when `CONFIG_BACKEND_KIND=envoy-backend`. Empty otherwise. pub envoy_backends: BTreeMap, + /// Populated when badger ext_authz integration is enabled. + pub security_policies: BTreeMap, + /// Populated when Gerbil UDP routing is enabled. + pub udp_routes: BTreeMap, } pub fn build_desired(cfg: &Config, dyn_config: &TraefikDynamicConfig) -> Desired { @@ -38,13 +45,21 @@ pub fn build_desired(cfg: &Config, dyn_config: &TraefikDynamicConfig) -> Desired let backend_index = backend::build_backends(cfg, &dyn_config.http.services, &mut desired); // 2. Routes: pangolin routers -> HTTPRoute - let route_index = route::build_routes(cfg, dyn_config, &backend_index, &mut desired); + let mut route_index = route::build_routes(cfg, dyn_config, &backend_index, &mut desired); + + // 2b. Optional static routes that are not part of pangolin's dynamic + // resource config, but are useful when moving the control plane itself + // behind Envoy Gateway. + dashboard::build_dashboard_routes(cfg, &mut route_index, &mut desired); + gerbil::build_udp_routes(cfg, &mut desired); // 3. Listeners: aggregate unique hostnames into one ListenerSet listener::build_listener_set(cfg, &route_index, &mut desired); - if let (Some(_), Some(_)) = (&dyn_config.tcp, &dyn_config.udp) { - warn!("pangolin response contains tcp/udp blocks; this controller only handles HTTP"); + if dyn_config.tcp.is_some() || dyn_config.udp.is_some() { + warn!( + "pangolin response contains dynamic tcp/udp blocks; this controller only translates HTTP dynamic config" + ); } desired diff --git a/src/transform/route.rs b/src/transform/route.rs index 9b35f66..f632b15 100644 --- a/src/transform/route.rs +++ b/src/transform/route.rs @@ -9,8 +9,12 @@ use gateway_api::apis::experimental::httproutes::{ }; use tracing::warn; -use crate::apply::{managed_metadata_with, owner_labels}; +use crate::apply::{managed_metadata, managed_metadata_with, owner_labels}; use crate::config::Config; +use crate::envoy_gateway::{ + HttpExtAuthService, SecurityPolicy, SecurityPolicyBackendRef, SecurityPolicyExtAuth, + SecurityPolicySpec, SecurityPolicyTargetRef, +}; use crate::pangolin::TraefikDynamicConfig; use crate::transform::Desired; use crate::transform::backend::BackendIndex; @@ -58,10 +62,14 @@ pub fn build_routes( } }; + let uses_badger = + middleware::references_badger(&router.middlewares, &dyn_config.http.middlewares); + let badger_handled_by_ext_auth = cfg.badger_ext_auth.is_some() && uses_badger; let filters = middleware::build_filters( router_name, &router.middlewares, &dyn_config.http.middlewares, + badger_handled_by_ext_auth, ); let matches = parsed.path.as_ref().map(|p| { @@ -144,8 +152,54 @@ pub fn build_routes( } } + if badger_handled_by_ext_auth && !is_terminating { + let policy = build_badger_security_policy(cfg, router_name, &route_name); + let policy_name = policy.metadata.name.clone().unwrap_or_default(); + desired.security_policies.insert(policy_name, policy); + } + desired.http_routes.insert(route_name, route); } index } + +fn build_badger_security_policy( + cfg: &Config, + router_name: &str, + route_name: &str, +) -> SecurityPolicy { + let ext_auth = cfg + .badger_ext_auth + .as_ref() + .expect("badger ext auth checked by caller"); + let name = prefixed_label("sp", router_name); + let labels = owner_labels(cfg, &name); + + SecurityPolicy { + metadata: managed_metadata(cfg, &name, labels), + spec: SecurityPolicySpec { + target_refs: Some(vec![SecurityPolicyTargetRef { + group: "gateway.networking.k8s.io".into(), + kind: "HTTPRoute".into(), + name: route_name.to_string(), + section_name: None, + }]), + ext_auth: Some(SecurityPolicyExtAuth { + http: Some(HttpExtAuthService { + backend_refs: vec![SecurityPolicyBackendRef { + group: Some(String::new()), + kind: Some("Service".into()), + name: ext_auth.backend_name.clone(), + namespace: ext_auth.backend_namespace.clone(), + port: ext_auth.backend_port, + }], + path: ext_auth.path.clone(), + headers_to_backend: Some(ext_auth.headers_to_backend.clone()), + }), + headers_to_ext_auth: Some(ext_auth.headers_to_ext_auth.clone()), + fail_open: Some(ext_auth.fail_open), + }), + }, + } +} diff --git a/tests/cluster_integration.rs b/tests/cluster_integration.rs index 4c6db77..085dc26 100644 --- a/tests/cluster_integration.rs +++ b/tests/cluster_integration.rs @@ -33,7 +33,7 @@ use serde::de::DeserializeOwned; use tokio_util::sync::CancellationToken; use url::Url; -use pangolin_gateway_controller::config::{BackendKind, Config}; +use pangolin_gateway_controller::config::{BackendKind, Config, ReconcileScope}; use pangolin_gateway_controller::envoy_gateway::Backend as EnvoyBackend; use pangolin_gateway_controller::{pangolin, reconcile}; @@ -82,6 +82,11 @@ fn integration_config( httproute_annotations: BTreeMap::new(), listenerset_annotations: BTreeMap::new(), + badger_ext_auth: None, + pangolin_dashboard: None, + gerbil_udp: None, + reconcile_scope: ReconcileScope::all(), + read_only: false, log_traefik_config: false, } diff --git a/tests/config_smoke.rs b/tests/config_smoke.rs index ca88cc1..9f2180c 100644 --- a/tests/config_smoke.rs +++ b/tests/config_smoke.rs @@ -4,7 +4,7 @@ //! within this binary — `std::env::set_var` is process-global and racy with //! parallel tests. -use pangolin_gateway_controller::config::{BackendKind, Config}; +use pangolin_gateway_controller::config::{BackendKind, Config, ReconcileKind, ReconcileScope}; const CONFIG_ENV_KEYS: &[&str] = &[ "CONFIG_ENDPOINT", @@ -36,6 +36,25 @@ const CONFIG_ENV_KEYS: &[&str] = &[ "CONFIG_MANAGED_ANNOTATION_VALUE", "CONFIG_HTTPROUTE_ANNOTATIONS", "CONFIG_LISTENERSET_ANNOTATIONS", + "CONFIG_BADGER_EXT_AUTH", + "CONFIG_BADGER_EXT_AUTH_BACKEND_NAME", + "CONFIG_BADGER_EXT_AUTH_BACKEND_NAMESPACE", + "CONFIG_BADGER_EXT_AUTH_BACKEND_PORT", + "CONFIG_BADGER_EXT_AUTH_PATH", + "CONFIG_BADGER_EXT_AUTH_HEADERS_TO_EXTAUTH", + "CONFIG_BADGER_EXT_AUTH_HEADERS_TO_BACKEND", + "CONFIG_BADGER_EXT_AUTH_FAIL_OPEN", + "CONFIG_PANGOLIN_DASHBOARD_HOST", + "CONFIG_PANGOLIN_SERVICE_NAME", + "CONFIG_PANGOLIN_SERVICE_NAMESPACE", + "CONFIG_PANGOLIN_API_PORT", + "CONFIG_PANGOLIN_NEXT_PORT", + "CONFIG_PANGOLIN_REDIRECT_HTTP_TO_HTTPS", + "CONFIG_GERBIL_UDP_ROUTE", + "CONFIG_GERBIL_SERVICE_NAME", + "CONFIG_GERBIL_SERVICE_NAMESPACE", + "CONFIG_GERBIL_UDP_PORTS", + "CONFIG_RECONCILE_ONLY", "CONFIG_READ_ONLY", "CONFIG_LOG_TRAEFIK_CONFIG", ]; @@ -76,6 +95,33 @@ fn backend_kind_parses_known_spellings() { assert!(err.contains("CONFIG_BACKEND_KIND")); } +#[test] +fn reconcile_scope_parses_selectors() { + let scope = + ReconcileScope::parse("HTTPRoute/hr-a,SecurityPolicy/sp-a,Service:svc-a,shared-name") + .unwrap(); + assert!(scope.includes(ReconcileKind::HttpRoute, "hr-a")); + assert!(scope.includes(ReconcileKind::SecurityPolicy, "sp-a")); + assert!(scope.includes(ReconcileKind::Service, "svc-a")); + assert!(scope.includes(ReconcileKind::ListenerSet, "shared-name")); + assert!(!scope.includes(ReconcileKind::HttpRoute, "sp-a")); +} + +#[test] +fn reconcile_scope_treats_bare_names_as_hostname_candidates() { + let scope = ReconcileScope::parse("app.example.com,Hostname:admin.example.com").unwrap(); + + assert!(scope.includes(ReconcileKind::HttpRoute, "app.example.com")); + assert!(!scope.includes(ReconcileKind::HttpRoute, "admin.example.com")); + assert_eq!( + scope.hostname_candidates(), + vec![ + "app.example.com".to_string(), + "admin.example.com".to_string() + ] + ); +} + #[test] fn from_env_smoke() { // Step 1: missing CONFIG_ENDPOINT → required-env error. @@ -141,6 +187,10 @@ fn from_env_smoke() { assert!(!cfg.read_only); assert!(cfg.httproute_annotations.is_empty()); assert!(cfg.listenerset_annotations.is_empty()); + assert!(cfg.badger_ext_auth.is_none()); + assert!(cfg.pangolin_dashboard.is_none()); + assert!(cfg.gerbil_udp.is_none()); + assert!(cfg.reconcile_scope.is_all()); // Step 5: full env override exercises every parser branch. clear_all(); @@ -171,6 +221,34 @@ fn from_env_smoke() { "CONFIG_LISTENERSET_ANNOTATIONS", "cert-manager.io/cluster-issuer=letsencrypt-prod", ); + set("CONFIG_BADGER_EXT_AUTH", "true"); + set("CONFIG_BADGER_EXT_AUTH_BACKEND_NAME", "badger-shim"); + set("CONFIG_BADGER_EXT_AUTH_BACKEND_NAMESPACE", "auth"); + set("CONFIG_BADGER_EXT_AUTH_BACKEND_PORT", "9003"); + set("CONFIG_BADGER_EXT_AUTH_PATH", "/authorize"); + set( + "CONFIG_BADGER_EXT_AUTH_HEADERS_TO_EXTAUTH", + "cookie,authorization,x-forwarded-for", + ); + set( + "CONFIG_BADGER_EXT_AUTH_HEADERS_TO_BACKEND", + "remote-user,remote-email", + ); + set("CONFIG_BADGER_EXT_AUTH_FAIL_OPEN", "true"); + set("CONFIG_PANGOLIN_DASHBOARD_HOST", "pangolin.example.com"); + set("CONFIG_PANGOLIN_SERVICE_NAME", "pangolin-api"); + set("CONFIG_PANGOLIN_SERVICE_NAMESPACE", "pangolin-system"); + set("CONFIG_PANGOLIN_API_PORT", "3001"); + set("CONFIG_PANGOLIN_NEXT_PORT", "3002"); + set("CONFIG_PANGOLIN_REDIRECT_HTTP_TO_HTTPS", "false"); + set("CONFIG_GERBIL_UDP_ROUTE", "true"); + set("CONFIG_GERBIL_SERVICE_NAME", "gerbil-edge"); + set("CONFIG_GERBIL_SERVICE_NAMESPACE", "pangolin-system"); + set("CONFIG_GERBIL_UDP_PORTS", "51820,21820"); + set( + "CONFIG_RECONCILE_ONLY", + "HTTPRoute/hr-one,SecurityPolicy/sp-one", + ); set("CONFIG_READ_ONLY", "true"); let cfg = Config::from_env().expect("full override"); @@ -202,6 +280,30 @@ fn from_env_smoke() { Some("letsencrypt-prod"), ); assert_eq!(cfg.listenerset_annotations.len(), 1); + let ext_auth = cfg.badger_ext_auth.as_ref().expect("badger ext auth"); + assert_eq!(ext_auth.backend_name, "badger-shim"); + assert_eq!(ext_auth.backend_namespace.as_deref(), Some("auth")); + assert_eq!(ext_auth.backend_port, 9003); + assert_eq!(ext_auth.path.as_deref(), Some("/authorize")); + assert_eq!(ext_auth.headers_to_ext_auth.len(), 3); + assert_eq!(ext_auth.headers_to_backend.len(), 2); + assert!(ext_auth.fail_open); + let dashboard = cfg.pangolin_dashboard.as_ref().expect("dashboard"); + assert_eq!(dashboard.hostname, "pangolin.example.com"); + assert_eq!(dashboard.service_name, "pangolin-api"); + assert_eq!(dashboard.api_port, 3001); + assert!(!dashboard.redirect_http_to_https); + let gerbil = cfg.gerbil_udp.as_ref().expect("gerbil"); + assert_eq!(gerbil.service_name, "gerbil-edge"); + assert_eq!(gerbil.ports, vec![51820, 21820]); + assert!( + cfg.reconcile_scope + .includes(ReconcileKind::HttpRoute, "hr-one") + ); + assert!( + !cfg.reconcile_scope + .includes(ReconcileKind::HttpRoute, "sp-one") + ); assert!(cfg.read_only); // managed_selector is the contract GC relies on — verify the format. diff --git a/tests/e2e_pangolin.rs b/tests/e2e_pangolin.rs index 651c57b..6904a18 100644 --- a/tests/e2e_pangolin.rs +++ b/tests/e2e_pangolin.rs @@ -31,7 +31,7 @@ use serde::de::DeserializeOwned; use tokio_util::sync::CancellationToken; use url::Url; -use pangolin_gateway_controller::config::{BackendKind, Config}; +use pangolin_gateway_controller::config::{BackendKind, Config, ReconcileScope}; use pangolin_gateway_controller::{pangolin, reconcile}; fn e2e_config(endpoint: &str, namespace: &str, parent_gateway: &str) -> Config { @@ -68,6 +68,11 @@ fn e2e_config(endpoint: &str, namespace: &str, parent_gateway: &str) -> Config { httproute_annotations: BTreeMap::new(), listenerset_annotations: BTreeMap::new(), + badger_ext_auth: None, + pangolin_dashboard: None, + gerbil_udp: None, + reconcile_scope: ReconcileScope::all(), + read_only: false, log_traefik_config: true, } diff --git a/tests/pangolin_client_smoke.rs b/tests/pangolin_client_smoke.rs index 6c42c68..eaa4fa7 100644 --- a/tests/pangolin_client_smoke.rs +++ b/tests/pangolin_client_smoke.rs @@ -14,7 +14,7 @@ use tokio::net::TcpListener; use tokio::sync::Mutex; use url::Url; -use pangolin_gateway_controller::config::{BackendKind, Config}; +use pangolin_gateway_controller::config::{BackendKind, Config, ReconcileScope}; use pangolin_gateway_controller::pangolin::{Client, FetchOutcome}; #[derive(Clone)] @@ -125,6 +125,10 @@ fn config_pointing_at(endpoint: &str) -> Config { managed_annotation_value: "pangolin-gateway-controller".into(), httproute_annotations: BTreeMap::new(), listenerset_annotations: BTreeMap::new(), + badger_ext_auth: None, + pangolin_dashboard: None, + gerbil_udp: None, + reconcile_scope: ReconcileScope::all(), read_only: false, log_traefik_config: false, } diff --git a/tests/transform.rs b/tests/transform.rs index d6536aa..ba97602 100644 --- a/tests/transform.rs +++ b/tests/transform.rs @@ -11,7 +11,10 @@ use std::time::Duration; use serde_json::json; use url::Url; -use pangolin_gateway_controller::config::{BackendKind, Config}; +use pangolin_gateway_controller::config::{ + BackendKind, BadgerExtAuthConfig, Config, GerbilUdpConfig, PangolinDashboardConfig, + ReconcileScope, +}; use pangolin_gateway_controller::pangolin::TraefikDynamicConfig; use pangolin_gateway_controller::transform::build_desired; @@ -44,6 +47,10 @@ fn test_config() -> Config { managed_annotation_value: "pangolin-gateway-controller".into(), httproute_annotations: BTreeMap::new(), listenerset_annotations: BTreeMap::new(), + badger_ext_auth: None, + pangolin_dashboard: None, + gerbil_udp: None, + reconcile_scope: ReconcileScope::all(), read_only: false, log_traefik_config: false, } @@ -344,3 +351,170 @@ fn redirect_router_drops_backend_refs() { assert_eq!(filters.len(), 1); assert!(filters[0].request_redirect.is_some()); } + +#[test] +fn badger_ext_auth_emits_security_policy() { + let mut cfg = test_config(); + cfg.badger_ext_auth = Some(BadgerExtAuthConfig { + backend_name: "badger-shim".into(), + backend_namespace: Some("gateway".into()), + backend_port: 9002, + path: None, + headers_to_ext_auth: vec!["cookie".into(), "authorization".into()], + headers_to_backend: vec!["remote-user".into()], + fail_open: false, + }); + + let dyn_config: TraefikDynamicConfig = serde_json::from_value(json!({ + "http": { + "routers": { + "r1": { + "rule": "Host(`protected.example.com`)", + "service": "s1", + "middlewares": ["badger"] + } + }, + "services": { + "s1": { + "loadBalancer": { "servers": [{"url": "http://10.0.0.7:8080"}] } + } + }, + "middlewares": { + "badger": { "plugin": { "badger": { "disableForwardAuth": true } } } + } + } + })) + .unwrap(); + + let desired = build_desired(&cfg, &dyn_config); + + assert_eq!(desired.http_routes.len(), 1); + assert_eq!(desired.security_policies.len(), 1); + let route_name = desired.http_routes.keys().next().unwrap(); + let policy = desired.security_policies.values().next().unwrap(); + let target = &policy.spec.target_refs.as_ref().unwrap()[0]; + assert_eq!(target.kind, "HTTPRoute"); + assert_eq!(&target.name, route_name); + let ext_auth = policy.spec.ext_auth.as_ref().unwrap(); + assert_eq!( + ext_auth.headers_to_ext_auth.as_ref().unwrap(), + &vec!["cookie".to_string(), "authorization".to_string()] + ); + let http = ext_auth.http.as_ref().unwrap(); + assert_eq!(http.backend_refs[0].name, "badger-shim"); + assert_eq!(http.backend_refs[0].port, 9002); + assert_eq!( + http.headers_to_backend.as_ref().unwrap(), + &vec!["remote-user".to_string()] + ); +} + +#[test] +fn dashboard_routes_split_api_web_and_redirect() { + let mut cfg = test_config(); + cfg.pangolin_dashboard = Some(PangolinDashboardConfig { + hostname: "pangolin.example.com".into(), + service_name: "pangolin".into(), + service_namespace: Some("gateway".into()), + api_port: 3000, + next_port: 3002, + redirect_http_to_https: true, + }); + + let dyn_config: TraefikDynamicConfig = serde_json::from_value(json!({ + "http": { + "routers": {}, + "services": {} + } + })) + .unwrap(); + + let desired = build_desired(&cfg, &dyn_config); + + assert_eq!(desired.http_routes.len(), 4); + let listener_set = desired.listener_sets.values().next().unwrap(); + assert!( + listener_set + .spec + .listeners + .iter() + .any(|l| l.hostname.as_deref() == Some("pangolin.example.com") && l.protocol == "HTTPS") + ); + + let api = desired + .http_routes + .values() + .find(|r| r.metadata.name.as_deref().unwrap().contains("api")) + .expect("api route"); + let api_ref = &api.spec.rules.as_ref().unwrap()[0] + .backend_refs + .as_ref() + .unwrap()[0]; + assert_eq!(api_ref.name, "pangolin"); + assert_eq!(api_ref.port, Some(3000)); + + let web = desired + .http_routes + .values() + .find(|r| r.metadata.name.as_deref().unwrap().contains("web")) + .expect("web route"); + let web_ref = &web.spec.rules.as_ref().unwrap()[0] + .backend_refs + .as_ref() + .unwrap()[0]; + assert_eq!(web_ref.port, Some(3002)); + + let redirect = desired + .http_routes + .values() + .find(|r| r.metadata.name.as_deref().unwrap().contains("redirect")) + .expect("redirect route"); + assert!( + redirect.spec.rules.as_ref().unwrap()[0] + .filters + .as_ref() + .unwrap()[0] + .request_redirect + .is_some() + ); +} + +#[test] +fn gerbil_udp_routes_add_udp_listeners() { + let mut cfg = test_config(); + cfg.gerbil_udp = Some(GerbilUdpConfig { + service_name: "gerbil".into(), + service_namespace: Some("gateway".into()), + ports: vec![51820, 21820], + }); + + let dyn_config: TraefikDynamicConfig = serde_json::from_value(json!({ + "http": { + "routers": {}, + "services": {} + } + })) + .unwrap(); + + let desired = build_desired(&cfg, &dyn_config); + + assert_eq!(desired.udp_routes.len(), 2); + let listener_set = desired.listener_sets.values().next().unwrap(); + for port in [51820, 21820] { + assert!( + listener_set + .spec + .listeners + .iter() + .any(|l| l.port == port && l.protocol == "UDP"), + "missing UDP listener for {port}" + ); + assert!( + desired.udp_routes.values().any(|route| route.spec.rules[0] + .backend_refs + .iter() + .any(|backend| backend.name == "gerbil" && backend.port == Some(port))), + "missing UDPRoute backend for {port}" + ); + } +}