Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
102 changes: 100 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,10 @@ applies the additions/updates with [Server-Side Apply], and deletes orphans.
| `middlewares.addPrefix` | HTTPRoute filter `URLRewrite{path.ReplacePrefixMatch}` |
| `middlewares.replacePath` | HTTPRoute filter `URLRewrite{path.ReplaceFullPath}` |
| `middlewares.replacePathRegex` | not in core Gateway API – logged + skipped |
| `middlewares.plugin.badger` | pangolin's auth plugin – configure via Envoy Gateway `SecurityPolicy` instead |
| `tcp.*` / `udp.*` | not yet handled (planned: `TCPRoute` / `UDPRoute`) |
| `middlewares.plugin.badger` | when `CONFIG_BADGER_EXT_AUTH=true`: per-route Envoy Gateway `SecurityPolicy.extAuth`; otherwise logged + skipped |
| Pangolin dashboard host | optional static `HTTPRoute`s for Next.js (`:3002`) and API/WebSocket (`:3000`) when `CONFIG_PANGOLIN_DASHBOARD_HOST` is set |
| Gerbil UDP ports | optional `UDPRoute`s + UDP listeners when `CONFIG_GERBIL_UDP_ROUTE=true` |
| `tcp.*` / other `udp.*` dynamic config | logged + skipped |

Unsupported rule constructs (`||` disjunction, `!` negation, `HostRegexp`,
`Method`, `Headers`, …) cause the affected router to be **logged and skipped**
Expand All @@ -81,6 +83,83 @@ direct `Service` `backendRef` — the controller does not synthesize a duplicate
Pick the default unless you specifically want FQDN backends or the `Backend`
CRD's other features (health checking via `BackendTrafficPolicy`, etc.).

## Badger external auth

Gateway API cannot express Traefik plugins directly. When
`CONFIG_BADGER_EXT_AUTH=true`, routers that reference Pangolin's `badger`
plugin get a managed Envoy Gateway `SecurityPolicy` targeting the generated
`HTTPRoute`.

The policy must point at an HTTP service that speaks Envoy's ext_authz protocol.
Pangolin's `/api/v1/badger/verify-session` endpoint expects Badger's JSON body,
so in today's Pangolin releases this normally means running a small shim service
that adapts Envoy's auth check request to Pangolin's badger verify-session API.
The controller defaults to a Service named `pangolin-badger-ext-authz` on port
`9002`; override that with the `CONFIG_BADGER_EXT_AUTH_*` settings.

## Scoped reconcile

Set `CONFIG_RECONCILE_ONLY` to a comma-separated allow-list to test a subset of
objects without touching the rest of the controller-owned state. The easiest
form is a hostname; the controller expands it to the matching `HTTPRoute`, its
badger `SecurityPolicy`, generated backend objects, and the shared
`ListenerSet`:

```sh
CONFIG_RECONCILE_ONLY="9-chris.example.com"
```

Explicit object selectors still work as `Kind/name` or `Kind:name`, for example
`HTTPRoute/hr-9-chris-connect-router`. Use `Hostname/name` if you need to force
a hostname selector that does not contain a dot.

Apply and GC both honor the scope. Objects outside the scope are left alone, and
selected objects that are no longer desired can still be garbage-collected.

## Migration config changes

For a dual-stack rollout, keep Traefik active and add the Gateway API pieces in
small steps:

1. Deploy a badger ext_auth shim Service, then set:

```sh
CONFIG_BADGER_EXT_AUTH=true
CONFIG_BADGER_EXT_AUTH_BACKEND_NAME=pangolin-badger-ext-authz
CONFIG_BADGER_EXT_AUTH_BACKEND_PORT=9002
```

2. Start with a narrow reconcile scope:

```sh
CONFIG_RECONCILE_ONLY="9-chris.example.com"
```

3. To serve Pangolin itself through Envoy Gateway, expose the Pangolin Service
ports for API/WebSocket (`3000`) and Next.js (`3002`), then set:

```sh
CONFIG_PANGOLIN_DASHBOARD_HOST=pangolin.example.com
```

4. To move Gerbil UDP traffic through Envoy Gateway, expose UDP ports on the
Gerbil Service and enable:

```sh
CONFIG_GERBIL_UDP_ROUTE=true
CONFIG_GERBIL_UDP_PORTS=51820,21820
```

Remove `CONFIG_RECONCILE_ONLY` only after the selected route, policy, dashboard,
and UDP paths have been validated against Envoy Gateway.

For the Gerbil public-IP migration pattern, including direct old-IP traffic plus
a separate Envoy Gateway IP, see
[docs/gerbil-gateway-migration.md](docs/gerbil-gateway-migration.md).
For the shared-IP path where only HTTP/HTTPS moves from Traefik to Envoy while
Gerbil keeps the WireGuard UDP ports, see
[docs/shared-ip-http-envoy-migration.md](docs/shared-ip-http-envoy-migration.md).

## Certificate handling with cert-manager

There are two ways the controller fills in `tls.certificateRefs` on its listeners:
Expand Down Expand Up @@ -181,6 +260,25 @@ upstream Go controller (`CONFIG_*`) where the concepts overlap.
| `CONFIG_BACKEND_KIND` | `service` | `service` (default) or `envoy-backend`. See [Backend strategy](#backend-strategy) |
| `CONFIG_HTTPROUTE_ANNOTATIONS` | _(unset)_ | `k=v,k=v` annotations stamped on every HTTPRoute. Typical: `cert-manager.io/cluster-issuer=letsencrypt-prod` |
| `CONFIG_LISTENERSET_ANNOTATIONS` | _(unset)_ | `k=v,k=v` annotations stamped on the ListenerSet |
| `CONFIG_BADGER_EXT_AUTH` | `false` | Emit Envoy Gateway `SecurityPolicy.extAuth` for routes that use badger |
| `CONFIG_BADGER_EXT_AUTH_BACKEND_NAME` | `pangolin-badger-ext-authz` | Service name of the Envoy ext_auth-compatible badger shim |
| `CONFIG_BADGER_EXT_AUTH_BACKEND_NAMESPACE` | controller namespace | Namespace of the ext_auth backend Service |
| `CONFIG_BADGER_EXT_AUTH_BACKEND_PORT` | `9002` | Service port for the ext_auth backend |
| `CONFIG_BADGER_EXT_AUTH_PATH` | _(unset)_ | Optional fixed auth check path |
| `CONFIG_BADGER_EXT_AUTH_HEADERS_TO_EXTAUTH` | `authorization,cookie,x-forwarded-for,x-forwarded-host,x-forwarded-proto,x-real-ip,p-access-token-id,p-access-token` | Headers forwarded to the auth service |
| `CONFIG_BADGER_EXT_AUTH_HEADERS_TO_BACKEND` | `remote-user,remote-email,remote-name,remote-role` | Headers copied from auth response to upstream backends |
| `CONFIG_BADGER_EXT_AUTH_FAIL_OPEN` | `false` | Allow traffic if the auth service is unavailable |
| `CONFIG_RECONCILE_ONLY` | _(unset)_ | Optional comma-separated allow-list of hostnames or `Kind/name` objects to apply/GC |
| `CONFIG_PANGOLIN_DASHBOARD_HOST` | _(unset)_ | When set, emit static dashboard HTTPRoutes for this hostname |
| `CONFIG_PANGOLIN_SERVICE_NAME` | `pangolin` | Service that backs the dashboard/API routes |
| `CONFIG_PANGOLIN_SERVICE_NAMESPACE`| controller namespace | Namespace of the Pangolin Service |
| `CONFIG_PANGOLIN_API_PORT` | `3000` | Pangolin API/WebSocket Service port |
| `CONFIG_PANGOLIN_NEXT_PORT` | `3002` | Pangolin Next.js Service port |
| `CONFIG_PANGOLIN_REDIRECT_HTTP_TO_HTTPS` | `true` | Emit an HTTP-to-HTTPS redirect route when HTTPS listeners are configured |
| `CONFIG_GERBIL_UDP_ROUTE` | `false` | Emit UDP listeners and UDPRoutes for Gerbil |
| `CONFIG_GERBIL_SERVICE_NAME` | `gerbil` | Service that backs Gerbil UDP traffic |
| `CONFIG_GERBIL_SERVICE_NAMESPACE` | controller namespace | Namespace of the Gerbil Service |
| `CONFIG_GERBIL_UDP_PORTS` | `51820,21820` | UDP ports to expose through the ListenerSet |
| `CONFIG_FIELD_MANAGER` | `pangolin-gateway-controller` | Server-Side Apply field manager |
| `CONFIG_MANAGED_LABEL_KEY` | `app.kubernetes.io/managed-by` | Used for GC selector |
| `CONFIG_MANAGED_LABEL_VALUE` | `pangolin-gateway-controller` | Used for GC selector |
Expand Down
86 changes: 84 additions & 2 deletions deploy/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,15 @@ and as a component from sibling infrastructure repos.
| File | Resource | Notes |
|---|---|---|
| `namespace.yaml` | `Namespace/pangolin-system` | Drop this if the namespace is created elsewhere — see [Embedding from another kustomization](#embedding-from-another-kustomization). |
| `rbac.yaml` | `ServiceAccount`, `ClusterRole`, `ClusterRoleBinding` | Cluster-wide read on Gateways; write on HTTPRoute / ListenerSet / Service / EndpointSlice / Envoy Gateway `Backend`. |
| `rbac.yaml` | `ServiceAccount`, `ClusterRole`, `ClusterRoleBinding` | Cluster-wide read on Gateways; write on HTTPRoute / UDPRoute / ListenerSet / Service / EndpointSlice / Envoy Gateway `Backend` and `SecurityPolicy`. |
| `deployment.yaml` | `Deployment/pangolin-gateway-controller` | Single replica, read-only rootfs, all caps dropped. |

The CRDs are **not** installed here. You need:

- Gateway API v1.5+ **experimental** channel (for `ListenerSet`):
`kubectl apply -k https://github.com/kubernetes-sigs/gateway-api/config/crd/experimental?ref=v1.5.1`
- Envoy Gateway, if you set `CONFIG_BACKEND_KIND=envoy-backend`:
- Envoy Gateway, if you set `CONFIG_BACKEND_KIND=envoy-backend` or
`CONFIG_BADGER_EXT_AUTH=true`:
see <https://gateway.envoyproxy.io/docs/install/>.

A `Gateway` object the controller can attach its `ListenerSet` to must exist
Expand Down Expand Up @@ -72,6 +73,13 @@ that file for authoritative defaults.
| `CONFIG_TLS_SECRET_TEMPLATE` | _(unset)_ | Required when HTTPS listeners are on. Supports `{hostname}` and `{hostname-dashed}` placeholders. |
| `CONFIG_HTTPROUTE_ANNOTATIONS` | _(empty)_ | `k=v,k=v` annotations stamped onto every `HTTPRoute`. Typical use: cert-manager cluster-issuer. |
| `CONFIG_LISTENERSET_ANNOTATIONS` | _(empty)_ | Same for `ListenerSet`. |
| `CONFIG_BADGER_EXT_AUTH` | `false` | Emit Envoy Gateway `SecurityPolicy.extAuth` for routes that use Pangolin's badger plugin. Requires an Envoy ext_auth-compatible shim service. |
| `CONFIG_BADGER_EXT_AUTH_BACKEND_NAME` | `pangolin-badger-ext-authz` | Auth shim Service name. |
| `CONFIG_BADGER_EXT_AUTH_BACKEND_PORT` | `9002` | Auth shim Service port. |
| `CONFIG_RECONCILE_ONLY` | _(empty)_ | Optional comma-separated allow-list of hostnames or object selectors such as `9-chris.example.com` or `HTTPRoute/hr-9-chris-connect-router`. |
| `CONFIG_PANGOLIN_DASHBOARD_HOST` | _(empty)_ | Emit static dashboard/API/WebSocket HTTPRoutes for this host. |
| `CONFIG_GERBIL_UDP_ROUTE` | `false` | Emit Gerbil UDP listeners and UDPRoutes; requires UDPRoute support in your Gateway implementation. |
| `CONFIG_GERBIL_UDP_PORTS` | `51820,21820` | UDP ports to route to the Gerbil Service. |
| `CONFIG_FIELD_MANAGER` | `pangolin-gateway-controller` | SSA field manager. Change only if you need to migrate ownership. |
| `RUST_LOG` | `info` | Standard `tracing_subscriber::EnvFilter`. |

Expand All @@ -98,6 +106,80 @@ annotation env vars to your cluster issuer:
cert-manager will see the annotated `ListenerSet`, mint a Certificate per
listener hostname, and write the Secret matching `CONFIG_TLS_SECRET_TEMPLATE`.

## Badger ext_authz (optional)

The controller can close the badger gap from the Traefik plugin side by emitting
Envoy Gateway `SecurityPolicy` objects, but it does not make Pangolin's badger
endpoint speak Envoy's auth protocol by itself. Run a shim Service that accepts
Envoy HTTP ext_auth checks and calls
`http://pangolin:3001/api/v1/badger/verify-session`, then enable:

```yaml
- name: CONFIG_BADGER_EXT_AUTH
value: "true"
- name: CONFIG_BADGER_EXT_AUTH_BACKEND_NAME
value: "pangolin-badger-ext-authz"
- name: CONFIG_BADGER_EXT_AUTH_BACKEND_PORT
value: "9002"
```

During migration, combine this with `CONFIG_RECONCILE_ONLY` to apply a single
hostname first. Hostname scopes expand to the matching `HTTPRoute`, badger
`SecurityPolicy`, generated backend objects, and the shared `ListenerSet`.

## Configuration changes for migration

When enabling the new Gateway API path in an existing dual-stack deployment,
change configuration in this order:

1. Keep Traefik serving production traffic.
2. Add or deploy a Service that fronts your badger ext_auth shim. The Service
name and port must match `CONFIG_BADGER_EXT_AUTH_BACKEND_NAME` and
`CONFIG_BADGER_EXT_AUTH_BACKEND_PORT`.
3. Make sure the Pangolin Service exposes the API/WebSocket port (`3000` by
default) and the Next.js port (`3002` by default) if you want
`CONFIG_PANGOLIN_DASHBOARD_HOST`.
4. Make sure the Gerbil Service exposes UDP `51820` and `21820`, or override
`CONFIG_GERBIL_UDP_PORTS`.
5. Patch the controller Deployment env vars.

Example kustomize patch:

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: pangolin-gateway-controller
namespace: pangolin-system
spec:
template:
spec:
containers:
- name: controller
env:
- name: CONFIG_BADGER_EXT_AUTH
value: "true"
- name: CONFIG_BADGER_EXT_AUTH_BACKEND_NAME
value: "pangolin-badger-ext-authz"
- name: CONFIG_BADGER_EXT_AUTH_BACKEND_PORT
value: "9002"
- name: CONFIG_RECONCILE_ONLY
value: "9-chris.example.com"
- name: CONFIG_PANGOLIN_DASHBOARD_HOST
value: "pangolin.example.com"
- name: CONFIG_GERBIL_UDP_ROUTE
value: "true"
```

For the first test, keep `CONFIG_RECONCILE_ONLY` narrow. Remove it only after the
selected hostname's `HTTPRoute` and `SecurityPolicy` are accepted by Envoy
Gateway and the auth shim returns the expected allow/redirect/deny responses.
For Gerbil's old-IP plus Envoy-IP migration pattern, see
[`docs/gerbil-gateway-migration.md`](../docs/gerbil-gateway-migration.md).
For a shared public IP where only TCP `80`/`443` moves from Traefik to Envoy,
see
[`docs/shared-ip-http-envoy-migration.md`](../docs/shared-ip-http-envoy-migration.md).

## Embedding from another kustomization

The deploy folder is a working stand-alone overlay. To embed it inside an
Expand Down
17 changes: 17 additions & 0 deletions deploy/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,23 @@ spec:
# value: "cert-manager.io/cluster-issuer=letsencrypt-prod"
# - name: CONFIG_LISTENERSET_ANNOTATIONS
# value: "cert-manager.io/cluster-issuer=letsencrypt-prod"
# Badger ext_authz (optional). Point this at an Envoy ext_auth-compatible
# shim for pangolin's /api/v1/badger/verify-session protocol.
# - name: CONFIG_BADGER_EXT_AUTH
# value: "true"
# - name: CONFIG_BADGER_EXT_AUTH_BACKEND_NAME
# value: "pangolin-badger-ext-authz"
# - name: CONFIG_BADGER_EXT_AUTH_BACKEND_PORT
# value: "9002"
# Test-mode reconcile scope (optional), e.g. only one hostname.
# - name: CONFIG_RECONCILE_ONLY
# value: "9-chris.example.com"
# Pangolin dashboard through Gateway API (optional).
# - name: CONFIG_PANGOLIN_DASHBOARD_HOST
# value: "pangolin.example.com"
# Gerbil UDP through Gateway API (optional; requires UDPRoute support).
# - name: CONFIG_GERBIL_UDP_ROUTE
# value: "true"
- name: RUST_LOG
value: "info,pangolin_gateway_controller=info"
resources:
Expand Down
11 changes: 6 additions & 5 deletions deploy/rbac.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,10 @@ metadata:
rules:
# Gateway API objects we own.
- apiGroups: ["gateway.networking.k8s.io"]
resources: ["httproutes", "listenersets"]
resources: ["httproutes", "listenersets", "udproutes"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["gateway.networking.k8s.io"]
resources: ["httproutes/status", "listenersets/status"]
resources: ["httproutes/status", "listenersets/status", "udproutes/status"]
verbs: ["get"]
# Core resources we synthesize for IP-backed pangolin targets.
- apiGroups: [""]
Expand All @@ -27,10 +27,11 @@ rules:
- apiGroups: ["gateway.networking.k8s.io"]
resources: ["gateways", "gatewayclasses"]
verbs: ["get", "list", "watch"]
# Envoy Gateway's Backend CRD. Only used when CONFIG_BACKEND_KIND=envoy-backend.
# Safe to leave granted even when unused — RBAC checks happen lazily.
# Envoy Gateway CRDs. Backend is used when CONFIG_BACKEND_KIND=envoy-backend;
# SecurityPolicy is used when CONFIG_BADGER_EXT_AUTH=true.
# Safe to leave granted even when unused — checks happen lazily.
- apiGroups: ["gateway.envoyproxy.io"]
resources: ["backends"]
resources: ["backends", "securitypolicies"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
Expand Down
Loading
Loading