Skip to content

Commit b430dc3

Browse files
committed
fix(security): bind Grafana to loopback and gate wide exposure on an admin password
RenderMonitoringCompose emitted the Grafana service with an unconditional host port publication and no environment block at all - no GF_SECURITY_ADMIN_PASSWORD, no auth hardening - so the deployed container kept grafana-oss's built-in admin/admin while docker published it on 0.0.0.0. On a cloud VM with a permissive security group that is the public internet: full node metrics, the ability to lock the operator out, and - because a Grafana admin can add arbitrary datasources and query them through the backend proxy - a server-side request primitive into the host's internal network and cloud metadata endpoints. Grafana is now bound to 127.0.0.1 by default. A new monitoring.grafana.expose restores the wide bind, but is rejected unless monitoring.grafana.admin_password_env is also set, so the escape hatch cannot recreate the finding; the renderer independently falls back to loopback for an unvalidated intent. The password is rendered as a required compose variable reference, so the secret never enters the 0644 compose file and an unset or empty value fails at interpolation before any container is created. Verified against real Docker: the default resolves to host_ip 127.0.0.1, and expose without a password falls back to loopback. Breaking: Grafana is no longer reachable off-host by default. Use an SSH tunnel, a reverse proxy, or the new expose opt-in.
1 parent d2e9d4d commit b430dc3

9 files changed

Lines changed: 317 additions & 14 deletions

File tree

CHANGELOG.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -254,6 +254,13 @@ The agent-ergonomics arc lands across four sequenced PRs:
254254
absent) is the only way to skip the check; results carry
255255
`verification_skipped` so `md5_verified: false` can no longer be read
256256
as "the mirror had no sidecar". Schema 1.12.2 → 1.12.3
257+
- **Breaking:** the rendered monitoring stack binds Grafana's host port
258+
to `127.0.0.1` instead of `0.0.0.0`; it previously published the
259+
grafana-oss default `admin/admin` login to every network that could
260+
reach the deployment host. Use an SSH tunnel, or opt back in with
261+
`monitoring.grafana.expose: true`, which now requires
262+
`monitoring.grafana.admin_password_env` (the NAME of an env var
263+
feeding `GF_SECURITY_ADMIN_PASSWORD`)
257264

258265
## [0.1.0-alpha] — 2026-XX-XX
259266

README.md

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -370,7 +370,20 @@ trond network status
370370
| `trond status` / `trond inspect <node> -o json` | Expose the stack's `prometheus_port` / `grafana_port` so agents can discover it |
371371
| `trond remove <node>` / `trond network destroy` | Automatically cleans up the monitoring stack |
372372

373-
After deployment, Grafana is available at http://localhost:3000 (admin/admin) with 5 dashboards: java-tron-server, java-tron-api, java-tron-api-statistic, java-tron-mechanism, and node-exporter-full. Prometheus is at http://localhost:9090.
373+
After deployment, Grafana is available at http://localhost:3000**on the deployment host's loopback interface only** — with 5 dashboards: java-tron-server, java-tron-api, java-tron-api-statistic, java-tron-mechanism, and node-exporter-full. Prometheus is at http://localhost:9090.
374+
375+
Grafana keeps the image's default `admin/admin` login until you give it a password, so it is bound to `127.0.0.1`: reach it from your workstation over an SSH tunnel (`ssh -L 3000:127.0.0.1:3000 user@host`) or a reverse proxy you control. To publish it on all interfaces instead, opt in explicitly — which requires an admin password:
376+
377+
```yaml
378+
monitoring:
379+
enabled: true
380+
grafana:
381+
port: 3000
382+
expose: true # bind 0.0.0.0 instead of 127.0.0.1
383+
admin_password_env: GRAFANA_ADMIN_PASSWORD # NAME of an env var, not the password
384+
```
385+
386+
`GRAFANA_ADMIN_PASSWORD` must be set in the environment that runs `trond apply` (compose refuses to start the stack otherwise). Grafana applies it when it first initialises its database, so rotate the password in Grafana itself for a stack that is already running.
374387

375388
**Limitations**: The single Prometheus instance loses visibility into nodes isolated by `trond partition`; metrics resume after `trond heal`. Monitoring is Docker-only (Prometheus and Grafana run as containers), so jar-runtime targets need Docker on the trond machine for the monitoring stack.
376389

examples/monitoring.yaml

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,11 @@
33
# trond apply --intent examples/monitoring.yaml --no-monitor (skip monitoring)
44
#
55
# Default behaviour:
6-
# - Grafana is exposed on port 3000.
6+
# - Grafana is published on port 3000 of the deployment host's loopback
7+
# interface (127.0.0.1) only, because the image ships with the
8+
# well-known admin/admin login. Reach it over an SSH tunnel, or set
9+
# grafana.expose (which requires grafana.admin_password_env) to
10+
# publish it on all interfaces.
711
# - Prometheus is NOT exposed on the host by default; it is reachable
812
# inside the Docker network by Grafana. To expose Prometheus add
913
# monitoring.prometheus.port explicitly.
@@ -25,6 +29,10 @@ monitoring:
2529
retention: 7d
2630
grafana:
2731
port: 3000
32+
# Uncomment BOTH lines to publish Grafana on every interface with an
33+
# admin password taken from the named environment variable:
34+
# expose: true
35+
# admin_password_env: GRAFANA_ADMIN_PASSWORD
2836

2937
nodes:
3038
- type: fullnode

internal/intent/fields_test.go

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -960,6 +960,70 @@ nodes:
960960
}
961961
}
962962

963+
func TestMonitoring_GrafanaExposeAndAdminPassword(t *testing.T) {
964+
monitoringYAML := func(grafana string) []byte {
965+
return []byte(fmt.Sprintf(`
966+
name: mon
967+
network: mainnet
968+
target: {type: local}
969+
monitoring:
970+
enabled: true
971+
grafana:
972+
%s
973+
nodes:
974+
- type: fullnode
975+
`, grafana))
976+
}
977+
978+
t.Run("defaults to not exposed", func(t *testing.T) {
979+
i, err := Parse(monitoringYAML(" port: 3000"))
980+
if err != nil {
981+
t.Fatal(err)
982+
}
983+
if i.Monitoring.Grafana.Expose {
984+
t.Error("grafana.expose should default to false")
985+
}
986+
if i.Monitoring.Grafana.AdminPasswordEnv != "" {
987+
t.Errorf("grafana.admin_password_env = %q, want empty",
988+
i.Monitoring.Grafana.AdminPasswordEnv)
989+
}
990+
})
991+
992+
t.Run("expose with admin_password_env parses", func(t *testing.T) {
993+
i, err := Parse(monitoringYAML(" expose: true\n admin_password_env: GRAFANA_ADMIN_PASSWORD"))
994+
if err != nil {
995+
t.Fatal(err)
996+
}
997+
if !i.Monitoring.Grafana.Expose {
998+
t.Error("grafana.expose not parsed")
999+
}
1000+
if i.Monitoring.Grafana.AdminPasswordEnv != "GRAFANA_ADMIN_PASSWORD" {
1001+
t.Errorf("grafana.admin_password_env = %q, want GRAFANA_ADMIN_PASSWORD",
1002+
i.Monitoring.Grafana.AdminPasswordEnv)
1003+
}
1004+
})
1005+
1006+
// Exposing Grafana on every interface with the image's default
1007+
// admin login is the combination this rejects.
1008+
t.Run("expose without admin_password_env rejected", func(t *testing.T) {
1009+
_, err := Parse(monitoringYAML(" expose: true"))
1010+
if err == nil {
1011+
t.Fatal("expected error for expose without admin_password_env")
1012+
}
1013+
if !strings.Contains(err.Error(), "admin_password_env") {
1014+
t.Errorf("error %q should name the missing field", err)
1015+
}
1016+
})
1017+
1018+
// The intent carries the env var NAME, never the password.
1019+
t.Run("literal password rejected", func(t *testing.T) {
1020+
_, err := Parse(monitoringYAML(` admin_password_env: "hunter2 !"`))
1021+
if err == nil {
1022+
t.Fatal("expected error for a non-env-name admin_password_env")
1023+
}
1024+
})
1025+
}
1026+
9631027
func TestMonitoring_InvalidRetention(t *testing.T) {
9641028
cases := []string{"7", "days", "1.5d", "week"}
9651029
for _, val := range cases {

internal/intent/loader.go

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -439,6 +439,22 @@ func validateMonitoring(m *Monitoring) error {
439439
return fmt.Errorf("monitoring.prometheus.retention must match ^\\d+[dwh]$ (e.g. \"7d\", \"2w\"), got %q", m.Prometheus.Retention)
440440
}
441441
}
442+
if m.Grafana.AdminPasswordEnv != "" {
443+
// Same contract as witness_key.private_key_env: the intent holds
444+
// the NAME of an env var, never the secret. The name is also
445+
// interpolated into the rendered compose file, so restricting it
446+
// to env-name characters keeps it from breaking out of ${...}.
447+
envVarPattern := regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_]*$`)
448+
if !envVarPattern.MatchString(m.Grafana.AdminPasswordEnv) {
449+
return fmt.Errorf("monitoring.grafana.admin_password_env %q is not a valid environment variable name; it must be the NAME of an env var holding the Grafana admin password (e.g., GRAFANA_ADMIN_PASSWORD), not the password itself", m.Grafana.AdminPasswordEnv)
450+
}
451+
}
452+
if m.Grafana.Expose && m.Grafana.AdminPasswordEnv == "" {
453+
// Publishing Grafana beyond loopback with the image's default
454+
// admin/admin login hands the dashboards and the datasource proxy
455+
// to anyone who can reach the port.
456+
return fmt.Errorf("monitoring.grafana.expose publishes Grafana on all host interfaces and therefore requires monitoring.grafana.admin_password_env: set it to the NAME of an environment variable holding the admin password (e.g., admin_password_env: GRAFANA_ADMIN_PASSWORD), or drop expose to keep Grafana bound to 127.0.0.1")
457+
}
442458
return nil
443459
}
444460

internal/intent/schema.go

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -437,6 +437,27 @@ type PromConfig struct {
437437
// GrafConfig configures the Grafana visualizer.
438438
type GrafConfig struct {
439439
Port int `yaml:"port,omitempty" json:"port,omitempty"`
440+
441+
// Expose publishes Grafana's host port on every interface (0.0.0.0)
442+
// instead of loopback only. Off by default: the grafana-oss image
443+
// ships a well-known default admin login, so a host-wide bind hands
444+
// the dashboards — and Grafana's datasource proxy, which can be
445+
// pointed at anything the host can reach — to whoever can reach the
446+
// port. Requires AdminPasswordEnv (see validateMonitoring).
447+
Expose bool `yaml:"expose,omitempty" json:"expose,omitempty"`
448+
449+
// AdminPasswordEnv is the NAME of an environment variable holding the
450+
// Grafana admin password — never the password itself. The rendered
451+
// compose references it as ${NAME:?...}, so the variable must be set
452+
// in the environment that runs `docker compose up` (trond's own
453+
// environment for local targets); compose refuses to start the stack
454+
// when it is unset or empty.
455+
//
456+
// Grafana applies GF_SECURITY_ADMIN_PASSWORD when it initialises its
457+
// database, i.e. on the first start of a fresh grafana_data volume.
458+
// Setting it for an already-initialised stack does not rotate an
459+
// existing admin password.
460+
AdminPasswordEnv string `yaml:"admin_password_env,omitempty" json:"admin_password_env,omitempty"`
440461
}
441462

442463
// BoolPtr is a helper for creating *bool values in intent construction.

internal/render/monitoring.go

Lines changed: 57 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,11 @@ const monitoringComposeTmpl = `services:
5757
limits:
5858
memory: 2g
5959
ports:
60-
- "{{.GrafanaPort}}:3000"
60+
- "{{.GrafanaPortMapping}}"
61+
{{- if .GrafanaAdminPasswordRef }}
62+
environment:
63+
- GF_SECURITY_ADMIN_PASSWORD={{.GrafanaAdminPasswordRef}}
64+
{{- end }}
6165
volumes:
6266
- ./grafana_data:/var/lib/grafana
6367
- ./grafana/provisioning:/etc/grafana/provisioning
@@ -81,26 +85,70 @@ var composeTmpl = template.Must(template.New("monitoring").Parse(monitoringCompo
8185
type monitoringComposeData struct {
8286
Name string
8387
PrometheusPort int
84-
GrafanaPort int
85-
Retention string
86-
NetworkName string
88+
// GrafanaPortMapping is the full compose "ports" entry for Grafana,
89+
// including the host bind address — see grafanaPortMapping.
90+
GrafanaPortMapping string
91+
// GrafanaAdminPasswordRef is the compose interpolation reference that
92+
// supplies GF_SECURITY_ADMIN_PASSWORD, or "" when the operator did
93+
// not configure one.
94+
GrafanaAdminPasswordRef string
95+
Retention string
96+
NetworkName string
8797
}
8898

8999
// RenderMonitoringCompose generates a docker-compose.yaml for the
90100
// Prometheus + Grafana monitoring stack.
91101
func RenderMonitoringCompose(name string, i *intent.Intent, targets []MonitoringTarget, networkName string) string {
92102
var buf bytes.Buffer
103+
g := i.Monitoring.Grafana
93104
data := monitoringComposeData{
94-
Name: name,
95-
PrometheusPort: i.Monitoring.Prometheus.Port,
96-
GrafanaPort: i.Monitoring.Grafana.Port,
97-
Retention: i.Monitoring.Prometheus.Retention,
98-
NetworkName: networkName,
105+
Name: name,
106+
PrometheusPort: i.Monitoring.Prometheus.Port,
107+
GrafanaPortMapping: grafanaPortMapping(g),
108+
GrafanaAdminPasswordRef: grafanaAdminPasswordRef(g),
109+
Retention: i.Monitoring.Prometheus.Retention,
110+
NetworkName: networkName,
99111
}
100112
_ = composeTmpl.Execute(&buf, data)
101113
return buf.String()
102114
}
103115

116+
// grafanaPortMapping builds Grafana's compose "ports" entry.
117+
//
118+
// A bare "<port>:3000" publishes on 0.0.0.0, which on any host with a
119+
// permissive firewall exposes the dashboard — and Grafana's datasource
120+
// proxy — to the network with the grafana-oss image's default admin
121+
// login. Bind to loopback instead; reaching it from elsewhere is then an
122+
// SSH tunnel (or a reverse proxy the operator controls) away.
123+
//
124+
// monitoring.grafana.expose opts back into the host-wide bind, and is
125+
// only honoured together with an admin password: intent validation
126+
// rejects that combination up front, and this stays fail-closed for any
127+
// caller that renders an unvalidated intent.
128+
func grafanaPortMapping(g intent.GrafConfig) string {
129+
if g.Expose && g.AdminPasswordEnv != "" {
130+
return fmt.Sprintf("%d:3000", g.Port)
131+
}
132+
return fmt.Sprintf("127.0.0.1:%d:3000", g.Port)
133+
}
134+
135+
// grafanaAdminPasswordRef returns the compose interpolation that feeds
136+
// GF_SECURITY_ADMIN_PASSWORD from the operator's environment, keeping the
137+
// secret out of the rendered file (the same shape composeEnvLines uses
138+
// for the witness keystore password).
139+
//
140+
// The ":?" form makes `docker compose up` fail loudly when the variable
141+
// is unset or empty rather than silently starting Grafana with an empty
142+
// or default admin password. The name is constrained to env-name
143+
// characters by intent validation.
144+
func grafanaAdminPasswordRef(g intent.GrafConfig) string {
145+
if g.AdminPasswordEnv == "" {
146+
return ""
147+
}
148+
return fmt.Sprintf("${%s:?set %s to the Grafana admin password (monitoring.grafana.admin_password_env)}",
149+
g.AdminPasswordEnv, g.AdminPasswordEnv)
150+
}
151+
104152
// RenderPrometheusConfig generates prometheus.yml content.
105153
//
106154
// The retention parameter is currently unused: Prometheus TSDB retention is

0 commit comments

Comments
 (0)