Ethereum-node-observability

Observability stack for Ethereum nodes: Prometheus, Grafana, Loki, Grafana Alloy, Jaeger, and Alertmanager with SLO burn rate alerting.

Note: Geth runs in --dev mode with no authentication or TLS configured. Intended for learning and local development only not production ready.

Stack

Service	Purpose	Port
geth	Ethereum dev node	8545 (RPC), 8546 (WS), 6060 (metrics)
Prometheus	Metrics scraping + SLO recording rules	9090
Grafana	Dashboards: node health, RPC, SLOs	3000
Loki	Log aggregation	3100
Grafana Alloy	Next-gen collector logs + metrics (replaces Promtail)	12345 (UI)
Alertmanager	Alert routing => Slack	9093
Jaeger	Distributed tracing + trace-log correlation	16686

Quick Start

git clone https://github.com/calvin-puram/ethereum-node-observability
cd ethereum-node-observability
docker compose up -d
open http://localhost:3000   # Grafana (admin/admin)
open http://localhost:12345  # Alloy pipeline UI

Grafana Alloy

Alloy replaces Promtail as the collection agent. Uses River syntax (alloy/config.alloy) and handles three pipelines:

• Geth container logs => parse log level + trace ID => Loki
• Geth Prometheus metrics => remote_write => Prometheus
• Alloy self-metrics => Prometheus

The debug UI at http://localhost:12345 shows a live graph of all pipelines useful for verifying logs are flowing and debugging label issues.

SLOs Defined

SLO	Target	Error Budget
Availability	99.9% of RPC requests non-5xx	43.8 min/month
Latency	95% of requests within 500ms	36 hours/month
Freshness	Block lag < 30s, 99% of time	7.3 hours/month

Alert Severity

Alert	Condition	Severity
EthNodeDown	Node unreachable > 1m	critical
RpcAvailabilityBurnRateCritical	14.4x burn rate (5m + 1h windows)	critical
RpcAvailabilityBurnRateHigh	6x burn rate (30m + 6h windows)	warning
EthNodeLowPeers	< 3 peers for 5m	warning
EthRpcLatencyHigh	rpc_duration_all p99 > 1s	warning
EthRpcMethodLatencyHigh	eth_getLogs p95 > 5s	warning
EthTxpoolPendingHigh	> 5000 pending txs	warning

Geth Prometheus Metric Names

p2p_peers                              peer count
ethereum_blockchain_head_header_number chain head block number
txpool_pending                         pending transactions
txpool_queued                          queued transactions
eth_db_chaindata_disk_size             chaindata disk usage (bytes)
rpc_success_count                      successful RPC calls
rpc_failure_count                      failed RPC calls
rpc_duration_all                       RPC duration histogram (nanoseconds)
rpc_duration_eth_getLogs               per-method duration histogram