A production-grade distributed rate limiting system built with Go, gRPC, Redis, and Docker. Designed for high-throughput environments with strict latency requirements, it enforces per-IP, per-user, and global request quotas across horizontally-scaled nodes.
Client Request
│
▼
HTTP Gateway (:8081)
│ Rate Limit Middleware
▼
gRPC Rate Limiter (:50051)
│ Lua Script (atomic check-and-increment)
▼
Redis (:6379)
│ allow / deny
▼
Upstream Service
Key properties:
- Consistent hashing routes each client key to a deterministic rate limiter node, minimizing Redis key scatter.
- A circuit breaker around Redis protects the system during outages — the gateway fails open to preserve availability.
- All rate limit decisions are atomic: a single Lua script performs the read-increment-expire cycle with no race conditions.
| Component | Technology |
|---|---|
| Language | Go 1.25 |
| API Gateway | Go net/http + Reverse Proxy |
| Rate Limiter | Redis + Lua Scripts |
| RPC | gRPC + Protocol Buffers |
| Observability | Prometheus |
| Containers | Docker + Docker Compose |
| Load Testing | k6 |
# Start the full stack (Redis, rate limiter, gateway, Prometheus)
docker compose up -d
# Verify all services are running
docker compose ps# Start Redis
docker compose up -d redis
# Start the gRPC rate limiter server
go run ./cmd/server
# Start the HTTP API gateway
go run ./cmd/gateway.
├── cmd/
│ ├── server/ # gRPC rate limiter entry point
│ ├── gateway/ # HTTP API gateway entry point
│ ├── client/ # CLI test client for the gRPC server
│ └── mockserver/ # Lightweight upstream mock for load testing
├── internal/
│ ├── limiter/ # Core rate limiting algorithms
│ │ ├── token_bucket.go
│ │ ├── sliding_window.go
│ │ ├── redis_limiter.go
│ │ ├── circuit_breaker.go
│ │ └── multi_tier.go
│ ├── gateway/ # Gateway logic and consistent hashing
│ ├── ratelimiter/ # gRPC server implementation
│ └── metrics/ # Prometheus metric definitions
├── proto/ # Protobuf service definition and generated stubs
├── tests/ # k6 load test scripts and results
├── grafana/ # Grafana dashboard and datasource provisioning
├── certs/ # TLS certificate generation script
├── Dockerfile.server
├── Dockerfile.gateway
├── docker-compose.yml
└── prometheus.yml
All configuration is supplied via environment variables. Default values are suitable for local development.
| Variable | Default | Description |
|---|---|---|
REDIS_ADDR |
localhost:6379 |
Redis server address |
GRPC_PORT |
50051 |
gRPC rate limiter listen port |
METRICS_PORT |
9090 |
Prometheus metrics endpoint port |
RATE_LIMIT |
100 |
Maximum requests allowed per window |
GRPC_ADDRS |
localhost:50051 |
Comma-separated list of rate limiter nodes (gateway) |
UPSTREAM_URL |
http://localhost:9999 |
Backend service the gateway proxies to |
HTTP_PORT |
8081 |
Gateway HTTP listen port |
TLS_CERT |
./certs/server.crt |
Path to TLS certificate (omit to run insecure) |
TLS_KEY |
./certs/server.key |
Path to TLS private key |
GET /health
{"status": "ok"}GET /get
X-API-Key: <client-key>
Rate limit decisions are made per X-API-Key. If the header is absent, the client IP address is used as the key.
Response headers on allowed requests:
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 99
X-RateLimit-Reset: 60000
Response on rate limit exceeded (HTTP 429):
{"error": "rate_limit_exceeded"}Additional headers on 429:
Retry-After: <seconds until window resets>
X-RateLimit-Limit: <configured limit>
X-RateLimit-Remaining: 0
GET /dashboard
Live HTML dashboard showing per-node metrics, circuit breaker state, and request throughput.
GET /api/metrics
Proxies the Prometheus metrics endpoint from the rate limiter node.
The multi-tier limiter enforces limits at three independent levels. A request is denied if any tier is exceeded.
| Tier | Default Limit | Window | Purpose |
|---|---|---|---|
| IP | 100 requests | 1 minute | Prevents abuse from a single source |
| User | 1,000 requests | 1 minute | Enforces per-account quotas |
| Global | 100,000 requests | 1 minute | Protects total system capacity |
Tokens accumulate at a fixed rate up to a maximum burst capacity. Each request consumes one token. Ideal for endpoints that must support legitimate burst traffic.
Tracks exact request timestamps within a rolling time window. Provides precise enforcement with no boundary artifacts, at the cost of higher memory usage per client.
A Lua script atomically increments a Redis counter and sets its TTL in a single round trip. This is the algorithm used in production; it is the only approach that is globally consistent across multiple gateway instances.
| Algorithm | Latency | Throughput | Accuracy | Best For |
|---|---|---|---|---|
| Token Bucket | 65 ns/op | ~15M req/s | Approximate | Single-node, high-throughput |
| Sliding Window | 65 µs/op | ~15K req/s | Exact | Strict per-window enforcement |
| Redis Lua | ~130 µs/op | ~7K req/s | Exact | Distributed, multi-node |
| Service | URL |
|---|---|
| HTTP Gateway | http://localhost:8081 |
| Gateway Dashboard | http://localhost:8081/dashboard |
| gRPC Rate Limiter | localhost:50051 |
| Prometheus Metrics | http://localhost:9090/metrics |
| Prometheus UI | http://localhost:9091 |
# Total rate limit decisions, by outcome and limit type
ratelimiter_decisions_total{decision="allowed|denied", limit_type="ip|user|global"}
# gRPC request latency histogram
ratelimiter_request_duration_seconds{method="CheckLimit"}
# Circuit breaker state (0 = closed, 1 = open, 2 = half-open)
ratelimiter_circuit_breaker_state{}
# Redis error count, by error category
ratelimiter_redis_errors_total{error_type="timeout|circuit_open|connection"}
# Gateway HTTP request count
gateway_requests_total{method, path, status}
# Gateway request latency histogram
gateway_request_duration_seconds{method, path}
# Run all unit tests
go test ./...
# Run benchmarks with memory allocation stats
go test -bench=. -benchmem ./internal/limiter/
# Run k6 load test
k6 run tests/load_test.jsBenchmarkTokenBucket_Allow-8 45,916,492 65 ns/op 0 B/op 0 allocs/op
BenchmarkSlidingWindow_Allow-8 136,651 65,060 ns/op 0 B/op 0 allocs/op
BenchmarkRedisLimiter_Allow-8 25,922 129,920 ns/op 690 B/op 22 allocs/op
Scenarios:
normal_load: 10 VUs × 30 s
spike: 200 VUs × 40 s
Results:
Total requests: ~38,327
Throughput: ~510 req/s
p50 latency: 4.9 ms
p95 latency: 9.7 ms ← within the <10 ms SLA
Max latency: 452 ms (spike peak)
Throughput: 86 req/s
Avg latency: 421 ms
p95 latency: 956 ms
The latency difference is entirely upstream network RTT; the gateway itself adds less than 1 ms overhead at these request rates.
O(1) time and space complexity. Supports controlled burst traffic without background goroutines or scheduled cleanup. Appropriate for single-node scenarios where the small approximation error is acceptable.
A Lua script executes atomically on the Redis server — no other command can interleave during its execution. This eliminates the check-then-act race condition that plagues MULTI/EXEC transactions, while also reducing round trips from two to one.
Protocol Buffer serialization is roughly 5× smaller and faster than JSON. gRPC provides built-in connection pooling, deadline propagation, and reflection for tooling — all relevant for a hot-path service.
When Redis becomes unavailable, synchronous calls to it would block every incoming request until timeout. The circuit breaker trips after a configurable failure threshold, causing the gateway to fail open: requests are allowed through without rate limiting rather than being dropped. This trades strict limiting for availability — the right default for most public APIs.
Adding or removing rate limiter nodes remaps approximately 1/N of existing keys rather than all of them. This keeps Redis key locality stable during rolling deployments and autoscaling events.
A single global counter cannot distinguish between a single abusive client and a legitimate traffic surge. Layering IP, user, and global limits lets each tier enforce the right policy at the right granularity without any tier interfering with the others.
The server and gateway support mutual TLS. Set TLS_CERT and TLS_KEY to enable encrypted gRPC transport. If the certificate file is absent at startup, the server logs a warning and falls back to plaintext — intentionally permissive for local development, not suitable for production.
To generate a self-signed certificate for development:
bash certs/generate.shSee LICENSE.md.