Skip to content

Repository files navigation

Distributed Rate Limiter & API Gateway

A production-grade distributed rate limiting system built with Go, gRPC, Redis, and Docker. Designed for high-throughput environments with strict latency requirements, it enforces per-IP, per-user, and global request quotas across horizontally-scaled nodes.


Architecture

Client Request
     │
     ▼
HTTP Gateway (:8081)
     │  Rate Limit Middleware
     ▼
gRPC Rate Limiter (:50051)
     │  Lua Script (atomic check-and-increment)
     ▼
Redis (:6379)
     │  allow / deny
     ▼
Upstream Service

Key properties:

  • Consistent hashing routes each client key to a deterministic rate limiter node, minimizing Redis key scatter.
  • A circuit breaker around Redis protects the system during outages — the gateway fails open to preserve availability.
  • All rate limit decisions are atomic: a single Lua script performs the read-increment-expire cycle with no race conditions.

Tech Stack

Component Technology
Language Go 1.25
API Gateway Go net/http + Reverse Proxy
Rate Limiter Redis + Lua Scripts
RPC gRPC + Protocol Buffers
Observability Prometheus
Containers Docker + Docker Compose
Load Testing k6

Quick Start

Docker (recommended)

# Start the full stack (Redis, rate limiter, gateway, Prometheus)
docker compose up -d

# Verify all services are running
docker compose ps

Run Locally

# Start Redis
docker compose up -d redis

# Start the gRPC rate limiter server
go run ./cmd/server

# Start the HTTP API gateway
go run ./cmd/gateway

Project Structure

.
├── cmd/
│   ├── server/           # gRPC rate limiter entry point
│   ├── gateway/          # HTTP API gateway entry point
│   ├── client/           # CLI test client for the gRPC server
│   └── mockserver/       # Lightweight upstream mock for load testing
├── internal/
│   ├── limiter/          # Core rate limiting algorithms
│   │   ├── token_bucket.go
│   │   ├── sliding_window.go
│   │   ├── redis_limiter.go
│   │   ├── circuit_breaker.go
│   │   └── multi_tier.go
│   ├── gateway/          # Gateway logic and consistent hashing
│   ├── ratelimiter/      # gRPC server implementation
│   └── metrics/          # Prometheus metric definitions
├── proto/                # Protobuf service definition and generated stubs
├── tests/                # k6 load test scripts and results
├── grafana/              # Grafana dashboard and datasource provisioning
├── certs/                # TLS certificate generation script
├── Dockerfile.server
├── Dockerfile.gateway
├── docker-compose.yml
└── prometheus.yml

Configuration

All configuration is supplied via environment variables. Default values are suitable for local development.

Variable Default Description
REDIS_ADDR localhost:6379 Redis server address
GRPC_PORT 50051 gRPC rate limiter listen port
METRICS_PORT 9090 Prometheus metrics endpoint port
RATE_LIMIT 100 Maximum requests allowed per window
GRPC_ADDRS localhost:50051 Comma-separated list of rate limiter nodes (gateway)
UPSTREAM_URL http://localhost:9999 Backend service the gateway proxies to
HTTP_PORT 8081 Gateway HTTP listen port
TLS_CERT ./certs/server.crt Path to TLS certificate (omit to run insecure)
TLS_KEY ./certs/server.key Path to TLS private key

API Reference

Health Check

GET /health
{"status": "ok"}

Proxied Request (Rate Limited)

GET /get
X-API-Key: <client-key>

Rate limit decisions are made per X-API-Key. If the header is absent, the client IP address is used as the key.

Response headers on allowed requests:

X-RateLimit-Limit:     100
X-RateLimit-Remaining: 99
X-RateLimit-Reset:     60000

Response on rate limit exceeded (HTTP 429):

{"error": "rate_limit_exceeded"}

Additional headers on 429:

Retry-After: <seconds until window resets>
X-RateLimit-Limit: <configured limit>
X-RateLimit-Remaining: 0

Dashboard

GET /dashboard

Live HTML dashboard showing per-node metrics, circuit breaker state, and request throughput.

Metrics (Prometheus)

GET /api/metrics

Proxies the Prometheus metrics endpoint from the rate limiter node.


Rate Limiting Tiers

The multi-tier limiter enforces limits at three independent levels. A request is denied if any tier is exceeded.

Tier Default Limit Window Purpose
IP 100 requests 1 minute Prevents abuse from a single source
User 1,000 requests 1 minute Enforces per-account quotas
Global 100,000 requests 1 minute Protects total system capacity

Algorithms

Token Bucket

Tokens accumulate at a fixed rate up to a maximum burst capacity. Each request consumes one token. Ideal for endpoints that must support legitimate burst traffic.

Sliding Window

Tracks exact request timestamps within a rolling time window. Provides precise enforcement with no boundary artifacts, at the cost of higher memory usage per client.

Redis Lua (Distributed)

A Lua script atomically increments a Redis counter and sets its TTL in a single round trip. This is the algorithm used in production; it is the only approach that is globally consistent across multiple gateway instances.

Algorithm Comparison

Algorithm Latency Throughput Accuracy Best For
Token Bucket 65 ns/op ~15M req/s Approximate Single-node, high-throughput
Sliding Window 65 µs/op ~15K req/s Exact Strict per-window enforcement
Redis Lua ~130 µs/op ~7K req/s Exact Distributed, multi-node

Observability

Service Endpoints

Service URL
HTTP Gateway http://localhost:8081
Gateway Dashboard http://localhost:8081/dashboard
gRPC Rate Limiter localhost:50051
Prometheus Metrics http://localhost:9090/metrics
Prometheus UI http://localhost:9091

Prometheus Metrics

# Total rate limit decisions, by outcome and limit type
ratelimiter_decisions_total{decision="allowed|denied", limit_type="ip|user|global"}

# gRPC request latency histogram
ratelimiter_request_duration_seconds{method="CheckLimit"}

# Circuit breaker state (0 = closed, 1 = open, 2 = half-open)
ratelimiter_circuit_breaker_state{}

# Redis error count, by error category
ratelimiter_redis_errors_total{error_type="timeout|circuit_open|connection"}

# Gateway HTTP request count
gateway_requests_total{method, path, status}

# Gateway request latency histogram
gateway_request_duration_seconds{method, path}

Testing

# Run all unit tests
go test ./...

# Run benchmarks with memory allocation stats
go test -bench=. -benchmem ./internal/limiter/

# Run k6 load test
k6 run tests/load_test.js

Benchmark Results

BenchmarkTokenBucket_Allow-8      45,916,492        65 ns/op     0 B/op    0 allocs/op
BenchmarkSlidingWindow_Allow-8       136,651    65,060 ns/op     0 B/op    0 allocs/op
BenchmarkRedisLimiter_Allow-8         25,922   129,920 ns/op   690 B/op   22 allocs/op

Load Test Results (k6 — mock upstream)

Scenarios:
  normal_load:  10 VUs × 30 s
  spike:       200 VUs × 40 s

Results:
  Total requests:  ~38,327
  Throughput:      ~510 req/s
  p50 latency:       4.9 ms
  p95 latency:       9.7 ms   ← within the <10 ms SLA
  Max latency:     452 ms     (spike peak)

Load Test Results (k6 — external upstream)

Throughput:     86 req/s
Avg latency:   421 ms
p95 latency:   956 ms

The latency difference is entirely upstream network RTT; the gateway itself adds less than 1 ms overhead at these request rates.


Design Decisions

Why Token Bucket?

O(1) time and space complexity. Supports controlled burst traffic without background goroutines or scheduled cleanup. Appropriate for single-node scenarios where the small approximation error is acceptable.

Why Lua Scripts in Redis?

A Lua script executes atomically on the Redis server — no other command can interleave during its execution. This eliminates the check-then-act race condition that plagues MULTI/EXEC transactions, while also reducing round trips from two to one.

Why gRPC?

Protocol Buffer serialization is roughly 5× smaller and faster than JSON. gRPC provides built-in connection pooling, deadline propagation, and reflection for tooling — all relevant for a hot-path service.

Why a Circuit Breaker?

When Redis becomes unavailable, synchronous calls to it would block every incoming request until timeout. The circuit breaker trips after a configurable failure threshold, causing the gateway to fail open: requests are allowed through without rate limiting rather than being dropped. This trades strict limiting for availability — the right default for most public APIs.

Why Consistent Hashing?

Adding or removing rate limiter nodes remaps approximately 1/N of existing keys rather than all of them. This keeps Redis key locality stable during rolling deployments and autoscaling events.

Why Multi-tier Limiting?

A single global counter cannot distinguish between a single abusive client and a legitimate traffic surge. Layering IP, user, and global limits lets each tier enforce the right policy at the right granularity without any tier interfering with the others.


TLS

The server and gateway support mutual TLS. Set TLS_CERT and TLS_KEY to enable encrypted gRPC transport. If the certificate file is absent at startup, the server logs a warning and falls back to plaintext — intentionally permissive for local development, not suitable for production.

To generate a self-signed certificate for development:

bash certs/generate.sh

License

See LICENSE.md.

About

A production-grade distributed rate limiting system built with Go, gRPC, Redis, and Docker. Designed for high-throughput environments with strict latency requirements, it enforces per-IP, per-user, and global request quotas across horizontally-scaled nodes.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages