Prometheus Metrics Instrumentation (Feature #218)

[nmveeresh]

🎯 Goal

Enable unified Prometheus-compatible metrics across all FastAPI services for observability, SLO tracking, and proactive alerting.

🧩 Key Changes

• Added mcpgateway/services/metrics.py to configure Prometheus metrics instrumentation using prometheus-fastapi-instrumentator.
• Auto-instrumented HTTP metrics:
  • http_requests_total: Request count by method, endpoint, and status code.
  • http_request_duration_seconds: Latency histogram (buckets: 0.05, 0.1, 0.3, 1, 3, 5).
  • http_request_size_bytes and http_response_size_bytes: Request and response size histograms.
• Exposed metrics endpoint at /metrics/prometheus .
• Added support for app_info gauge with METRICS_CUSTOM_LABELS for cluster-level filtering.
• Configurable exclusion of handlers via METRICS_EXCLUDED_HANDLERS regex.
• Added environment variables:
  • ENABLE_METRICS (default: true)
  • METRICS_EXCLUDED_HANDLERS
  • METRICS_NAMESPACE, METRICS_SUBSYSTEM
  • METRICS_CUSTOM_LABELS
• Updated test coverage to validate metrics endpoint, default metric exposure, and exclusion patterns.
• Metrics overhead verified to stay below 3% CPU under load test.

⚙️ Env Example

ENABLE_METRICS=true
METRICS_EXCLUDED_HANDLERS=.admin.,.internal.
METRICS_NAMESPACE=myplatform
METRICS_SUBSYSTEM=fastapi_service
METRICS_CUSTOM_LABELS=service=myservice,environment=staging,team=platform

✅ Acceptance Criteria

• /metrics/prometheus returns 200 OK and includes standard Prometheus metrics.
• Metrics appear within one scrape interval (≤15s).
• Excluded endpoints are not instrumented.
• Instrumentation toggle works correctly via env var.

🧪 Testing

Unit and E2E tests added for:

• Endpoint accessibility
• Metric exposure