A light weight vLLM simulator, for mocking out replicas.

llm-d benchmark scripts and tooling

Helm charts for llm-d

Simplified model deployment on llm-d

llm-d is a Kubernetes-native high-performance distributed LLM inference framework

Website for llm-d: This repository builds the website seen at llm-d.ai

Inference scheduler for llm-d

Distributed KV cache coordinator

Incubating P/D sidecar for llm-d