Component
Run
Desired use case or feature
The goal of this feature is to include easy to use, highly reproducible, instructions on how to use llm-d-benchmark with each one of the guides on llm-d.
Given llm-d-benchmark has itself capabilities to standup stacks and run workloads against it, the main goal of this feature is to allow only the latter (being the former done by simply following the guide)
Proposed solution
This will be carried out through several PRs, both on llm-d-benchmark and the llm-d repositories.
- PR 566 Benchmark runner for an existing stack (already merged on
llm-d-benchmark)
- PR 559 Simple benchmarking guide, towards benchmarking well-lit paths (
llm-d)
- PR 585 Configuration template for precise-prefix-cache-aware well-lit path benchmarking (
llm-d)
- PR 586 Pd template (
llm-d)
Alternatives
No response
Additional context or screenshots
No response