This page is currently in progress. We are filling out the recipe names and descriptions incrementally, however, you can find all the sample recipe JSON in the files in this folder.
You may use any recipe json as the payload in the /deployment endpoint. You are free to change any parameter as you would like, however, the parameters chosen have been tested and validated. Any changes you make to these sample recipes, we can not guarantee it's efficiency or correctness.
| Recipe Name | Type | File Name | Description |
|---|---|---|---|
| Autoscaling | Inference | autoscaling_recipe.json | vLLM inference with autoscaling and Meta Lllama 3 70B instruct model |