Skip to content

Make MODEL runtimes pluggable #2379

@ieaves

Description

@ieaves

Define a syntax that simplifies adding new runtimes.
Current runtimes:

llama.cpp
vllm
stable-diffusion
OpenVINO

Sub-issues

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions