Skip to content

Conversation

SimBe195
Copy link
Collaborator

This adds a new LM class OnnxStatelessLm. This is a simple ONNX-based Language Model without any state caching. The entire token history is fed into the ONNX model for each score request. This trades efficiency for simplicity and flexibility. Thus, it is mostly useful for prototyping and models with a relatively small search space.

@SimBe195 SimBe195 requested review from curufinwe and larissakl May 27, 2025 14:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant