Is your feature request related to a problem? Please describe.
The OpenAI-compatible server owns one mutable cached model handle. Reset-by-default is safe, while GenieX-KeepCache: true retains state without a session identifier, parent revision, transcript-lineage check, or final commit record.
That is insufficient for multi-step agents that can prune history, retry after a malformed response, branch after planning, switch roles, or cancel an in-flight generation. After an append-only request, send a different continuation from an earlier transcript or interrupt generation and retry. The server currently cannot determine whether the retained state belongs to that request lineage, so a caller must either reset every request or trust mutable cache state blindly.
Describe the solution you'd like
Add an opt-in managed cache protocol for chat completions:
- a random session header and a parent revision returned by the last successful request;
- exact transcript-extension and full model/runtime identity checks before reuse;
- one pending transaction and one committed lineage for the single mutable model handle;
- commit only after successful generation;
- reset and clear state on branch, session switch, stale parent, cancellation, disconnect, or generation failure;
- final blocking or streaming metadata reporting
cold, reused, or reset and the committed revision;
- unchanged reset-by-default behavior for clients that do not opt in.
Session identifiers would separate cache lineage, not provide authentication or tenant isolation.
Describe alternatives you've considered
- Keep reset-by-default only: correct, but repeatedly evaluates an expanding prompt.
- Continue exposing raw
GenieX-KeepCache: useful for controlled append-only tests, but it leaves lineage correctness entirely to the caller.
- Save and restore file-based KV checkpoints: deferred because safe conversation-phase restoration is not yet established across both QAIRT and llama.cpp plugins.
Additional context
A focused implementation with deterministic revisions, branch/session/cancellation tests, streaming metadata, reset-on-error handling, and documentation is ready in samkwak188/GenieX:feat/managed-cache-lineage. No hardware performance claim is attached to this proposal; the immediate goal is cache correctness and a protocol that can be measured safely afterward.
Is your feature request related to a problem? Please describe.
The OpenAI-compatible server owns one mutable cached model handle. Reset-by-default is safe, while
GenieX-KeepCache: trueretains state without a session identifier, parent revision, transcript-lineage check, or final commit record.That is insufficient for multi-step agents that can prune history, retry after a malformed response, branch after planning, switch roles, or cancel an in-flight generation. After an append-only request, send a different continuation from an earlier transcript or interrupt generation and retry. The server currently cannot determine whether the retained state belongs to that request lineage, so a caller must either reset every request or trust mutable cache state blindly.
Describe the solution you'd like
Add an opt-in managed cache protocol for chat completions:
cold,reused, orresetand the committed revision;Session identifiers would separate cache lineage, not provide authentication or tenant isolation.
Describe alternatives you've considered
GenieX-KeepCache: useful for controlled append-only tests, but it leaves lineage correctness entirely to the caller.Additional context
A focused implementation with deterministic revisions, branch/session/cancellation tests, streaming metadata, reset-on-error handling, and documentation is ready in
samkwak188/GenieX:feat/managed-cache-lineage. No hardware performance claim is attached to this proposal; the immediate goal is cache correctness and a protocol that can be measured safely afterward.