Skip to content

Preserve chat turn order during non-interrupting back-to-back speech #454

Description

@andimarafioti

Problem

When interrupt_response=False, a second transcription can be appended to Chat while the first response is still generating. The first response then appends its assistant history after that newer user item.

The resulting order is:

user A -> user B -> assistant A

The queued response for B therefore receives the same incorrectly ordered provider snapshot.

Reproduction

  1. Start generation for transcript A.
  2. While A is still generating, deliver non-interrupting speech/transcription B.
  3. Let A finish and commit its assistant response.
  4. Inspect Chat.buffer or the provider input for B.

This reproduces on unmodified main and affects both local/OpenAI-compatible history paths and both realtime transports.

Additional provisional-history failure mode

With #453's provisional response history, the same overlap can also make rollback incomplete:

  1. Response A writes provisional assistant output.
  2. Non-interrupting turn B arrives before A's delivery is final.
  3. trim_if_needed() compacts the buffer and summarizes A's provisional output, replacing its tracked item IDs.
  4. A is cancelled, so rollback removes the original IDs but cannot remove A's content from the summary.

A fix must keep turn order correct and prevent compaction from absorbing provisional items that may still be rolled back.

Expected behavior

Conversation history and the provider input for B should preserve turn order:

user A -> assistant A -> user B

Cancelling A must also remove all of A's provisional content, including when the chat exceeds its compaction threshold.

Scope note

The base ordering race was found while reviewing #453 but is pre-existing on main, so it is tracked separately rather than expanding that PR. The provisional-compaction variant depends on that same overlapping-turn race and belongs in this fix as well.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions