Skip to content

fix: track real message count since last summary - #5642

Merged
markbackman merged 3 commits into
pipecat-ai:mainfrom
pradneshfernandez:context-summarizer-fix
Sep 6, 2026
Merged

fix: track real message count since last summary#5642
markbackman merged 3 commits into
pipecat-ai:mainfrom
pradneshfernandez:context-summarizer-fix

Conversation

@pradneshfernandez

Copy link
Copy Markdown
Contributor

Fixes #5504

Summary

LLMContextSummarizer._should_summarize() was calculating unsummarized messages using len(context.messages) - 1. The issue was that this calculation never reset after running a summary, and it blindly assumed a leading system message was always present.

Because of this, max_unsummarized_messages was effectively evaluating total context size rather than new messages. It also caused context summarization to loop continuously, as the newly injected summary message was counted as a fresh turn and immediately triggered another summary.

Fix

Following the fix suggested in #5504:

  • Replaced the hardcoded - 1 with a _messages_since_summary_baseline marker to track state accurately.
  • Updated _apply_summary() to reset the baseline to num_system_preserved + 1, accounting for both the preserved system message and the newly added summary message.
  • Updated _should_summarize() to compute messages_since_summary against this new baseline instead of len(messages) - 1.

Note: This PR only addresses the message-count trigger. The related token-trigger issue (#5505) is intentionally left out and will be handled separately.

Test plan

  • uv run pytest tests/test_llm_context_summarizer.py tests/test_context_summarization.py
  • Verified manually that summaries no longer re-trigger immediately after running.

@pradneshfernandez pradneshfernandez changed the title Track real message count since last summary fix: track real message count since last summary Sep 5, 2026
The message-count trigger excludes the leading system message and the
summary injected by the last applied summary. Both are derived from the
context at check time, with the summary matched by identity, so replacing
the messages resets the count instead of leaving a stale offset.
@markbackman

markbackman commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Thanks for the fix. I pushed a commit on top (bc49297) that keeps your approach but changes when the count is computed.

Your version cached the number of excluded leading messages in _messages_since_summary_baseline, set once in __init__ and again in _apply_summary. That number goes stale in two cases: the app replaces the context after a summary (via set_messages or LLMMessagesUpdateFrame), or a system message is added after the summarizer was constructed.

The commit replaces the cached int with a reference to the summary message the summarizer injected (_summary_message), and a new _messages_since_summary() that computes the count from the current context each time _should_summarize runs. It excludes messages[0] if it is a system message, and the next slot if it is the same object as _summary_message. Matching by identity means a reset context naturally drops the summary from the count, and a user message that happens to look like a summary is not mistaken for one.

I also added three tests in tests/test_llm_context_summarizer.py: the threshold fires at exactly N messages with and without a leading system message, only messages added since a summary count, and replacing the context resets the count. The last one fails against the cached-int version.

@markbackman markbackman left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for the contribution.

@markbackman
markbackman merged commit 1f8a513 into pipecat-ai:main Sep 6, 2026
5 checks passed
@codecov

codecov Bot commented Sep 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Files with missing lines Coverage Δ
...t/processors/aggregators/llm_context_summarizer.py 96.85% <100.00%> (+0.23%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@pradneshfernandez
pradneshfernandez deleted the context-summarizer-fix branch September 8, 2026 18:29
@pradneshfernandez
pradneshfernandez restored the context-summarizer-fix branch September 8, 2026 18:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LLMContextSummarizer: messages_since_summary never resets and counts the injected summary message

2 participants