Commit 7240cdb
committed
fix(models): surface empty LiteLlm streaming completions as error event
Streaming completions where the provider returns a finish_reason but
no text + no tool calls currently produce ZERO yielded LlmResponse
events: ``aggregated_llm_response`` only gets set when
``(text or reasoning_parts)`` is truthy, and ``aggregated_llm_response_with_tool_call``
needs a function_call. With neither, the loop simply exits and the
downstream Runner observes a silent successful empty stream.
This pattern is reported across multiple stalled fix attempts:
* #5394 — AnthropicLlm never populates finish_reason on LlmResponse
* #5006 — retry with resume message when model returns empty response
* #5636 — surface error when model returns STOP with empty content
* #3618 / #3699 — Handle empty message in LiteLLM response
It hits providers under several real conditions: anthropic
content_filter, gemini 2.5-flash-lite STOP-with-empty after tool calls,
0-token completions under safety, model_not_found responses normalized
to stop, etc. From the user's perspective the agent "successfully" ends
a turn with no visible output.
Fix
- Track ``last_finish_reason`` + ``last_model_version`` across the
stream so we can attribute the empty response.
- After both ``aggregated_llm_response`` and
``aggregated_llm_response_with_tool_call`` checks, if BOTH are None
AND a finish_reason was observed, yield ONE LlmResponse with
``error_code`` set to the mapped finish_reason, ``error_message``
describing the failure mode, and the provider's ``model_version``
preserved. ``usage_metadata`` + ``grounding_metadata`` (if any)
attach to that response so callers do not lose them.
- Minimum-surface change: the guard only fires when the stream
produced no aggregated response AND a finish_reason was observed.
Streams that genuinely yield nothing (test doubles, empty
iterators) stay byte-identical.
Tests
- tests/unittests/models/test_litellm.py adds 4 cases:
* content_filter-empty → surfaces with SAFETY error_code
* stop-empty → surfaces with STOP finish_reason + error_message
* normal text stream → empty-guard does NOT fire (regression)
* literally-empty stream (no chunks, no finish_reason) →
byte-identical zero responses
281 lite_llm tests pass + 1 skip; 0 regressions.1 parent 8c9fff8 commit 7240cdb
2 files changed
Lines changed: 193 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2413 | 2413 | | |
2414 | 2414 | | |
2415 | 2415 | | |
| 2416 | + | |
| 2417 | + | |
| 2418 | + | |
| 2419 | + | |
| 2420 | + | |
| 2421 | + | |
| 2422 | + | |
| 2423 | + | |
| 2424 | + | |
| 2425 | + | |
2416 | 2426 | | |
2417 | 2427 | | |
2418 | 2428 | | |
| |||
2502 | 2512 | | |
2503 | 2513 | | |
2504 | 2514 | | |
| 2515 | + | |
| 2516 | + | |
2505 | 2517 | | |
| 2518 | + | |
| 2519 | + | |
2506 | 2520 | | |
2507 | 2521 | | |
2508 | 2522 | | |
| |||
2616 | 2630 | | |
2617 | 2631 | | |
2618 | 2632 | | |
| 2633 | + | |
| 2634 | + | |
| 2635 | + | |
| 2636 | + | |
| 2637 | + | |
| 2638 | + | |
| 2639 | + | |
| 2640 | + | |
| 2641 | + | |
| 2642 | + | |
| 2643 | + | |
| 2644 | + | |
| 2645 | + | |
| 2646 | + | |
| 2647 | + | |
| 2648 | + | |
| 2649 | + | |
| 2650 | + | |
| 2651 | + | |
| 2652 | + | |
| 2653 | + | |
| 2654 | + | |
| 2655 | + | |
| 2656 | + | |
| 2657 | + | |
| 2658 | + | |
| 2659 | + | |
| 2660 | + | |
| 2661 | + | |
| 2662 | + | |
| 2663 | + | |
| 2664 | + | |
| 2665 | + | |
| 2666 | + | |
| 2667 | + | |
2619 | 2668 | | |
2620 | 2669 | | |
2621 | 2670 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5275 | 5275 | | |
5276 | 5276 | | |
5277 | 5277 | | |
| 5278 | + | |
| 5279 | + | |
| 5280 | + | |
| 5281 | + | |
| 5282 | + | |
| 5283 | + | |
| 5284 | + | |
| 5285 | + | |
| 5286 | + | |
| 5287 | + | |
| 5288 | + | |
| 5289 | + | |
| 5290 | + | |
| 5291 | + | |
| 5292 | + | |
| 5293 | + | |
| 5294 | + | |
| 5295 | + | |
| 5296 | + | |
| 5297 | + | |
| 5298 | + | |
| 5299 | + | |
| 5300 | + | |
| 5301 | + | |
| 5302 | + | |
| 5303 | + | |
| 5304 | + | |
| 5305 | + | |
| 5306 | + | |
| 5307 | + | |
| 5308 | + | |
| 5309 | + | |
| 5310 | + | |
| 5311 | + | |
| 5312 | + | |
| 5313 | + | |
| 5314 | + | |
| 5315 | + | |
| 5316 | + | |
| 5317 | + | |
| 5318 | + | |
| 5319 | + | |
| 5320 | + | |
| 5321 | + | |
| 5322 | + | |
| 5323 | + | |
| 5324 | + | |
| 5325 | + | |
| 5326 | + | |
| 5327 | + | |
| 5328 | + | |
| 5329 | + | |
| 5330 | + | |
| 5331 | + | |
| 5332 | + | |
| 5333 | + | |
| 5334 | + | |
| 5335 | + | |
| 5336 | + | |
| 5337 | + | |
| 5338 | + | |
| 5339 | + | |
| 5340 | + | |
| 5341 | + | |
| 5342 | + | |
| 5343 | + | |
| 5344 | + | |
| 5345 | + | |
| 5346 | + | |
| 5347 | + | |
| 5348 | + | |
| 5349 | + | |
| 5350 | + | |
| 5351 | + | |
| 5352 | + | |
| 5353 | + | |
| 5354 | + | |
| 5355 | + | |
| 5356 | + | |
| 5357 | + | |
| 5358 | + | |
| 5359 | + | |
| 5360 | + | |
| 5361 | + | |
| 5362 | + | |
| 5363 | + | |
| 5364 | + | |
| 5365 | + | |
| 5366 | + | |
| 5367 | + | |
| 5368 | + | |
| 5369 | + | |
| 5370 | + | |
| 5371 | + | |
| 5372 | + | |
| 5373 | + | |
| 5374 | + | |
| 5375 | + | |
| 5376 | + | |
| 5377 | + | |
| 5378 | + | |
| 5379 | + | |
| 5380 | + | |
| 5381 | + | |
| 5382 | + | |
| 5383 | + | |
| 5384 | + | |
| 5385 | + | |
| 5386 | + | |
| 5387 | + | |
| 5388 | + | |
| 5389 | + | |
| 5390 | + | |
| 5391 | + | |
| 5392 | + | |
| 5393 | + | |
| 5394 | + | |
| 5395 | + | |
| 5396 | + | |
| 5397 | + | |
| 5398 | + | |
| 5399 | + | |
| 5400 | + | |
| 5401 | + | |
| 5402 | + | |
| 5403 | + | |
| 5404 | + | |
| 5405 | + | |
| 5406 | + | |
| 5407 | + | |
| 5408 | + | |
| 5409 | + | |
| 5410 | + | |
| 5411 | + | |
| 5412 | + | |
| 5413 | + | |
| 5414 | + | |
| 5415 | + | |
| 5416 | + | |
| 5417 | + | |
| 5418 | + | |
| 5419 | + | |
| 5420 | + | |
| 5421 | + | |
0 commit comments