feat(logging): improve event logging and add bot thinking display #1434
Conversation
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
There was a problem hiding this comment.
Looks good, just a few small comments.
BEFORE MERGING: Could you run a local integration test with all the reasoning-models we claim to support? Copy-and-paste the output into the description as a test-plan. We need to check this new approach works with production models
| bot_thinking_logs = [ | ||
| record | ||
| for record in caplog.records | ||
| if "Event :: BotThinking" in record.message |
There was a problem hiding this comment.
nit: Should we define an enum with these strings if there are a static set of events? It would be more robust to typos
There was a problem hiding this comment.
I think for the tests it makes sense to use the actual value, but for the implementation I agree, we must refactor all the events that we are using as these are mostly strings scattered all over the codebase.
| for event in next_events: | ||
| if event["type"] != "EventHistoryUpdate": | ||
| event_type = event["type"] | ||
| log.info( |
There was a problem hiding this comment.
nit: Should this be a log.debug() rather than log.info()? How much noise would this make in logs?
There was a problem hiding this comment.
| if event_name == "BotThinking": | ||
| title = f"[blue]{title}[/] [yellow bold]{event_name}[/]" | ||
| elif event_name.startswith("Start") and event_name.endswith("Action"): | ||
| title = f"[magenta][bold]Start[/]{event_name[5:]}[/]" |
There was a problem hiding this comment.
Where does the magic number 5 come from here?
There was a problem hiding this comment.
we actually have these numbers after line 120. I'm going to revert these changes as it just helped me to debug while looking at the events having different colors.
| elif event_name.startswith("Start") and event_name.endswith("Action"): | ||
| title = f"[magenta][bold]Start[/]{event_name[5:]}[/]" | ||
| elif event_name.endswith("ActionFinished"): | ||
| title = f"[magenta]{event_name[:-8]}[bold]Finished[/][/]" |
There was a problem hiding this comment.
Another magic number here, is there a danger we add longer-named events and this breaks?
There was a problem hiding this comment.
we actually have these numbers after line 120. I'm going to revert these changes as it just helped me to debug while looking at the events having different colors.
Stack Info
This PR is part of a stack:
#1427 → #1431 → #1432
Description
Enhanced event logging in the runtime and callbacks to better display bot thinking events with reasoning traces. Moved event logging to occur after event generation for proper ordering, added reasoning content extraction from LLM completions, and improved console output formatting for different event types.
Changes