Skip to content

feature: realtime agents#368

Open
bonk1t wants to merge 1 commit intomainfrom
docs-realtime-agents
Open

feature: realtime agents#368
bonk1t wants to merge 1 commit intomainfrom
docs-realtime-agents

Conversation

@bonk1t
Copy link
Collaborator

@bonk1t bonk1t commented Sep 25, 2025

  • Add realtime orchestration entry points so agencies can run interactive sessions.
  • Package fully working demos: hosted web UI stack, Twilio bridge, and interactive example under examples/interactive/realtime/.
  • Document the new capability in docs/additional-features/, update site metadata/images, and surface the flow from the examples README.
  • Cover the behavior with tests/integration/realtime/test_realtime.py to lock in the realtime pipeline.

Note

Introduce realtime voice agents with a FastAPI websocket bridge, voice-aware agents, packaged web/Twilio demos, docs, and tests.

  • Core runtime (realtime):
    • Add RealtimeAgency/RealtimeAgent wrappers and Agency.to_realtime() for OpenAI Realtime sessions.
    • New integrations.run_realtime FastAPI bridge with /realtime websocket, voice switching, turn detection, audio formats, Twilio media-stream support.
    • Agent voice support (voice param, allowed voices) and agency-level voice randomization (randomize_agent_voices, voice_random_seed).
  • FastAPI:
    • run_fastapi can mount /your_agency/realtime websockets via enable_realtime + realtime_options.
  • UI/Demos:
    • Packaged browser realtime demo (FastAPI app + WebAudio client) and examples/interactive/realtime/demo.py launcher.
    • Twilio phone bridge demo with media stream handler and README.
  • Docs:
    • New Voice Agents docs (Overview, Deployment); update FastAPI integration and API reference; docs navigation updated.
  • Exports:
    • Re-export run_realtime and realtime modules from package.
  • Tests:
    • Integration tests for realtime orchestration and FastAPI /realtime endpoint; agent/agency voice features.

Written by Cursor Bugbot for commit d5ee450. This will update automatically on new commits. Configure here.

@bonk1t bonk1t force-pushed the docs-realtime-agents branch from 9966bec to 9470f54 Compare October 12, 2025 02:49
@bonk1t bonk1t changed the title feat: add realtime agents documentation Realtime Agents Oct 13, 2025
@bonk1t bonk1t changed the title Realtime Agents feature: add packaged realtime demo launcher Oct 13, 2025
@bonk1t bonk1t requested a review from ArtemShatokhin October 13, 2025 02:28
@bonk1t bonk1t changed the title feature: add packaged realtime demo launcher feature: package realtime demo stack Oct 13, 2025
Copy link
Collaborator

@ArtemShatokhin ArtemShatokhin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works well, the only minor issue I could find is that when sending images, the text response is not updated in real time and after a bit of delay, it gets printed in full right away.

@bonk1t bonk1t changed the title feature: package realtime demo stack feature: realtime agents Oct 13, 2025
@bonk1t bonk1t force-pushed the docs-realtime-agents branch from c297f4c to 8c4e60a Compare October 14, 2025 22:36
@bonk1t bonk1t marked this pull request as draft October 18, 2025 02:49
@bonk1t bonk1t force-pushed the docs-realtime-agents branch from bf0a06e to 84dd880 Compare October 20, 2025 01:18
@bonk1t bonk1t marked this pull request as ready for review October 20, 2025 01:18
@bonk1t bonk1t requested a review from ArtemShatokhin October 20, 2025 01:18
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@bonk1t bonk1t reopened this Dec 5, 2025
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the final PR Bugbot will review for you during this billing cycle

Your free Bugbot reviews will reset on December 27

Details

Your team is on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle for each member of your team.

To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

@github-actions github-actions bot removed the stale label Dec 6, 2025
- add realtime session bridge with per-agent voice support
- mount /{agency}/realtime websocket routes via run_fastapi
- ship packaged browser + Twilio realtime demos
- document voice agents and FastAPI integration options
- add regression tests for voice config and realtime endpoints
@bonk1t
Copy link
Collaborator Author

bonk1t commented Dec 27, 2025

@codex review

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +103 to +105
async connect() {
try {
this.ws = new WebSocket(`ws://localhost:8000/ws/${this.sessionId}`);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Use server host/port when opening realtime demo websocket

The realtime demo UI hard-codes ws://localhost:8000/ws/${this.sessionId} for its websocket, so the browser always dials the local machine on port 8000 regardless of how RealtimeDemoLauncher.start was configured. Running the demo on a different port, remote host, or behind HTTPS will make the frontend fail to connect to the backend because it points to the wrong origin. Build the websocket URL from window.location (or a relative path) so it follows the actual server host/port/protocol.

Useful? React with 👍 / 👎.

@github-actions github-actions bot removed the stale label Dec 28, 2025
@github-actions github-actions bot added the stale label Jan 8, 2026
Repository owner deleted a comment from github-actions bot Jan 8, 2026
Repository owner deleted a comment from github-actions bot Jan 8, 2026
Repository owner deleted a comment from github-actions bot Jan 8, 2026
Repository owner deleted a comment from github-actions bot Jan 8, 2026
Repository owner deleted a comment from github-actions bot Jan 8, 2026
Repository owner deleted a comment from github-actions bot Jan 8, 2026
@bonk1t bonk1t removed the stale label Jan 8, 2026
@github-actions
Copy link
Contributor

This PR is stale because it has been open for 10 days with no activity.

@github-actions github-actions bot added the stale label Jan 19, 2026
@github-actions
Copy link
Contributor

This PR was closed because it has been inactive for 7 days since being marked as stale.

@github-actions github-actions bot closed this Jan 26, 2026
@bonk1t bonk1t reopened this Jan 26, 2026
@github-actions github-actions bot removed the stale label Jan 27, 2026
@github-actions
Copy link
Contributor

github-actions bot commented Feb 6, 2026

This PR is stale because it has been open for 10 days with no activity.

@github-actions github-actions bot added the stale label Feb 6, 2026
@github-actions
Copy link
Contributor

This PR was closed because it has been inactive for 7 days since being marked as stale.

@github-actions github-actions bot closed this Feb 13, 2026
@bonk1t bonk1t reopened this Feb 13, 2026
@github-actions github-actions bot removed the stale label Feb 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

Comments