Any thoughts on supporting speculative generation while user is still active? goal is to reduce response latency as low as possible. This is possible and I've explored it in my recent paper - https://arxiv.org/abs/2606.13450.
This would rely on another turn-taking signal to continuously predict an upcoming end-of-turn. This would lead to much lower response latency than the current speculative framework. The audio will be played only when an actual end-of-turn is predicted.
open-source options for end-of-turn anticipation signal -
- Voice activity projection (https://github.com/inokoj/VAP-Realtime)
- Endpoint anticipation (https://huggingface.co/viks66/endpoint-anticipation)
- Kyutai/DSM VAD head 0/1 can signal an upcoming end of turn.
Happy to add a PR for it if there is interest!
Any thoughts on supporting speculative generation while user is still active? goal is to reduce response latency as low as possible. This is possible and I've explored it in my recent paper - https://arxiv.org/abs/2606.13450.
This would rely on another turn-taking signal to continuously predict an upcoming end-of-turn. This would lead to much lower response latency than the current speculative framework. The audio will be played only when an actual end-of-turn is predicted.
open-source options for end-of-turn anticipation signal -
Happy to add a PR for it if there is interest!