Realtime Voice Agents with Frontier Intelligence — Bohan Li, EliseAI
Bohan Li from EliseAI presents techniques for AI voice agents that feel fast even when using slow but intelligent language models.
He borrows ideas from self-driving cars: transcription is perception, the language model is planning, and speech synthesis is control. For speed, he runs two transcribers in parallel — a fast one that responds immediately and a slower one that can correct errors. For speech synthesis, he uses a cache that checks if the same phrases were already generated before and can start playing them while the rest of the sentence is still being written, hiding any wait from the user.
Open on YouTube →
Vibekollen prepared this summary with AI from the original publication. The content belongs to AI Engineer.
