YouTube15 Sept 2026
15m

Tolan: Voice-First AI Companion — Paula Dozsa, Tolan

Podcast cover

AI Engineer

Building a voice-first AI companion requires prioritizing low latency and managing conversational volatility, as users frequently interrupt and switch topics. Maintaining a sub-two-second response time is critical for immersion, necessitating a tiered model architecture that routes high-stakes interactions to frontier models while using faster, cheaper models for casual turns. Effective memory management relies on a vector database that extracts and compresses facts, preferences, and emotional signals rather than maintaining a full transcript. Beyond the product itself, the team leverages a fleet of autonomous agents to automate code generation, peer review, and bug triage. This agentic workflow, which includes rigorous evaluation against production logs, has improved the app's crash-free rate to 99.9% and enabled rapid character development, demonstrating that managing AI agents requires many of the same organizational skills as managing human teams.

Outlines

Sign in to continue reading, translating and more.

Open full episode in Podwise