๐—›๐—ฒ๐—ฎ๐—ฑ๐—น๐—ถ๐—ป๐—ฒ: Most Voice AI agents still feel like glorified walkie-talkies. So I built one t...๐—›๐—ฒ๐—ฎ๐—ฑ๐—น๐—ถ๐—ป๐—ฒ: Most Voice AI agents still feel like glorified walkie-talkies. So I built one t...
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Vimal's avatar
โ€ข 9h
๐—›๐—ฒ๐—ฎ๐—ฑ๐—น๐—ถ๐—ป๐—ฒ: Most Voice AI agents still feel like glorified walkie-talkies. So I built one that actually listens like a human. Most voice bots today follow a rigid, linear loop: You speak โž” Wait โž” Bot speaks. If you dare interrupt them mid-sentence? They either completely ignore you or their context buffer gets totally mangled. ๐—ง๐—ผ ๐˜€๐—ผ๐—น๐˜ƒ๐—ฒ ๐˜๐—ต๐—ถ๐˜€, I engineered Astra Duplex Agent โ€” a sub-50ms, ultra-low latency, full-duplex conversational Voice AI built to handle natural human interruptions seamlessly. Here is what went into building the architecture under the hood: True Full-Duplex WebSockets: Built on FastAPI, allowing real-time bi-directional audio streaming instead of turn-based HTTP requests. Edge Voice Activity Detection (VAD): Deployed Silero VAD using ONNX Runtime to catch user barge-ins at the edge with near-zero overhead. LangGraph State Machine: Instead of a simple monolithic script, the conversation logic is modeled as a state graph, giving precise control over execution flow. Partial State Tracking & Memory Persistence: This was the trickiest part. When an interruption happens, Astra doesn't just stop audio playback โ€” an async kill-switch halts Groq's LLaMA-3.1 mid-token, calculates the exact partial sentence actually spoken by the TTS, and saves only that partial context to an Upstash Redis buffer. Sub-50ms Inference & Streaming: Leveraged Groq LPU (LLaMA-3.1 + Whisper Large v3) paired with ElevenLabs Turbo v2.5 streaming. The result? An agent that you can interrupt mid-thought, change topics with on the fly, and ask follow-up questions without it losing historical context. Containerized with Docker and live on Render & Vercel. Check out the full architecture & live demo below! https://lnkd.in/eC6yak3i https://lnkd.in/eN38HGZD #VoiceAI #GenerativeAI #LangGraph #SystemDesign #FastAPI #MachineLearning #WebSockets #Python #AIENGINEERING
Back to feed
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started