𝗛𝗲𝗮𝗱𝗹𝗶𝗻𝗲: Most Voice AI agents still feel like glorified walkie-talkies. So I built one that actually listens like a human.
Most voice bots today follow a rigid, linear loop: You speak ➔ Wait ➔ Bot speaks. If you dare interrupt them mid-sentence? They either completely ignore you or their context buffer gets totally mangled.
𝗧𝗼 𝘀𝗼𝗹𝘃𝗲 𝘁𝗵𝗶𝘀, I engineered Astra Duplex Agent — a sub-50ms, ultra-low latency, full-duplex conversational Voice AI built to handle natural human interruptions seamlessly.
Here is what went into building the architecture under the hood:
True Full-Duplex WebSockets: Built on FastAPI, allowing real-time bi-directional audio streaming instead of turn-based HTTP requests.
Edge Voice Activity Detection (VAD): Deployed Silero VAD using ONNX Runtime to catch user barge-ins at the edge with near-zero overhead.
LangGraph State Machine: Instead of a simple monolithic script, the conversation logic is modeled as a state graph, giving precise control over execution flow.
Partial State Tracking & Memory Persistence: This was the trickiest part. When an interruption happens, Astra doesn't just stop audio playback — an async kill-switch halts Groq's LLaMA-3.1 mid-token, calculates the exact partial sentence actually spoken by the TTS, and saves only that partial context to an Upstash Redis buffer.
Sub-50ms Inference & Streaming: Leveraged Groq LPU (LLaMA-3.1 + Whisper Large v3) paired with ElevenLabs Turbo v2.5 streaming.
The result? An agent that you can interrupt mid-thought, change topics with on the fly, and ask follow-up questions without it losing historical context.
Containerized with Docker and live on Render & Vercel.
Check out the full architecture & live demo below!
https://lnkd.in/eC6yak3i
https://lnkd.in/eN38HGZD
#VoiceAI #GenerativeAI #LangGraph #SystemDesign #FastAPI #MachineLearning #WebSockets #Python #AIENGINEERING