Building a Voice AI Assistant That Turns Phone Calls Into Thoughtful Follow‑Through
I built a Django backend for a family-focused AI phone assistant. A registered caller connects through Twilio, and Django Channels streams the call to an OpenAI Realtime session for a natural, two-way conversation.
The backend adds caller-specific context, selected voices, subscription minute limits, and interruption handling. After the call, background tasks generate a separate summary. A follow-up email is sent only when requested and the recipient is confirmed.
I also built APIs for managing relations, subscriptions, and dashboard usage data. One of the most interesting parts was connecting the whole lifecycle: incoming call, real-time conversation, usage tracking, and post-call follow-through.
Sending the follow-up email only after the recipient is confirmed is a careful rule: a wrong name heard on a call could send it to the wrong person. How does the confirmation work: does the assistant repeat the name during the call, or does the caller confirm later in the dashboard?
Grouping settings by the task someone is trying to finish makes sense here. Keeping delivery separate from notifications is a nice detail: sending a document and tracking it are two different jobs.