Inbound AI voice agent for a dental clinic

The clinic's phone rang while the administrator was with a patient. Calls went unanswered - and an unanswered call at a dental practice is a booking that goes somewhere else.

Now an AI agent answers that line. It speaks Latvian, Russian and English, and switches mid-call the way patients actually do. It checks live calendar availability, books the appointment, sends an SMS confirmation, and forwards anything unusual to the administrator by email. Every call is logged and transcribed.

Under two million people speak Latvian, and most speech engines mangle it. I benchmarked three speech-to-text stacks against real call recordings - Tilde, Gladia and Soniox - and the one that won wasn't offered natively by the platform. So I built a custom real-time WebSocket transcriber bridge and ran it on Fly.io.

The first working version answered in 5.2 seconds, which is long enough for a caller to assume the line is dead. Getting to 3.2s meant moving text-to-speech from OpenAI at roughly 1520ms to Azure Neural at roughly 470ms, then tuning endpointing and start-speaking thresholds against real recordings instead of accepting the defaults.

No speech engine reliably hears eleven digits over a phone line in Latvian. Phone numbers and national ID numbers are collected on the keypad instead, so the booking lands on the right patient rather than a plausible-looking wrong one.

About half of all calls were ending in dead silence. The model spoke its filler line - "one moment, let me check the calendar" - and then never executed the tool call. The transcripts looked clean, which is why it survived so long.

I found it by counting actual tool invocations in the call artifacts instead of trusting what the agent said it had done, and fixed it by changing the model and moving the filler to the platform's request-start hook, so speech and execution stopped sharing a path.

Running in production on the clinic's real phone number.