Optimizing EU-Hosted Voice Agent Latency Without Changing ModelsOptimizing EU-Hosted Voice Agent Latency Without Changing Models
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Back to feedPost
Alireza's avatar
Alireza Mohagheghi
• Oct 9
My voice agent's median response time went from 1.7 s to 1.08 s without changing the model, with every part of the stack running in Europe.
I'm building an AI receptionist that answers phone calls for businesses in Cyprus in Greek, English and Russian, and books appointments into their calendars.
The hard constraint is data residency. Speech-to-text, the LLM, text-to-speech and all customer data have to be processed in the EU. That rules out a lot of the usual choices. Some models that work really well on phone calls I couldn't find from a good EU provider at all.
So I benchmarked every model with tool support I could get from EU providers, about 40 of them, using the real receptionist prompt in Greek, Russian and English. Only one passed every check, faithful to the business facts, right language, right register.
That choice has a cost, and you can see it in the budget. When I first measured every stage, the LLM was about a third of the wait. After fixing everything around it, it's more than half.
Telephony and network - 100-150 ms
End-of-turn detection - 200-250 ms
LLM first token - 550-600 ms
TTS first audio - 100-120 ms
What made the difference.
1. Keeping the model connection warm. Node's fetch was closing idle connections after about 4 seconds, so most turns paid for a full TCP and TLS handshake. Keeping it open saved about 145 ms per request.
2. Hedging slow requests. The first token usually lands in about 0.5 s, but can spike past 2 s when the provider is congested. If nothing arrives within 1 s, the same request goes out again and the first answer wins. If the model stalls completely, a second model takes over.
3. Fillers only when they're needed. "One moment, let me check" plays only if a calendar lookup takes longer than 350 ms. Fast lookups add no spoken delay.
4. One fix I rejected on purpose. Waiting 500 ms longer before answering halved how often the agent cut callers off mid-sentence, but it added half a second to every single response.
Most calls now land between 0.9 and 1.2 s.
Under EU residency you can't just pick the fastest model on the market, so everything around the model has to be tight. Profile the whole turn before you blame the model.
If you're building voice agents under EU data residency, what first-token times are you seeing from your LLM?
voiceAI Agent DevelopmentConversational AI Development
Post image
Back to feed
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started