I built a 3D AI academic advisor that talks back in real-time — here's how it works.
Most AI chatbots are boring text boxes. I wanted to build something that actually feels alive.
Meet TUF Avatar Talk — a 3D animated academic advisor with real-time voice conversation, lip-sync, and natural responses powered by Google Gemini.
The stack:
→ React Three Fiber for the 3D avatar
→ FastAPI backend with WebSockets for real-time streaming
→ Google Gemini for conversation
→ Custom lip-sync engine tied to audio output
What it does:
You speak → it listens → Gemini thinks → the avatar responds with synced lip movement and voice, all in real-time. No reload, no waiting.
The hardest part wasn't the AI — it was getting the latency low enough that the conversation feels natural. WebSockets streaming + chunked audio playback was the answer.
Live demo: tuf-avatar-talk.vercel.app (http://tuf-avatar-talk.vercel.app)
I build full-stack AI products like this for founders and teams. If you've got an AI idea sitting in your Notion that needs to actually get shipped, my services are live on my profile 👇