What if your PC could actually listen, understand, and do the work for you?

I’m building that idea into something real.

Meet AURA — my personal AI voice assistant for Windows.

Instead of building another chatbot that only answers questions, I’m experimenting with an AI agent that can actually interact with my computer and help with everyday work.

So far, AURA can:

• Have live voice conversations • Convert speech to text and respond with voice • Open applications like Excel, Power BI, Chrome, and Windows Settings • Search the web and YouTube through voice • Perform basic PC and file operations • Understand requests through Gemini • Execute actions through a safety-controlled automation layer • Continue listening after completing a task

But this is only the beginning.

My long-term goal is to make AURA understand my actual workspace — including what’s happening on my screen and inside applications — so I can simply say:

“Open my Excel file, check the sales data, clean the date column, and add a formula for total revenue.”

And let AURA handle the workflow.

I also want it to become useful for my own Data Analytics work — from Excel and Power BI workflows to repetitive desktop tasks, research, and automation.

I’m building AURA with Python, AI, voice technologies, desktop automation, Excel, Power BI, and LLMs, while keeping safety and human control at the core.

It’s still far from the final vision.

But that’s exactly what makes the project interesting.

I’m not just learning how AI agents work. I’m trying to build one that I can actually use every day.

Building in public. Learning by building. 🚀

#AI #AIAgents #Python #AIAssistant #VoiceAI #DesktopAutomation #DataAnalytics #PowerBI #Excel #Automation #BuildInPublic

The screen-reading step is where I would put the guardrail. Once AURA can read what is in the window, everything on that screen is input - and a page it opened from a voice search can hold a sentence written to be read as an instruction. I spent a while trying to rob my own...

The handoff is the part nobody tests.

Nearly every agent I read on here tonight hands the conversation to a person at some point. A WhatsApp bot that routes to a human on Telegram. An FAQ bot that escalates when it cannot answer. A run canvas with one amber row that stops before money moves. Good work, all of it.

The flowchart gets one arrow for that moment. Almost nobody writes down what happens after it.

Three ways it comes apart. I have caused the first two on my own benches; the third is just what an append does.

One. The human takes over and goes to lunch. Twenty minutes later the bot decides it is back on duty and answers mid-thread, in a voice the customer had already stopped talking to.

Two. The human replies on the other channel, the bot never sees it, and the follow-up sequence fires on schedule. Someone gets chased about a thing they already sorted.

Three. Two people book the same slot four seconds apart. An append has no lock, so both get a confirmation and you find out at the appointment.

None of these throw an error. Every step passes. That is what makes them expensive.

What fixed it for me was making ownership an explicit value on the conversation instead of something inferred from who spoke last: bot, human, or nobody. The bot may only speak while it holds ownership. A human reply takes it. It returns to the bot on a timer, and the customer is told when that happens rather than discovering it.

If you build these: what takes the conversation back, and does the customer know it happened?