40+ days of reverse engineering a game. At this point, I'm not sure what's harder to deal with: the engine or the AI.
I'm working on an accessibility mod for Resonance: A Plague Tale Legacy. The idea is to give visually impaired players something similar to the navigation assistance in The Last of Us Part II.
I thought it would be difficult. I didn't think I'd still be fighting the navigation system over 40 days later.
The game has no source code available, so pretty much everything has to be figured out through reverse engineering. I've managed to reconstruct the story progression and objectives across the game, which was already a ridiculous amount of work.
But knowing where the player needs to go is one thing. Figuring out how to actually guide them there using whatever the engine has loaded at that moment is another nightmare entirely.
And working with AI makes the whole experience even more interesting. Not always in a good way.
I use AI extensively for coding. Without it, I probably wouldn't have attempted something like this in the first place. But the longer the project goes on, the more time I spend making sure it doesn't undo its own work.
We've had working systems replaced with broken ones, the same problems investigated over and over, and fixes that somehow bring back bugs from weeks ago. Sometimes I spend an entire day just trying to get back to where things were before the latest "improvement."
And the funniest part? The AI will confidently explain why its new approach is definitely the right one. Again.
I'm not giving up on the project, though. I've already put way too much into it, and I genuinely want this accessibility system to work.
If anything, this experience has taught me that AI can help you build some pretty incredible things. Just don't expect it to remember what the hell you're building.
Everyone is shipping AI agents right now. Almost no one is testing them — and that is the whole problem.
A demo agent and a production agent are different animals. The demo works because you walked it down the happy path. Production breaks on the paths you never imagined.
A real example from my own work: a support assistant correctly handed a conversation to a human when someone wrote "I want to talk to a person" in Spanish — but not in English, because one phrase was missing from a rule. Every automated test passed. It only surfaced when I ran real conversations through it.
That is the gap the "agents will replace everything" wave keeps skipping. The hard part was never making an agent act. It is making it:
• say "I don't know" instead of guessing,
• hand control back to a human at the right moment,
• and prove, with tests, that it still behaves when the input gets strange.
Autonomy is cheap. Trustworthy autonomy is the real engineering.
Before you let an AI agent near real users or real data — what is the first thing you check?
Most AI startups don't have an AI problem. They have a systems engineering problem. And I suspect this distinction will become increasingly expensive to ignore.
We've reached an interesting point in the evolution of software. The intelligence available to developers is advancing faster than our ability to engineer reliable systems around it.
A sufficiently capable language model can reason through complex instructions, generate software, interpret documents, interact with tools and coordinate workflows.
Yet somehow, connecting that same model to an actual business process can produce a system that is fragile, unpredictable and extraordinarily difficult to maintain.
Why?
Because model capability and system reliability are fundamentally different engineering problems. A model can be remarkably intelligent while the system surrounding it remains architecturally incompetent.
Consider an AI agent that successfully processes 95% of customer requests. Impressive? Perhaps. Now consider what happens to the remaining 5%.
Does it recognise uncertainty? Does it escalate appropriately? Can it recover from partial execution? Does it maintain transactional integrity? Can its decisions be reconstructed? What prevents it from exceeding its authority?
And perhaps most importantly: Can you demonstrate that these properties hold beyond the handful of examples used in your product demonstration?
This is where the conversation becomes considerably more interesting. The problem is no longer simply whether a model possesses the capability to perform a task.
It is whether we can construct an environment in which that capability can be exercised predictably, economically and within explicitly defined boundaries.
This requires more than prompt engineering. It requires understanding distributed systems, state management, observability, evaluation, security boundaries, failure recovery and the economics of computation.
It requires thinking about AI systems as systems. Not as particularly impressive API calls. There's also a deeper architectural misunderstanding emerging across the industry.
Probabilistic intelligence does not eliminate the need for deterministic guarantees. It makes those guarantees more important. When software follows explicitly defined instructions, we can reason about its behaviour through established engineering methods.
When software delegates decisions to probabilistic components, the space of possible behaviours becomes substantially more difficult to characterise. That doesn't mean we should attempt to make intelligence deterministic. It means we need to engineer the surrounding infrastructure so that uncertainty does not become uncontrolled authority.
The distinction is subtle, but consequential. An agent deciding what action to propose is one problem. A system determining whether that agent is authorised to execute the action is another.
An agent generating an answer is one problem.A system establishing whether that answer is sufficiently supported to influence a consequential decision is another.
An agent completing a workflow is one problem. A system guaranteeing that a failed workflow doesn't leave the organisation in an inconsistent state is another.
These are not peripheral implementation details. They are the product. And this leads to a somewhat uncomfortable observation.
As foundational models become increasingly capable and accessible, raw intelligence becomes a less defensible source of differentiation for companies building on top of them.
The enduring advantage may instead come from everything surrounding that intelligence: The quality of the architecture. The precision of the evaluation system. The reliability of execution. The integration with real organisational processes. The ability to understand failures, recover from them and improve the system over time.
In other words, the competitive advantage shifts from merely accessing intelligence to engineering its application. I think we're entering a period where the ability to generate software will become increasingly commoditised, while the ability to reason rigorously about software systems becomes disproportionately valuable.
The companies that understand this distinction will have a meaningful advantage. Because the real question facing AI development is no longer just:
What can we make the model do? It's:
What can we prove the entire system will do, under which conditions, with what authority, at what cost, and what happens when it fails?
A successful demonstration establishes possibility.
Engineering establishes the conditions under which that possibility becomes dependable. And I suspect the next generation of serious AI companies will be defined by how well they understand the difference.