AI Risk Verification Layer for Deepfake Fraud Prevention

In January 2024, a finance employee at engineering firm Arup joined a video call with five colleagues and authorized 15 wire transfers totaling $25.6M. Every other participant on that call was an AI-generated deepfake.

Ferrari, WPP, LastPass, and Wiz all faced the same style of attack that year: a cloned executive voice or video asking for an urgent payment. And all four caught it. Not with software. With a person noticing something felt off. Ferrari's executive asked about a book the real CEO had recommended just days earlier, and the caller went quiet. LastPass's employee just didn't buy the channel or the urgency. Wiz's clone came from a recording where the CEO sounded slightly different than usual, and people picked up on it right away.

The scale behind these stories is what makes this more than a few unlucky companies. Deepfake-enabled voice phishing rose roughly 1,600% between Q4 2024 and Q1 2025. Over half of surveyed organizations reported financial losses tied to deepfake or AI voice fraud in the past year. The average confirmed loss per incident is over $280,000, and close to 20% of affected organizations lost more than $500K in a single incident. Deloitte projects AI-enabled fraud could hit roughly $40B a year by 2027. Fewer than 5% of victims even report it, so the real number is almost certainly worse than what's public.

This attack targets executives directly. It's not phishing a junior employee's inbox. It also bypasses cybersecurity entirely. No network gets breached, no malware gets installed. It's just a phone call or a video conference exploiting trust and authority, which puts it in finance and operations, not IT.

Every catch above happened because someone improvised well in the moment, not because a system caught it. That's not something you can keep counting on as the technology keeps improving. And a single incident like Arup's can move a mid-size company's financials and bring the kind of press coverage that damages trust regardless of the dollar amount.

Instead of racing to build a better deepfake detector, an arms race you're always a step behind in, score every high-value, executive-attributed request on how well it matches known, safe patterns. Anything that doesn't match gets routed through a channel a deepfake can't reach.

The score looks at channel consistency (is this coming through a pre-registered channel, or somewhere new), request novelty (a new payee or bank account is the single strongest signal across every documented incident), urgency and pressure language, how much the request deviates from the executive's normal approval pattern, and media risk (a live voice or video call is riskier than a request through an authenticated system).

Based on that score:

A low score, 0 to 39, proceeds normally with no added friction.

A medium score, 40 to 69, triggers a mandatory callback to a pre-registered secondary channel before anything moves.

A high score, 70 to 100, gets held for dual human authorization plus a live, unscripted question. That last step is exactly what caught every thwarted attempt in the research.

Detection tools that analyze audio or video for signs it's synthetic are genuinely useful and genuinely hard to build. But none of the saves above came from a detector. They came from a verification step the deepfake simply couldn't reach. This design leans into that. It stays effective no matter how convincing the fake gets, because it never tries to answer "is this real" in the first place.

The first version ships as a transparent, rules-based scorecard looking only at request novelty and channel consistency. It's fast to build and easy to explain to an auditor or a board. The next phase adds a learned model incorporating behavioral baseline and language patterns, with a reason code attached to every score so a controller can see exactly why something got flagged.

Explainability isn't optional here. These decisions need to hold up to an auditor and a board after the fact, not just make a fast call in the moment. And the cost of a false positive, delaying a legitimate, time-sensitive request from an executive, gets tracked just as seriously as the fraud numbers.

The first six to eight weeks cover wire transfers only, with a rules-based scorecard. Quarter two extends to vendor bank-detail changes and swaps in the learned model, with the callback flow fully automated. Quarter three adds behavioral baseline modeling per executive and tunes down false positives using real outcome data. Quarter four and beyond extends the same approach to other high-stakes approvals, like contract sign-off or privileged system access, wherever the same impersonation pattern shows up.

The main metric is a measurable drop in successful social-engineering fraud losses. Sitting right alongside it, tracked just as seriously, are the guardrails: false-positive rate stays under an agreed ceiling, added time on a legitimate approval stays in minutes rather than hours, executive and finance-team satisfaction holds up despite the extra step, and the cost of running the system shows a clear positive return against what it prevents.

Adding friction has a real cost when the request is genuinely urgent and legitimate, which is why the guardrails matter as much as the fraud numbers. Any voice or behavioral data collected on executives needs clear governance and retention limits before this ever rolls out. This system raises the bar significantly, but it doesn't make impersonation fraud impossible, and it has to be pitched to leadership as risk reduction, not risk elimination. The callback and live-challenge steps only work if the registered contact channels stay current, so stale data quietly reopens the exact gap this is meant to close. And once this kind of defense becomes standard, attackers will start adapting their scripts to get around known challenge patterns, so the model and its challenge library need an ongoing review, not a one-time build.

Fraud scoring itself is a well-understood pattern. The real judgment is realizing the winning move isn't a harder detector. It's redesigning the approval process so a convincing fake stops mattering, and then figuring out exactly how much friction that redesign can add before it costs more in delayed legitimate business than it saves in prevented fraud. That trade-off belongs to product, finance, security, and the board together, which is what makes this an AI PM problem and not just a security engineering one.