The first honest backtest of by Sebastien GuidettiThe first honest backtest of by Sebastien Guidetti

The first honest backtest of

Sebastien Guidetti

Sebastien Guidetti

The first honest backtest of my trading platform returned zero passing strategies out of 142.
Not "a few underperformed." Zero.
I halted all downstream development. Not because the system caught itself — because I looked at the results, decided the foundation was rotten, and refused to build anything else on top of it until it was verified.
Then I ran a bigger validation sweep — 300 cells — to check whether the first result was an artifact. **Also zero clean.** 136 rejected outright, 47 indeterminate, 11 timeouts, and 106 that failed on a single infrastructure bug I'll come back to.
**METIS ORACLE** is what's being rebuilt. What exists:
- **367 engine source files**, an 8-gate signal validation pipeline, direction-aware regime detection
- **A 12-agent specialist council** — technical patterns, manipulation cycles, on-chain flow, macro regime, system health, validation, execution timing, consensus challenge, multi-signal aggregation, risk oversight — routed across five model providers with weighted ensemble scoring and automated fallback
- **A statistical gauntlet:** deflated Sharpe (Bailey / López de Prado), PBO-CSCV overfitting correction, walk-forward out-of-sample validation, Benjamini-Hochberg FDR control at q=0.10
- **Graduated risk zones:** shadow → strict → leverage, where each tier unlocks only from the previous tier's proven results and leverage capital is funded exclusively by strict-zone profit
- **Prop-firm eligibility rules** encoded and machine-checked in advance of any live evaluation
Things this system caught in itself — the reason I'll trust it eventually and don't yet:
**The measurement framework was forging its own grades.** It compared trade results in *dollars* against formulas expecting *percentages*, then applied the wrong calendar factor. A 37% gap between reported and actual. For eight days, the entire strategy leaderboard was fiction. A whole cull of strategies got executed on that broken ruler. Fix the ruler before you trust anything it measures.
**Ten of twelve council agents were silently disabled for two months.** An aliasing bug. Not one error thrown. Separately, a runtime thesis producer sat dead for 39 days while every panel downstream of it rendered clean, honest, entirely empty output. Nothing looks more finished than a working UI with no data behind it.
**Confidence looked inverted — and then that finding was itself invalidated.** An audit of the council's stated confidence against outcomes returned a Brier score of 0.537, worse than a coin flip, with the 80–90% confidence bucket going 0 for 9. Leverage got clamped to 1×. Six days later a paper-trade P&L contamination bug was found in the input data, and the finding was stamped INVALIDATED pending a re-run. The clamp stayed, on independent grounds. I'm telling you about a finding that turned out to be wrong because the alternative — quietly dropping it — is the exact behaviour I audit other people's systems for.
**A gate that existed and did nothing.** Remember the 106 failures in the 300-cell run? They were all one bug class — and I had already written the gate that catches it, five days earlier. It was on disk. Nothing called it. That's a failure mode I now track separately from "never built": **BUILD ≠ WIRED.** The fix existing is not the fix working.
**Status: shadow mode. No live capital. Cleanup ongoing and incomplete. No edge proven.**
I'll say so publicly the day that changes, and not one day earlier. If you ever see me post a return number, you'll also see the audit that produced it.
**If something in your own pipeline is supposed to work but doesn't, quite — that's the thing I'd want to look at.**
#AlgorithmicTrading #QuantitativeFinance #AIEngineering #Backtesting #SystemsArchitecture
Like this project

Posted Aug 19, 2026

The first honest backtest of my trading platform returned zero passing strategies out of 142. Not "a few underperformed." Zero. I halted all downstream devel...