Your A/B test said "winner" — but did revenue per visitor move? CVR is the metric everyone watche...Your A/B test said "winner" — but did revenue per visitor move? CVR is the metric everyone watche...
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Back to feedPost
Mo's avatar
Mo Rahman
• Sep 12
Your A/B test said "winner" — but did revenue per visitor move?
CVR is the metric everyone watches. It's easy to track and moves fast. But CVR going up doesn't mean the variant made you more money.
I've seen checkout tweaks that lifted CVR 8% while AOV dropped enough to net a 6% loss in revenue per visitor. The variant "won" — and got shipped — because no guardrail was in the readout.
Before you call a winner😍 → Check revenue per visitor alongside CVR
→ Run a power check — underpowered tests call false winners ~30-40% of the time
→ If you tested multiple metrics, your p-values need correction (Benjamini-Hochberg)
The stat is only as useful as what you're measuring.
Data AnalysisStatisticsgrowth
Piet's avatar
Piet Baudoin
pro
• Oct 27
Revenue per visitor over conversion rate is the right call. How often do you see teams skip the power check and still ship the winner?
Back to feed
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started