Your A/B test said "winner" — but did revenue per visitor move? CVR is the metric everyone watche...Your A/B test said "winner" — but did revenue per visitor move? CVR is the metric everyone watche...
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Your A/B test said "winner" — but did revenue per visitor move?
CVR is the metric everyone watches. It's easy to track and moves fast. But CVR going up doesn't mean the variant made you more money.
I've seen checkout tweaks that lifted CVR 8% while AOV dropped enough to net a 6% loss in revenue per visitor. The variant "won" — and got shipped — because no guardrail was in the readout.
Before you call a winner😍 → Check revenue per visitor alongside CVR
→ Run a power check — underpowered tests call false winners ~30-40% of the time
→ If you tested multiple metrics, your p-values need correction (Benjamini-Hochberg)
The stat is only as useful as what you're measuring.
Back to feed
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started