The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Back to feedPost
Most A/B tests don’t fail during analysis. They fail before launch.
Common causes:
• The sample is too small
• The minimum worthwhile effect was never defined
• There is no stopping rule
• Too many metrics are treated as primary
• Important guardrail metrics are missing
The result is weeks of traffic and a dashboard nobody can confidently act on.
Before launch, I calculate the required sample size, expected runtime, minimum detectable effect, statistical power, and guardrails—then provide a clear go/don’t-go recommendation.
If your SaaS or e-commerce team has an experiment planned, I offer a three-day pre-launch review:
contra.com
Pre-Launch A/B Test Review — Power & Sample Size Check by Mo Rahman
Read more about Pre-Launch A/B Test Review — Power & Sample Size Check by Mo Rahman on Contra.
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Related posts
Which static ad do you prefer?
4 voted
80%
1 voted
20%
5 votes
Closed
Vertical light trail. The cyan on the second line gives it a clear focal point.
Built an n8n CRM automation system that captures leads, creates and updates CRM records, assigns tasks, sends team notifications, automates follow-ups and generates reports.
One ad, three first seconds.
Same 20-second body. Three different openings:
1. A five-star review
2. A price comparison
3. A direct call-out: "Dull skin by 3pm?"
Most ads don't fail in the middle. They fail before the product even shows up. Testing the opening is the cheapest test you can run.
Which one would you put money behind first?
Concept ad for a fictional skincare brand.