Contra - A professional network for the jobs and skills of the futureMost A/B tests don’t fail during analysis. They fail before launch. Common causes: • The sample i...
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Back to feedPost
Mo's avatar
Mo Rahman
• Aug 27
Most A/B tests don’t fail during analysis. They fail before launch.
Common causes:
• The sample is too small • The minimum worthwhile effect was never defined • There is no stopping rule • Too many metrics are treated as primary • Important guardrail metrics are missing
The result is weeks of traffic and a dashboard nobody can confidently act on.
Before launch, I calculate the required sample size, expected runtime, minimum detectable effect, statistical power, and guardrails—then provide a clear go/don’t-go recommendation.
If your SaaS or e-commerce team has an experiment planned, I offer a three-day pre-launch review:
https://contra.com/s/bvFXlQvv-pre-launch-ab-test-review-power-and-sample-size-check
StatisticsData AnalysisProduct Data Analytics

contra.com

Pre-Launch A/B Test Review — Power & Sample Size Check by Mo Rahman

Read more about Pre-Launch A/B Test Review — Power & Sample Size Check by Mo Rahman on Contra.

Back to feed
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started