What this calculator does
This MDE and test duration calculator connects your real traffic to what is actually testable. Give it your weekly traffic, baseline conversion rate, and traffic split, and it shows the minimum detectable effect week by week — the smallest lift your test could reliably confirm after 1, 2, 3, … 12 weeks of runtime. Give it a target MDE, and it tells you how many weeks the test must run. Add a start date and the weeks become calendar dates: the last day of the test, and the first day you may read the result.
This is the planning question that comes before the sample size calculator: not "how many visitors do I need?" but "given the visitors I actually have, what experiments are even worth running?"
The math
Each week adds visitors to the control and to each variant in proportion to your traffic split, giving sample sizes and . The calculator then inverts the two-proportion power formula for those two sample sizes, solving for the effect size at which power reaches your target:
solved numerically by bisection, where is the pooled rate weighted by the two sample sizes. Because the sample sizes appear under square roots, the MDE shrinks proportionally to — the source of the diminishing returns you see in the week-by-week table. Both arms count: an uneven split is penalized for its small arm, but not as if both arms were that small. With several variants, is divided by the number of variants, matching the correction applied at analysis.
A worked example
Your checkout flow gets 20,000 visitors per week, converts at 10%, and you plan a 50/50 split (10,000 per arm per week) at 95% significance and 80% power. The table shows: after week 1 you can detect only a ~12% relative lift; by week 4 that improves to ~6%; by week 9 you reach ~4%. If the realistic upside of your change is a 5% lift, plan for about 6 weeks. If the realistic upside is 2%, the same test would need about 36 weeks, which is not runnable at your traffic — better to know that today.
When to use it
- Roadmap triage: kill untestable experiment ideas before design and engineering time is spent.
- Setting expectations: give stakeholders a runtime with a rationale, instead of "we'll see how it goes".
- Choosing test populations: see how much faster the test finishes if you include more pages or segments in scope.
Common mistakes
- Confusing traffic with test traffic. Only visitors who actually reach the tested experience count. If the change is on step 3 of checkout, use step-3 visitors, not site-wide sessions.
- Stopping mid-week. Weekend and weekday visitors behave differently; partial weeks bias the sample. Round runtimes to whole weeks.
- Extending a running test again and again. Deciding week by week whether to continue based on the current p-value is a form of peeking. Set the runtime up front — or use the sequential testing calculator, which is designed for continuous monitoring.
- Ignoring variance reduction. If you have pre-experiment data on the same users, CUPED can cut these runtimes by 25–50% at typical correlations.