A/B Test Sample Size Calculator 2026

At minimum 1 full business week to capture weekly traffic patterns (weekday vs. weekend behaviour often differs significantly). Use the sample size calculator to determine minimum duration: required visitors per variant ÷ daily visitors to that page. Ideally run for 2–4 weeks for robust results. If your site has very high traffic (100,000+ visitors/month per page), you may reach significance in days — but still run for at least 7 days. Never stop early just because one variant appears to be winning.

MDE is the smallest improvement you want your test to reliably detect. If you set MDE at 10% relative improvement (e.g. 3% baseline → 3.3% test), the calculator tells you the sample size needed to detect that change with the chosen confidence level. Smaller MDE requires much larger samples — detecting 2% absolute improvement requires far more traffic than detecting 10% improvement. Set MDE based on the minimum change that would be commercially meaningful, not the change you hope to see.

Statistical significance (confidence level) is the probability that the observed difference between variants is not due to random chance. 95% confidence means there is a 5% chance of a false positive — concluding one variant is better when the difference is actually random noise. 99% confidence reduces this to 1% but requires larger sample sizes. Most practitioners use 95% as the standard. Note: significance alone does not prove practical importance — a statistically significant 0.1% improvement may not be worth the development cost to implement.