Sample Size Calculator for A/B Testing
Use this free sample size calculator to figure out how many visitors you need before you can trust your A/B test results.
Running a test without a sample size is like flipping a coin a few times and calling it a trend. You might get lucky, but you might also make decisions based on randomness. This tool gives you a clear target, so you know when your results are actually reliable.
Just enter your current conversion rate, the smallest lift you want to detect, and your confidence settings. You’ll see exactly how many visitors each variation needs.
Works for any conversion-rate A/B test, such as landing pages, checkout flows, signup forms, emails, and more.
What does "sample size" mean in A/B testing?
Sample size is just how many people need to see each version of your test before you can trust the results.
Why care about this? If you don't have enough people, you could miss a real winner. Version B might actually be better, but your data looks like a coin flip. You'll never know the true answer because there wasn't enough data to show it.
But waiting too long is a problem, too. You're sitting around collecting more data than you need while that winning version just sits there doing nothing.
A large enough sample size gets you the answer without dragging things out. Enough people to trust the results. Not so many that you're wasting time.
How to use this sample size calculator (step-by-step)
To calculate sample size and determine the minimum sample size needed for your A/B test, follow these steps:
1. Enter your baseline conversion rate
This is your current conversion rate as a percentage. It represents the performance of your control variant, the page or element you're testing against.
For example, if 4 out of every 100 visitors currently convert, enter 4 as your baseline.
Tip: Pull this number from your analytics over the past 30 days or another stable period. If last month included a promotion, outage, or an unusual traffic spike, exclude that data to avoid distorting your baseline.
2. Choose your minimum detectable effect (MDE)
The minimum detectable effect represents the smallest improvement you want your test to reliably identify. This is expressed as a percentage lift over your baseline.
For example, with a 4% baseline and a 20% MDE, you're looking to detect changes that would move your conversion rate from 4% to at least 4.8%. That's a 0.8 percentage point difference.
How to choose your MDE:
Smaller MDE (10–15%): Requires a larger sample size but can detect subtle improvements
Larger MDE (25–50%): Requires fewer samples but only catches big wins
A simple rule: pick the smallest lift you'd actually ship, given effort, risk, and impact. A 5% lift on a high-traffic page might matter a lot. On lower-traffic pages, you may need a bigger effect to justify changes.
If your required sample size looks too large, increase your MDE and test a bolder change. You can also start on a higher-traffic page or reduce the number of variants.
3. Set statistical power
Statistical power measures the probability that your test will detect a real difference when one actually exists.
80% power: Industry standard for most A/B tests
90% power: Use when the cost of missing a real improvement is high
Higher power requires a larger sample size but reduces the risk of a Type II error. That's when you fail to identify a true result that exists.
4. Set statistical significance (confidence level)
The confidence level tells you how strict you want to be about avoiding false positives. A 95% confidence level means you accept about a 5% chance of detecting a difference that isn't real. This level of certainty helps researchers make confident decisions.
90%: Acceptable for exploratory tests
95%: Standard for most business decisions
99%: Best for high-stakes changes that affect revenue or user experience
This is also called the alpha level. A 95% confidence level corresponds to an alpha of 0.05.
Important: Confidence level does not mean there's a 95% probability that Variant B is better. It's a way to control the false-positive rate across experiments.
5. Calculate and plan your test duration
After entering your parameters, the calculator displays:
Required sample size per variant: The minimum number of visitors each version needs to reach statistical validity
Total sample size: The combined number across all variants. For a standard A/B test, this is double the per-variant number.
Important: Make sure you reach the per-variant number in each group. Don't just rely on the total.
Once you know your required sample size, estimate how long your test needs to run:
Test duration = Total sample size ÷ Eligible visitors per day
Use eligible visitors per day. These are the subjects who actually enter the experiment after targeting rules, device filters, geo filters, and audience conditions. It's not your total site traffic.
For example, if you need 20,634 total visitors and your page receives 1,000 eligible visitors per day, plan to run your test for approximately 21 days.
Important: Run tests for at least one full business cycle (typically 7 days) to account for day-of-week variation in user behavior. Even if you reach your sample size sooner, don't stop early. And avoid peeking at results mid-test.
How is this different from a survey sample size calculator?
Sample size calculators, like those from Raosoft and SurveyMonkey, are completely different from this tool. Their tools are sample-size calculators that tell you how many subjects to survey so your results represent the target population.
It uses inputs such as population size, margin of error, confidence interval, and, sometimes, standard deviation. The primary goal is to measure what people in your target group think, with a given confidence level and a plus-or-minus figure for accuracy.
On the other side, our tool is an A/B test sample size calculator that tells you how many visitors you need to detect a difference in conversion rates between two variants. The sample size formula uses inputs such as the baseline conversion rate, the minimum detectable effect, and statistical power.
| A/B Test Calculator | Survey Calculator | |
|---|---|---|
| Purpose | Detect a difference between two variants | Represent a target population's views |
| Key inputs | Baseline conversion rate, MDE, statistical power, significance level | Population size, margin of error, confidence interval, standard deviation |
| Output | Visitors per variant | Survey responses needed |
| Typical use case | Testing landing pages, checkout flows, and email campaigns | Market research, customer feedback, employee surveys |
Takeaway:
If you're running an experiment to see which version converts better, use this calculator.
If you're trying to understand opinions across a group or measure what proportion of the general population holds a certain view, use a survey sample size calculator.
What are the use cases of this calculator?
Honestly, it works for many different tests. But the sample size you need changes depending on what you're testing. Let me break down some common ones.
E-commerce conversion tests
Running an online store? You're probably testing your checkout page, product layouts, or different price points. The tricky part with e-commerce is that conversion rates tend to be pretty low. Like 1-4% in most cases.
When your starting number is that small, you need a lot more visitors before you can spot a real difference. Just how it works.
Lead generation landing pages
Lead gen is a different story. These pages usually convert at a much higher rate, around 10-30%. That means you don't need as many people to see if something's working. But here's the thing.
Each lead is probably worth more money to you. So even a small bump in conversions could be a big deal. You might want to set your test to catch those smaller improvements, even if it takes a bit longer.
Email marketing experiments
Email tests have a built-in limit. Your list size. That's all the people you've got to work with.
If you have a massive list, no problem. But if your list is on the smaller side, you have to make some trade-offs.
Either test only for larger changes, or accept being a little less certain about your results. Not ideal, but sometimes that's just reality.
Market research surveys
This calculator is really built for A/B tests. But surveys aren't totally different when you think about it.
The big thing with surveys is that you're pulling from a set group of people. You're not getting fresh traffic every day like a website. So you have to think about how big your audience actually is.
How accurate do your results need to be? How many people will realistically fill out your survey? Those answers shape everything about how you set it up.
Tips for running a successful A/B test
Calculating sample size gets you started. But how about running the test right? That's where it counts.
Don't stop early: One version looks like it's winning after a few days. Tempting to call it, right? Don't. Early results bounce around a lot. Wait until you actually hit your sample size.
Test one thing at a time: Change the headline, button, and layout all at once? Cool. Now you have no idea what worked. Pick one variable. Test it. Then move on.
Make sure traffic is randomly split: If one group gets all morning visitors and the other gets evening visitors, your data is skewed. You need both groups pulling from the same population. Most tools handle this automatically, but double-check.
Figure out how long it'll take: Do the math before you launch. How many visitors per day? How many do you need total? If you're looking at months, test something bolder or pick a higher-traffic page.
Run for at least seven days: Even if you hit your numbers in three days, keep going. People behave differently on Tuesdays than on Saturdays. A full week captures the real pattern.