Tools Free · Calculators

Is the winner real?

Check whether an A/B test result is statistically significant, plan how many visitors a test needs, and work out conversion rates. For marketers testing landing pages, emails and ads.

Inputs

Variant A (control)

Variant B

Confidence

B wins

96.6%

B converts +17.2% relative to A, and the result is significant at 95% confidence (p = 0.0337).

Relative uplift
+17.2%
CVR A
3.2%
CVR B
3.75%
Absolute diff
+0.55%
Percentage points
p-value
0.0337
z = 2.12, two-sided
95% CI of diff
+0.04% to +1.06%

Conversion rate with 95% interval

A

3.2% (2.86% to 3.54%)

B

3.75% (3.38% to 4.12%)

Overlapping intervals do not always mean no difference. Trust the p-value, and decide your sample size before you start.

Get new tools first

Optional. New free tools and growth notes, roughly once a month.

How to use it

  1. Enter visitors and conversions for variant A and variant B.
  2. Pick a confidence level and read the p-value, uplift and verdict.
  3. Before your next test, use Sample size to plan how long to run it.

Want this done for you? I run PR and growth for AI and Web3 founders. Book a free 30-minute teardown →

Questions founders ask

What does statistically significant mean in an A/B test?

It means the difference between variants is unlikely to come from chance alone. At 95% confidence, a p-value under 0.05 is significant. It does not mean the uplift is large, only that it is probably real.

How many visitors do I need for an A/B test?

It depends on your baseline conversion rate and the smallest lift you care about. Detecting a 15% relative lift on a 3% baseline at 95% confidence and 80% power needs about 24,000 visitors per variant. Smaller lifts need far more traffic.

How long should an A/B test run?

Run it until you reach the sample size you planned, and for at least one or two full weeks to cover weekday and weekend behaviour. Stopping as soon as a result looks significant inflates false positives.