=Calculator
Does your result
actually hold?
Four numbers are enough. You get not just a yes or no, but the interval, how large the difference plausibly is.
How it is calculated
- The interval for the difference is built with Newcombe’s method, the same procedure as in abdriven.
- If zero lies inside the interval, the difference does not hold, however large the average looks.
- This calculator assumes a fixed-horizon test that was planned in advance and not stopped early.
- For several variations against one original you also need a correction. abdriven applies that automatically.
FAQ
Significance calculator
Why is my result not significant even though the rate is higher?
Because the difference is within what chance produces at this sample size. Either data is missing, or there simply is no difference.
Can I stop the test as soon as this says “significant”?
No. Checking daily and stopping at the first spike produces false winners systematically. Plan the sample size in advance and stick to it.
Calculators
One more calculator.
Make your next change an informed one
Less guesswork.
More now we know.
One question is a good place to start.