A/B Test Significance Calculator
Paste your impressions and conversions from Google Play Store Listing Experiments or App Store Connect Product Page Optimization to check confidence level, lift, and whether you have a real winner.
Control
Variant
Confidence
88.5%
Relative Lift
+37.5%
p-value
0.1148
Want AI-generated variants and a full experiment planner too?
The full ASO A/B Testing Suite plans tests, writes variant copy, and tracks history — free to start.
Why Statistical Significance Matters for ASO
Both Google Play's Store Listing Experiments and Apple's Product Page Optimization run the significance math for you inside their own dashboards, but it is easy to peek at early results and call a winner before the test has actually reached significance. This calculator uses the same two-proportion z-test method to let you sanity-check a result, or evaluate data you have exported, without waiting on either platform's UI.
Frequently Asked Questions
What is a two-proportion z-test?+
A statistical test that compares the conversion rate of two groups (e.g. a control and a variant) to determine whether the difference between them is likely real or just random noise.
Why did my test show a lead but not reach significance?+
Early results are noisy. A lead that looks large after a few hundred impressions often narrows or reverses with more traffic — this is why both Apple and Google withhold a verdict until a confidence threshold is met.
Should I always use 90% confidence?+
Most tests are fine at 90%. For higher-stakes changes like your app icon, where a wrong call is expensive to reverse, use 95% instead.