A/B Test Significance Calculator
A/B Test Significance: a planned Super Calculator tool. Covers visitors and conversions per variant and confidence level (%). Stub page. Not published yet.
A/B Test Significance Calculator is a planned calculator in the mathematics section of Super Calculator, covering experimentation and A/B testing.
What this calculator will compute
Inputs: visitors and conversions per variant, confidence level (%), one or two tailed and frequentist or Bayesian.
Outputs: conversion rate per variant, absolute and relative lift (%), z, p value, confidence interval on lift and probability B beats A.
Formula, recorded as scope and not yet verified against a primary source: Two-proportion z test: z = (p1 - p2) / sqrt(pbar(1-pbar)(1/n1 + 1/n2)) with pbar the pooled conversion rate. Confidence interval on the lift from the unpooled standard error. Bayesian alternative reports P(B beats A) from Beta posteriors under a Beta(1,1) prior.
Status
This page is a stub. The calculator is not built, no figure on this page has been verified against the authority that sets it, and nothing here should be used to make a decision. The page exists so that the site structure is complete and so that articles can link this concept on first mention.
The stub carries noindex, nofollow and is excluded from the sitemap.