A/B Test Calculator

Enter recipients and conversions (opens or clicks) for each variant to see whether the difference is statistically significant, with a p-value and a confidence interval for the difference. Plan the required sample size before sending.

Workspace

Variant A (control)

Variant B (test)

Sample size planner

How the result is calculated

Two-proportion z-test (two-sided): p̂ = (xA+xB)/(nA+nB), SE = √(p̂(1−p̂)(1/nA+1/nB)), z = (pB−pA)/SE, and p-value = 2·(1−Φ(|z|)). The difference is significant if p < 1 − confidence level.

Confidence interval for the difference (unpooled Wald): (pB−pA) ± z·√(pA(1−pA)/nA + pB(1−pB)/nB). Sample size per variant = (zα/2√(2p̄(1−p̄)) + zβ√(p1(1−p1)+p2(1−p2)))² ÷ (p2−p1)², where p2 = p1(1+lift) and p̄ is their mean; α = 5% two-sided.

Limits: “Not significant” does not mean “no difference”; the sample may be too small. The test assumes independent samples and random assignment, and that sample size was fixed before looking at results: stopping at the first significance (peeking) or testing many variants without correction inflates false positives. The normal approximation is weak if any conversions or non-conversions count is under 5 (flagged). The numbers' accuracy is your responsibility.

FAQ

What does p = 0.03 mean?

If there were no real difference, the chance of seeing a gap this large or larger by luck is 3%. It is not the probability that B is better.

Should I measure opens or clicks?

Clicks or conversions. Opens are inflated by Apple Mail privacy protection, which makes them a weak judge even for subject-line tests.

Newest tools

Recently added free tools — they run in your browser without uploading your data

All tools →

AES and ChaCha20 keys with correct IV/nonces via Web Crypto, Hex, Base64 and JWK formats, verification and bulk export

Local

Paste discussions (Reddit, Quora, YouTube) to extract pain points, repeated questions, phrases, topic clusters and title ideas

Local

A bookmarklet that analyses any page as your browser sees it after JavaScript: title, description, H1, links, images, Open Graph, JSON-LD and hreflang

Local

Legitimate link-building tactics, outreach message templates, a linkable-asset idea generator and a local opportunity tracker with CSV and JSON export

Local