Creative testing
A/B test sample size, explained simply
Why small tests mislead, and a rule of thumb for knowing when a creative test has run long enough.
OppRise Editorial · Reviewed 28 September 2026 · 3 min read
Why small tests mislead
A test with fifty clicks per variant can show a winner that is pure luck. Small samples exaggerate differences, so teams scale the wrong ad and kill the right one.
A practical rule of thumb
For creative tests on paid social, wait until each variant has at least a few thousand impressions and enough conversions to compare, often thirty or more per variant. For low volume accounts, test bigger differences instead of small tweaks, because big swings need less data to detect.
Decide the end before you start
Write down the metric, the minimum run time and the decision rule before launching the test. Checking results daily and stopping when one variant looks good is the most common way tests go wrong.
Put this into practice with