Incrementality Testing for Ads: Design, Math and Decisions

Incrementality testing for ads answers one question: how many conversions happened because of the advertising, and how many would have happened anyway? You answer it with a controlled experiment. One group sees the ads, a comparable group does not, and the difference in outcomes is the lift your spend caused. This guide covers the test designs, the setup, the calculations and the decision rules.

TL;DR

• An incrementality test compares an exposed group with a held-out control group. The gap in conversions is the causal effect of the ads.
• Pick the design by the question: platform conversion-lift studies for single-channel questions, geo experiments for cross-channel or total-outcome questions.
• Define the decision, the primary KPI, the holdout share, the duration and the action thresholds before launch.
• Incremental ROAS (incremental revenue divided by media spend) is the number to budget on, not platform-reported ROAS.
• Contamination, seasonality and short test windows are the usual causes of misleading results.

Table of Contents

• 1. What incrementality means in advertising
• 2. How incrementality differs from attribution and MMM
• 3. Types of incrementality tests
• 4. Designing the test
• 5. How to calculate incrementality
• 6. Turning lift into a budget decision
• 7. What makes a test unreliable
• 8. Perspective: operations decide whether a test is usable
• 9. A checklist before you launch
• Running tests across several brands
• FAQ
• Sources

1. What incrementality means in advertising

Incrementality is the additional business outcome that marketing causes, compared with what would have happened without it. The IAB and IAB Europe guidelines (November 2025) define it in those terms: the additional outcomes directly driven by a campaign or tactic, set against a no-marketing counterfactual.

The practical consequence is that some conversions credited to your ads are not yours to claim. A customer who was already going to buy, and who clicked a branded search ad on the way, shows up as an ad conversion. In an incrementality test, that customer appears in the control group too, so the test nets them out.

Google’s guidance describes the test as a randomized controlled experiment that can measure revenue, profit or a valuable action on a site or app. The outcome you measure is your choice, and it should match the business decision.

2. How incrementality differs from attribution and MMM

Three methods are often confused because they all produce a number next to a channel name. They answer different questions.

An eMarketer analysis from April 2026 draws the same line: attribution tracks user journeys to assign credit, MMM models aggregate inputs and outcomes, and incrementality uses experiments to prove causation. In its survey of US marketers, 27.6% rated MMM the most reliable methodology, ahead of multi-touch attribution at 19.4% and unified measurement at 18.9%.

These methods work best in sequence. Attribution gives you the daily signal. Incrementality tests calibrate it. If you are still weighing attribution models, our piece on cross channel attribution covers how to compare them as tracking signals weaken.

3. Types of incrementality tests

There are three practical designs. Each suits a different question.

Platform conversion-lift studies

The ad platform splits eligible users into a treatment group that can see ads and a control group that is held back. Google’s documentation for Conversion Lift states the arithmetic plainly: incremental conversions equal treatment conversions minus control conversions. This design is fast to set up and gives single-channel answers, but it only sees that one platform.

Geo experiments

You assign matched geographic markets to treatment or control and compare outcomes. Geo tests do not depend on user identifiers, and they measure total outcomes, including sales that never touch a tracked click. Google’s Meridian documentation names three intervention types:

• Holdback: for a brand-new ad strategy. Treatment geos get the new spend and control geos get none.
• Go-dark: for an existing live campaign. Spend is switched off in the test geos.
• Heavy-up: for a budget increase. Extra spend is added in the test geos on top of existing spend.

Audience holdouts and ghost ads

You randomize individuals into exposed and unexposed groups inside a channel, sometimes recording the ads that would have been shown to the control group. This works when you can reliably suppress media for specific people. It gives user-level precision for one channel, with no view across channels.

Choosing between them

• If the question is “which creative, page or offer is better?”, run an A/B test. Our guide to testing one variable at a time is the right starting point, though the correct URL for it is on our resources hub.
• If the question is “does this channel or tactic create value at all?”, run a lift test.
• If you need cross-channel comparability or total business outcomes, use geography.
• If you run one channel and can suppress ads for individuals, use a platform study.

4. Designing the test

Most failed tests fail before launch, because the decision was never defined. Settle these items in writing first.

1. The decision. For example: “Do we raise, hold or cut prospecting spend on Meta next quarter?”
2. One primary KPI. Orders, qualified leads or contribution margin. Secondary metrics are for context only.
3. Randomization unit. Users, households, or geographic markets.
4. Holdout share. The share of the audience or markets that receive no ads.
5. Minimum detectable effect. The smallest lift that would change your decision.
6. Pre-period and test period. A baseline window and a long enough treatment window.
7. Decision thresholds. What result triggers scaling, holding, cutting or retesting.
8. Analysis method. For geo tests, difference-in-differences is the usual approach.

Sample size and statistical power

The smaller the lift you want to detect, the more data you need. Minimum detectable effect depends on the confidence level, statistical power, metric noise and the size of each group. Common planning values are a 95% confidence level (z = 1.96) and 80% power (z = 0.842). Confirm your sample sizes with a calculator or your analyst before spend begins. Our explainer on A/B test sample size covers the same logic in plain terms.

Assignment rules

Treatment and control units must be mutually exclusive. Where a characteristic matters, such as region, customer tier or past purchase behavior, stratify so both groups contain it in similar proportions. Audit exposure during the test to confirm the control group is genuinely not receiving the ads.

Tracking must also be sound before you begin. If conversion events fire twice or not at all, the test measures the error. Run through a conversion tracking checklist first.

5. How to calculate incrementality

The formulas are short. The discipline is in using the right inputs.

• Incremental conversions = treatment conversions − expected control conversions (scaled to the treatment population)
• Absolute lift = treatment rate − control rate
• Relative lift = (treatment rate − control rate) ÷ control rate
• Incremental revenue = incremental conversions × approved value per conversion
• Incremental ROAS (iROAS) = incremental revenue ÷ media spend

Google’s guidance defines incremental ROAS as incremental revenue divided by campaign media spend and positions it as the figure for budget allocation.

A hypothetical worked example

An ecommerce team runs a prospecting campaign with a 50/50 split. This is an illustration, not a benchmark.

• Incremental orders: 1,500 − 1,200 = 300
• Absolute lift: 0.6 percentage points
• Relative lift: 0.6 ÷ 2.4 = 25%
• Share of treatment orders that were incremental: 300 ÷ 1,500 = 20%
• Approved value per order (contribution margin): $80
• Incremental revenue: 300 × $80 = $24,000
• Media spend: $18,000
• iROAS: $24,000 ÷ $18,000 = 1.33

The platform would report all 1,500 orders as ad-driven. At the same $80 value that is $120,000 on $18,000 of spend, a ROAS of 6.67. The two figures describe different quantities, and only the second is caused by the ads. Our guide to reading CPA and ROAS explains why reported ratios mislead.

A note on conventions: some sources divide the rate difference by the control rate (relative lift), others by the treatment rate (share of treatment conversions that were incremental). Both are valid. Name the one you use in the report, and never compare figures across the two.

Choose the value definition before the test

Multiplying incremental orders by top-line average order value flatters the result. Agree in advance whether value means revenue or contribution margin after product and fulfillment costs. That decision determines the break-even iROAS you compare against.

6. Turning lift into a budget decision

A lift percentage is a finding. A recommendation is an action. Pre-agree a decision rule so the result cannot be reinterpreted after the fact.

In the example above, an iROAS of 1.33 may or may not clear the threshold. If contribution margin already accounts for product and shipping costs, the break-even iROAS is around 1.0 before overhead, so the campaign is profitable but thinly. A team might hold the budget, test a heavy-up to learn the response to extra spend, and shift budget only after a second reading. Our notes on budget reallocation across channels and budget pacing across platforms cover the follow-through.

Report the confidence interval, not just the point estimate. An interval that includes zero is not necessarily useless: if the whole interval sits below your business threshold, it still supports reducing investment.

7. What makes a test unreliable

The most common problems are practical, not mathematical.

• Too small a sample. The test cannot separate signal from noise, so the outcome is ambiguous.
• Too short a window. Delayed purchases, repeat orders, refunds and returns fall outside it. Run through a full purchase cycle.
• Seasonality and market changes. Google’s measurement guidance warns that season can confound results because it affects both marketing activity and sales. Promotions, price changes, stock-outs and competitor moves do the same.
• Contamination. When identifiers belonging to one person are split across groups, for example through cookies or multiple devices, the control group is partly exposed. Academic work on counterfactual ad measurement describes cookie and cross-device contamination as undermining causal estimates.
• Spillover. Ads in one channel or geography influence outcomes in another, blurring the contrast.
• No credible counterfactual. The IAB guidelines state that a credible counterfactual, bias control and separation of signal from noise are the core requirements. Without them, measured differences may reflect correlation, not impact.

Mitigation is operational: freeze unrelated changes during the test, log every promotion and budget edit, and keep one change log that the whole team reads.

8. Perspective: operations decide whether a test is usable

We see the same pattern in teams that manage many campaigns. The statistics of an incrementality test are rarely the weak point. The plumbing is: audience definitions that drift between accounts, budgets edited mid-test, creative swapped without a record, and reports assembled by hand from five dashboards.

OppRise is built around a connected workflow: campaign planning, ad execution, performance reporting and approvals in one place, with analytics integrations that include Google Analytics, Search Console, Meta, Google Ads, TikTok and Shopify. For a team running experiments, that structure means a test brief can be approved before launch, spend changes are visible to everyone, and results are read from the same reporting layer as everything else.

Multi-brand workspaces matter here too. An agency that tests three clients in parallel needs each test, each holdout definition and each decision threshold kept separate and auditable. Our view is that measurement discipline is an operations problem first, and tooling that connects planning to reporting makes the discipline repeatable.

Brittany

9. A checklist before you launch

1. Write the decision and the action each outcome triggers.
2. Choose one primary KPI and the value definition behind it.
3. Confirm conversion tracking is accurate across platforms.
4. Select the design: platform lift, geo or audience holdout.
5. Set the holdout share, minimum detectable effect, pre-period and duration.
6. Freeze non-test changes, and log any that cannot wait.
7. Audit exposure weekly to confirm the control stays unexposed.
8. Analyze, report the interval, and recommend an action.

Pair the output with a one-page summary so stakeholders see the result next to the rest of performance. Our template for an end of month marketing report and the framework in marketing performance reporting fit this purpose.

Running tests across several brands

If your team manages multiple brands, ad accounts and approval chains, the coordination load grows with each test. OppRise brings planning, execution, reporting and collaboration into one workflow. Plans are tiered by brands, users and features: Solo is $102.99 per month, Growth is $258.99 per month and Agency is $622.99 per month, with Agency Plus available without a published price. AI credits are sold separately, from 500 credits at $76.99 one-off. See the plans and credit packs for what each tier includes, or read why OppRise takes a connected approach.

FAQ

What is the difference between incrementality testing and A/B testing?

An A/B test compares two versions of something, such as creative or a landing page, to find which performs better. An incrementality test compares ads against no ads to find out whether the tactic creates value at all. Use A/B tests for optimization and lift tests for validating spend.

How long should an incrementality test run?

Long enough to cover a complete purchase cycle, so delayed purchases, repeat orders and returns are captured, and long enough to reach the sample size your minimum detectable effect requires. Avoid periods with promotions or major seasonal swings. Your analyst should confirm the duration during planning.

How big should the holdout group be?

It depends on the expected lift, the baseline conversion rate, the variance, your decision threshold and the cost of withholding ads. A smaller holdout lowers the cost but widens the confidence interval. Check the minimum detectable effect before launch and the interval after.

How do I calculate incremental ROAS?

Divide incremental revenue by media spend. Incremental revenue is incremental conversions multiplied by an agreed value per conversion, ideally contribution margin. Compare the result with your break-even threshold before deciding to scale, hold or cut.

Sources

• Use incrementality testing for effective marketing (Think with Google)
• Understand your Conversion Lift based on users measurement (Google Ads Help)
• Guidelines for Incremental Measurement in Commerce Media (IAB and IAB Europe)
• FAQ on incrementality (eMarketer)
• Types of experiments (Meridian, Google for Developers)
• Measuring Effectiveness: Three Grand Challenges (Google, 2019)
• Counterfactual-based Incrementality Measurement (arXiv)

Recommended

• A/B test sample size, explained simply
• Testing one variable at a time
• Reading CPA and ROAS without fooling yourself
• How to Increase ROAS: A Practical Playbook for Ad Teams