Incrementality Testing
An experimental method that measures the true causal impact of a marketing campaign by comparing a test group exposed to ads with a control group that was not, isolating lift attributable to advertising. Common approaches include geo-based holdout tests, ghost ads (bid but do not serve), and randomized controlled trials using platform tools like Meta Conversion Lift or Google Brand Lift. Statistical significance typically requires a minimum of 2 to 4 weeks of data and sufficient sample sizes, often 10,000 or more users per group.
Why It Matters
Incrementality testing answers the fundamental question: would this sale have happened without the ad? Studies show that 20-40% of retargeting conversions would have occurred organically, meaning brands often overpay for non-incremental results. By isolating true incremental impact, it prevents brands from paying for conversions that would have occurred organically, ensuring every ad dollar drives real growth. It also serves as a calibration tool for attribution models, grounding modeled estimates in experimentally validated lift data.
Example
A brand runs a geo-based incrementality test: 10 cities see retargeting ads while 10 matched cities do not. The test cities show 12% higher sales. Since matched control cities grew 5% naturally, the incremental lift from retargeting is 7 percentage points, revealing that only 58% of attributed retargeting conversions were truly incremental. Based on these findings, the brand reduced retargeting spend by 25%, saving $75,000 per quarter, and redirected the budget to prospecting campaigns that delivered a higher incremental ROAS of 3.8:1.
Related Terms
A/B Testing
A controlled experiment comparing two or more versions of a webpage, email, ad, or other marketing asset to determine which performs better against a defined goal. Traffic is randomly split between variants, and statistical analysis determines whether observed differences are significant or due to chance. A/B testing is also called split testing. More advanced forms include multivariate testing, which evaluates multiple variables simultaneously, and multi-armed bandit testing, which dynamically allocates traffic to better-performing variants.
Marketing Mix Modeling
A statistical technique that uses regression analysis on historical data to estimate the impact of each marketing channel on sales while controlling for external factors like seasonality, economic conditions, competitor activity, and pricing changes. MMM typically requires 2 to 3 years of weekly data to produce reliable results. Modern approaches use Bayesian methods and tools like Google Meridian or Meta Robyn to reduce data requirements and improve accuracy, making MMM accessible to mid-market brands, not just enterprises.