This is the most comprehensive article you can find about incrementality. 

• What is Incrementality ?
  • Everything marketers need to know about incrementality testingThe ins and outs of incrementality testing
• What is incrementality in marketing?
• Doesn't attribution indicate incrementality ?
• What is the difference between Incrementality and Lift? 
• What is Incrementality Testing and How Do You Measure Incrementality?

Are you familiar with the true value of your marketing budget? Focusing on Last Touch Attribution and Return On Ad Spend (ROAS) is important key performance indicators, those do not provide marketers with the relevant data to determine the value of their advertising spend, if any.

How can you know that your marketing budget spend add value ? incrementality testing is the answer.

What is Incrementality ? 

Incrementality measures the true effectiveness of Advertising activities irregardless of tracking.

The goal of paid advertising is to create incremental revenues. Whether if it is to establish a stronger brand equity, or to push people to download an app and use it.

Incrementality testing requires Advertising to create various scenarios to isolate conversions data. Tests use changes in the marketing activities to compare how a change in activity influenced campaigns performance over time.

The goal for marketing in any organization is to drive growth by driving customers and prospects through the marketing funnel. Awareness > Interest > Desire > Action

Advertising efficiency is reached when the Advertising budget spend produces results that would not have happened if it was not for the Advertising activities.

Read More: What is Incrementality Testing?

Why Do Advertisers Need Incrementality ? 

Measuring incrementality is the best way to ensure that the sales results (attributed to paid marketing) are results that would not happen if it was not for the advertising activities. 

Without incrementality measurement, advertisers could be spending advertising budget continuously, believing that their advertising activities are producing incremental sales, while the reality could be that the activities are actually cannibalizing sales that would already happen without advertising. 

What Incrementality Isn't ?

Incrementality is not a replacement of attribution, nor a method to track clicks, impressions or conversions. Some ad networks offer incrementality tests, showing the marketer that their own incremental test prooves that they produce incremental ROAS - but Return Over Ad Spend is not the correct measuremnet of Incrementality. 

True Incrementality is measured in total ROI - Return Over Investment. Making sure that the advertising activities are producing additional value. 

What Questions Can Be Answered with Incrementality Measurement ?

Incrementality measurement can help answer very important business questions, such as:

• What is the real value of the advertising spend ?
• Are the advertising spend in media vendor X producing incremental sales ?
• Are the advertising activities in media vendor X cannibalizing the organic sales results ?
• Why did the performance of a certain campaign change without doing anything ?
• Did a price increase or a budget increase produce incremental sales ?
• Which campaigns are generating an incremental lift with other paid channels ?

How Does Incrementality Differ From Traditional Measurement ?

Traditional measurement only counts in "plus". Counting impressions, clicks, conversions, always produces an additional count. Traditional measurement uses a very simple "matching" approach, where as long as the measurement platform finds a match between a click and a conversion, the measurement platform will count this conversion as a +1 for paid marketing.

Even when a marketing channel produces no incremental sales but is only attributing results that would have happened with no advertising - traditional measurement will continue increasing the count. 

Incrementality is a measurement of value. measuring the true value of advertising spend allows advertisers to understand if a campaign or a channel is creating additional value, or subtracting value by cannibalizing the results that would have otherwise been generated organically.

How Could You Test Incrementality Easily?

Marketers have opportunities to test incrementality regularly. One easy way to is look at your analytics towards the end of the month when your media vendors run out of budget. 

Are your total results declining in correlation with the paid campaigns ? or are the totals actually increasing to compensate? If you see a graph that looks similar to the one below - this can act as proof that your paid media campaigns cannibalize your own organic sales results:

What about week days vs. weekends ? Do you have a regular trend in performance causing your results to appear better or worse during specific days? Is there a correlation between the performance of your paid channels vs. your organic results ? If so - you have an opportunity to levearge this trend to increase your own ROI.

Incrementality Measurement is the only True Measurement 

The role of Marketing in every organization is to drive growth.

Measurement is a critical component in every marketing role. The ability to understand the influence of the marketing activities to results allows a marketer to refine their broader marketing strategy to achieve their goals.

Last Touch Attribution is a method of tracking which gives credit to where a user last engaged with a campaign before conversion.

While Last Touch Attribution is an important indicator – it does not allow Advertisers to measure the effectiveness of their Advertising.

Incrementality Measurement offers Advertisers with a way to attribute the revenue contribution of various Advertising campaigns.

Learn More: TV Ad Tracking

Last Touch Attribution 

Why waste marketing budget over ad creatives that do not yield results? How to focus more spend over ads with a higher engagement ? Last Touch attribution is the answer.

Last touch attribution offers a benefit to Advertisers able to utilize the method.

The method allows marketers to associate the last ad the user engaged with before the conversion point. This method works in real time allowing Advertisers to use the data by optimizing activities and saving waste over non performing campaign elements.
There’s no need for any historical data , as the attribution happens based on a match between the click and the conversion.

The ability to make changes in campaign activity based on real time attribution data offers many advertisers an advantage.

The mechanism of last touch attribution usually required an identifier match between an click or an impression of an advertisement.

During the past several years, user privacy advocacy groups resisted the accessibility of device level data and tracking, causing the companies behind browsers and Operating System to make device level identification not available.

Advertising fuels the Free Internet, and tracking is a necessary part of this.

While user level tracking is slowly being faded out, alternative methods for last touch attribution are becoming available, on a campaign level rather than a user level.

Last Touch Attribution is not a good method for measurement. Credit is given only to the ad the user clicked or saw last before converting often leads to over attribution.

Over attribution represents the tracking of conversions where the user unintentionally interacted with an ad on their way to the conversion point.

Unintentional interaction with ads may happen as a result of Fraudulent publisher activities, but may also happen as a result of a high correlation between organic marketing results and paid  marketing results.

The graph here shows how over attribution may influence Advertising effectiveness.

Campaigns starting on “New Vendor” are tracked and attributed by Last Touch showing a positive Return Over Ad Spend (ROAS) as seen in the smaller window.

Conversions by Source graph shows that the new vendor receives credit for conversions that would have happened irregardless of campaigns starting with the new vendor.

Mobile App Tracking is susceptible to “attribution gaming” as fraudulent publishers realized that if they trigger an ad engagement, with or without the user knowledge before the user reaches the conversion point – the mobile attribution platform will credit the conversion to them, thus, gaining budget or higher prices from the Advertiser.

Last touch attribution is a good means of tracking, but it does not replace incrementality measurement.

 Marketing Strategy and Marketing Mix Modeling 

Marketing is a broad term describing a company’s activities to promote the sales of a product or service.

Marketing strategy revolves around defining these four areas:

• Product – the product or service being marketed and the messaging to use to define it to customers
• Price – the pricing and pricing model for the product or service
• Place – defining the where can customers acquire or engage with the product or service
• Promotion – setting the strategy for Advertising activities to create awareness leading to action

In today’s globally competitive world, Advertising activities are a necessary part of almost every product in market. If you are a mobile app developer, you are competing against thousands of app companies from around the world.

Media vendors have made Advertising lucrative by offering Performance Pricing schemes where Advertisers can sit back and pay conversions rather than risk the costs of media which may not lead to any conversions.

While performance pricing models are tempting to Advertisers – the caveat in Performance Pricing is that it incentivizes media vendors to optimize their own media strategy towards the low hanging fruit: Targeting users that may likely would have converted , thus reducing the media inventory that is required to be used.

Incrementality measurement allows Advertisers to measure the true value of their advertising performance.

Media Mix Modeling 

Media Mix Modeling is a statistical method to estimate the impact of various marketing tactics on sales in order to better forecast and come up with a better marketing strategy.

The method was developed in econometrics for the consumer packaged goods industry and has become common with brand cross platform Advertisers in the last years.

Media Mix Models require historical data to have any helpful outputs. Often, the data needs to include external influencing factors such as competitor’s activity, product launches, financial events, weather and any major event that may have influenced the performance of a product (i.e. during an Olympics year, more people buy sport goods).

Due to these requirements - Media Mix Models work best for refining a strategy, expecting influencing factors such as changes in the media mix and/or external factors to help understand what would be the best media mix to market a product over time.

Media Mix does not work well for new product launches, as without historical data - there are simply too many unknown variables.

B2B2C brands (i.e., consumer goods, retail, consumer electronics) often must rely on Media Mix Modeling to analyze the effectiveness and impact of their paid marketing activities. 

Attempts to use a simple version of Media Mix Modeling are tested regularly by Advertising being turned off completely , allowing Advertisers to analyze sales activities with no Advertising in place - however, switching off all Advertising is extremely hurtful to most Advertisers as while doing so - competition may consume market share, thus, hurting long term brand equity for an Advertiser experimenting with turning off the lights across all marketing activities. 

Attempts to use a simple version of Media Mix Modeling are tested regularly by Advertising being turned off completely , allowing Advertisers to analyze sales activities with no Advertising in place - however, switching off all Advertising is extremely hurtful to most Advertisers as while doing so - competition may consume market share, thus, hurting long term brand equity for an Advertiser experimenting with turning off the lights across all marketing activities. 

Incrementality testing is a subset of media mix modeling, providing Advertisers with insights to improve the value they get from their marketing activities.

Performance Marketing Does Not Guarantee Incrementality

In a recent well known news, Uber Technologies filed a lawsuit claiming Fraud, Negligence, Unfair competition against several ad networks.

Uber ran advertising campaigns with networks where it only paid the network if and once a user became a customer, ordering their 1^st “ride”.

Uber found that when pausing some of the advertising campaigns, the total number of new customers was unaffected.

Uber went on to cut 80% of their marketing spend with no significant impact over new customer acquisition.

All this, while using modern day app tracking and attribution technologies.

While the above example shows a clear picture of cannibalization, the reality for most marketing campaigns is more subtle.

The opposite to incrementality is commonly known as cannibalization.

Cannibalization in Advertising refers to the Advertising activities claiming credit for revenues that would have happened if it was not for the Advertising activities.

 Incrementality measurement is the gold standard of Advertising measurement. Measuring value, rather than measuring traffic.
Incrementality testing importance grew rapidly due to the infestation of attribution fraud over the past several years. With the deprecation of identifiers (IDFA, Cookies, ITP, GAID) Incrementality is most accurate way of measuring and optimizing marketing spend.
 

Incrementality Testing Methods 

There are various ways Advertisers use to measure incrementality.
We wanted to provide an objective view over the various methods:

Incrementality A/B Testing: 

The process of A>B testing is a relatively simple one - creating two campaigns or line items, where one would show users PSA or “ghost ads” while the second campaign displays the normal creatives.
The purpose of this test is to create an uncontrolled split in the audience and compare the performance of the two campaigns.
 

Pro’s:

- Easy to execute and understand the results of the test
- Even if the results are not very accurate - they do provide an indication
 

Cons:

- Relevant for Advertisers working only with 1 vendor, as spillage would influence the result of the test
- The advertiser must waste media costs over PSA or Ghost ads
- Requires the ability to create user level frequency capping
- Not a valid option in a post-IDFA world

Incrementality GEO Split 
 

A GEO split is extremely common for FMCG multi-country brands, as market penetration cost are exceptionally high and require an upfront investment.
The method requires the Advertiser to control a product promotion or launch while making the product available, but limiting the promotion and access to the product in another GEO.
This method is commonly used in Media Mix Modeling
 

Pro’s:

- Suitable for Brands where market penetration requires a large upfront investment
- Easy to execute the test for mobile and digital only advertisers
 

Con’s:

- Markets will behave differently due to various factors that may not be examined leading to wrong assumptions
- The results of the test will be far from conclusive for operational / tactical decision making.

Randomized Control Group 
 

Very similar to A>B testing and GEO Split - a randomized control group seeks to split the user base being advertised to in a homogeneous way, so that there is an almost equilibrium between group A and group B.

The groups are served with either PSA ads or the campaign creatives and the performance is measured to reach a conclusive outcome.
 

Pro’s:

- This method can provide rather conclusive results as long as the Advertiser can fully control the targeting limiting against any spillage or overlap
 

Con’s:

- Given the requirement to identify “users” on publishers websites and apps - this method became obsolete as the % of LAT users (Limit Ad Tracking) grew, as well as Apple deprecating IDFA
- Performing this test in a single platform rather than across all media platforms used will not provide a reliable outcome.

Advertising Blackout for Incrementality Testing 

Probably the easiest and most effective way to measure incrementality in advertising is to stop advertising.

It may sound ridiculous, but Advertisers who experiment with this are able to improve their advertising effectiveness.

The method requires Advertisers to stop ALL advertising activities for some time, to allow a true “baseline” to be identified, followed by a slow, gradual process, of reactivating campaigns and media vendors, while monitoring the effectiveness of each one in an isolated manner.

TV Advertisers commonly use this approach.

Blackout is done by switching off Advertising for a period of time, reviewing sales lift, and switching TV advertising back on to monitor the delta.

Blackout is probably the most common method of TV advertising attribution - and for Advertisers who use TV exclusively - this is a very good way to run measurement. 

For Advertisers who use multiple mediums and a media mix model - this method is not effective, and creates a potential opportunity cost that exceeds the gains.

Pro’s:

- The most effective method to reach a conclusive outcome
- No additional media costs
 

Con’s:

- Opportunity cost may outweigh the outcome
- Any external factors influencing the results may cause the test to be flawed
- The test needs to be performed multiple times in order to continuously measure effectiveness

Incrementality Measurement using Causal Inference 

The most recent method for incrementality measurement. Causal Inference is an algorithmic process to draw conclusions about the causality of results in a multi-variant and noisy environment.
Causal inference is used in epidemiological research, in economics and most recently, in AI.
Applying causal inference models requires research and development to come up with hyper-parameters across a time series to provide a digestible insight over the value of marketing activities.

Pro’s:

- Causal inference can digest digital as well as offline campaign activities
- Actionable insights can be granular enough to provide operational suggestions
- Campaign data is used as is without the need for any disruption
- Resilient to a post-IDFA world
 

Con’s:

- Requires significant data research to reach the right hyper-parameters
- A one of research would become incorrect over time (continuous research needed)

Calculating Incrementality 

Calculating incrementality can be very simple. It just requires you to follow these steps:

1. Stop ALL advertising and marketing activities for a period of time to get an idea of your organic baseline with no marketing spend.

2. Reactivate media campaigns, vendor by vendor. No more than one per week so that you have clean data to analyze.

3. Analyze reports to learn the real value your advertising activities have as well as monitor the impact of starting a new vendor over any already active vendors.

4. Repeat this process several times a year.

Looking at this graph, we can see several observations:

During Week 8, with the launch of marketing activities with Vendor 1 – there is an additional increase of Organic conversions.

During Week10, with the launch of Vendor 3 – there is a sudden decline in Organic conversions, as well as a decline of conversions attributed to Vendor 1. 

We could conclude the following:

Vendor 1 has a positive incremental value.

Vendor 3 has a negative incremental value (i.e. cannibalization) over both organic conversions as well as those of Vendor 1.

While this test shows clear results – most marketers will not be willing nor able to perform such a test.

The opportunity cost may be much bigger than the gains

As competition operates in the market – marketing data will never reach “laboratory” conditions for noiseless data 

ABOUT INCRMNTAL

The INCRMNTAL platform allows marketers to understand the true incremental value of their marketing activities. The platform points out activities adding value, activities that take value from other activities, as well as activities that add no value.

The technology behind the platform works using Causal Inference, using attribution data from MMPs or SKAdnetwork in aggregate form, to calculate the incremental value, and incremental ROAS, for marketers to make strategic and tactical decisions.

The platform is completely self-service and requires no SDK or code for integration.

If you want to learn more, visit INCRMNTAL or Book a Free Demo today