Updated Apr 15, 2024

Incrementality Testing

& Football

Do I Need an Incrementality Testing Software ?

Incrementality measurement is such an innovative method of measurement, that we have to use analogies to make what we do clear to marketers. Whether or not you understand football/soccer is besides the point - measurement is like a team sport, that's what is important to understand.

This article reviews the following areas:

What is an MMP (Mobile Measurement Partner)?
What is Incrementality Testing and How Do You Measure Incrementality?
How Do You Calculate Incrementality?
Everything marketers need to know about incrementality testing
The What, Why & How of Incrementality Testing
Five Ways to Do Incrementality Testing with Digital Ads

“True Attribution Focuses on Incrementality” was a quote from LUMA Partners Digital Marketing Summit, showing that the digital marketing world dependence on Last Touch attribution leads to wrong decisions and waste.

While every method of attribution has value - optimization, forming strategy - depending only on a traffic attribution as means for measurement leads to catastrophic results.

The Football Analogy

Football (“Soccer” to our American readers) is a perfect analogy to Marketing. For a team to score a goal (acquire a new customer) a coordinated effort is needed amongst the players.

External factors influence the game (the audience, the weather, the location) and it is typically not ONE THING that makes a team win or lose. Even at a game where victory is achieved by a score of 1:0.

Last Touch - The Single Player Team

Last Touch Attribution tells you that 100% of the credit goes to the vendor that “touched” the user last. Some even go out on a lim and call this “deterministic attribution”.

But let's imagine what would happen if you make your budget decisions based on the player that touched the ball before the goal.

Even if it is someone like Lionel Messi, or the legendary Pelé - they are most likely going to lose every game against an 11 player team.

Last Touch Attribution would tell you, in a deterministic way, that the best player ever scored goals in past games. That is correct.

But from here - to assume that you can, in a determinis way, that this single player is the cause of the goal...well, that’s just not correct.

Multi Touch - The Impossible Lineup

Multi-Touch attribution is a great concept. By tracking every user engagement within their journey to build a model giving various steps in time different weights in order to give appropriate credit to each step.

This would mean reporting who touched the ball, when they touched the ball, how hard they kicked the ball, and how many players touched the ball before scoring a goal, in order to assign a value for each player.

While in theory this sounds great, there are 2 challenges to multi-touch attribution:

11 players, 90 minutes game ends up with an infinite number of combinations given the amount of data.
IT’S NOT POSSIBLE - since the data is NOT AVAILABLE.

Advertisers must utilize a platform to consolidate data from each media vendor (i.e. each player). Unfortunately, some of the largest players in the world (Facebook, Google) do not share even level data with 3rd party independent attribution solutions.

And with the coming changes: IDFA Deprecation , Cookies becoming obsolete - any hope to make sense of the data goes away.

Incrementality - The Adaptable Lineup

Incrementality will not report who touched the ball last. It depends on attribution data to provide the reporting, but uses machine learning and algorithms to understand causality.

Who scored goals? What were the weather conditions? What was the lineup? Was there a change in the lineup? Was there a change in the conditions?

How many of the players touched the ball, and for how long? Did the team win? What happened during last season? How big was the audience? Did any player get a significant raise? Did the team uniform change? Is there a new player?

By observing ALL the observable changes across hundreds of teams and thousands games - data scientists are able to create hyper parameters (a combination of parameters across a time series) that can indicate what is the expected outcome of a change.

The word “testing” does not make justice to the process of evaluating incrementality - as true incrementality testing does not require a seperate “test” in a laboratory setting , but uses the available data and changes in the data to to provide results.

Incrementality testing has been a holy grail in marketing , but only very few invested the resources required to evaluate it.

With the current market conditions eliminating identifiable data companies are forced to make the effort of researching incrementality as the best alternative to measurement.

Want to improve your measurement ? Get a Grip over incrementality.

You can also find the following useful articles:

This article reviews the following areas:

What is an MMP (Mobile Measurement Partner)?
What is Incrementality Testing and How Do You Measure Incrementality?
How Do You Calculate Incrementality?
Everything marketers need to know about incrementality testing
The What, Why & How of Incrementality Testing
Five Ways to Do Incrementality Testing with Digital Ads

What is Incrementality ?

Incrementality measures the true effectiveness of Advertising activities regardless of tracking.

The goal of paid advertising is to create incremental revenues. Whether if it is to establish a stronger brand equity, or to push people to download an app and use it.

Incrementality testing requires Advertising to create various scenarios to isolate conversions data. Tests use changes in the marketing activities to compare how a change in activity influenced campaigns performance over time.

The goal for marketing in any organization is to drive growth by driving customers and prospects through the marketing funnel. Awareness > Interest > Desire > Action

Advertising efficiency is reached when the Advertising budget spend produces results that would not have happened if it was not for the Advertising activities.

What Incrementality Isn't ?

Incrementality is not a replacement of attribution, nor a method to track clicks, impressions or conversions. Some ad networks offer incrementality tests, showing the marketer that their own incremental test prooves that they produce incremental ROAS - but Return Over Ad Spend is not the correct measuremnet of Incrementality.

True Incrementality is measured in total ROI - Return Over Investment. Making sure that the advertising activities are producing additional value.

What Questions Can Be Answered with Incrementality Measurement ?

Incrementality measurement can help answer very important business questions, such as:

What is the real value of the advertising spend ?
Are the advertising spend in media vendor X producing incremental sales ?
Are the advertising activities in media vendor X cannibalizing the organic sales results ?
Why did the performance of a certain campaign change without doing anything ?
Did a price increase or a budget increase produce incremental sales ?
Which campaigns are generating an incremental lift with other paid channels ?

How Does Incrementality Differ From Traditional Measurement ?

Traditional measurement only counts in "plus". Counting impressions, clicks, conversions, always produces an additional count. Traditional measurement uses a very simple "matching" approach, where as long as the measurement platform finds a match between a click and a conversion, the measurement platform will count this conversion as a +1 for paid marketing.

Even when a marketing channel produces no incremental sales but is only attributing results that would have happened with no advertising - traditional measurement will continue increasing the count.

Incrementality is a measurement of value. measuring the true value of advertising spend allows advertisers to understand if a campaign or a channel is creating additional value, or subtracting value by cannibalizing the results that would have otherwise been generated organically.

How Could You Test Incrementality Easily?

Marketers have opportunities to test incrementality regularly. One easy way to is look at your analytics towards the end of the month when your media vendors run out of budget.

Are your total results declining in correlation with the paid campaigns ? or are the totals actually increasing to compensate? If you see a graph that looks similar to the one below - this can act as proof that your paid media campaigns cannibalize your own organic sales results:

What about week days vs. weekends ? Do you have a regular trend in performance causing your results to appear better or worse during specific days? Is there a correlation between the performance of your paid channels vs. your organic results ? If so - you have an opportunity to levearge this trend to increase your own ROI.

You can also find the following useful articles:

INCRMNTAL is an incrementality testing platform enabling marketers to test the incremental value of their marketing activities. The platform provides actionable insights in a granular level.

If you want to learn more, visit INCRMNTAL or book a demo today!