Incrementality measurement has been a well-established practice for some time now. Traditional methodologies used to involve extensive testing and experimentation, which required significant amounts of time and resources, and presented numerous flaws and challenges.

When we launched INCRMNTAL, we wanted to leverage advancements in technology to bring incrementality measurement into today’s world using AI, machine learning, and causal data science. 
While developing the product, we wrote several white papers addressing our assumptions, research, and methodology, however, we are still often asked to explain “how the platform works?”

In a recent customer meeting, we were asked if we could explain our methodology so that non-technical people could relate. Explain it without going into predictive modeling algorithms, terms that only data science and statisticians would understand. We were asked to “explain it like I’m five” 
 

The request was valid, and our solution was to come up with a bunch of analogies that everyone could relate to. We went for Pizza. 

Instead of “What was the incrementality of the campaign you launched on TikTok?” – lets go for a different question: Why did you gain 10 pounds (4.5 kilos) in the past 2 weeks ?

Assumptions, data requirements, scoring, and logging:

Weight is affected by several variables. To name a few: Genetics, Age, Gender, Diet, Exercise and physical activity, Habits, Culture, Mood

Each of these variables may affect people differently, hence, there is no simplified score for any of these variables. Each of these variables would need to be considered in order to answer the question.

When integrating with our platform to answer the weight gain question, we would need some data: your weight by day for the past 12+ months, as well as some general information about you. 
Thanks to our miraculous activity detection technology, once you authenticate us with your kitchen, we would be able to retrospectively log what you had for breakfast, lunch, and dinner over the past months.

Your weight will have some volatility in your data. Most people gain weight during the holidays. Most people lose weight during summer. But you might be unique in your own way, and we will not be assuming anything about your own weight data. 
 

We would be expecting that holidays would show an increase in weight, but if your own data shows that you have no inclination to gain weight during holidays, our platform would reduce the scoring given to holidays as an influencing factor to your weight.

Automated analysis of your weight data may show that your weigh varies between -3 and +5 pounds amongst week days vs. weekends. This will act as an important factor in measuring your weight gain cause.
Analysis may also show that you lose around 6 pounds during the 1st week of the year, but gain 1.5 pounds every 28 day.

Analysis may also show that once you’ve started exercising more, your weight started declining. While we don’t know what sort of exercise you’re practicing, our platform would  assign a higher scoring to exercise as a factor.

We wouldn’t ask you to share your mood swings with us. That’s personal data, and we believe that personal data should remain personal. The tradeoff between sharing such data with the world for the slight potential improvement in accuracy is not worth it. Especially since what you care about is to understand is why you gained 10 pounds – a figure that is quite statistically significant.

Introducing Pizza!

To measure why you gained 10 pounds over the past 14 days, our platform would orchestrate several data modeling to apply your own genetics, average weight gains/losses over weekends, weekdays, weeks of the year, months of the year, applying the impact of holidays, special days. 
The platform will also create a lot of micro-analysis overlaying your exercise vs. weight, but most importantly for our platform would be to score the influence of your diet log over weight changes. i.e. how does your weight change when you have pasta, salads, fish, burger, steak, deserts?
How does weight changes against various foods combined with different level of exercise ?

If our magical food logger would detect that you’ve introduced Pizza to your diet, the platform would likely suggest: “hey – why do you check out if Pizza was it?”

Measurement using our platform is a retrospective prediction. That means that you’ll be able to measure: “how much weight did I gain from adding Pizza to my diet 14 days ago?”

Our platform would utilize your data to create a synthetic “base line”. What should your weight have looked like over the past 14 days, if you had not introduced Pizza?

Given the volatility in your weight, the prediction would not be “flat”. It would be volatile, and consider the weekends, weekdays, holidays, special days, whatever else you’ve had for breakfast, lunch, dinner, your exercise level.
Any outliers that go beyond what we do know, would be attributed to unknown. As mentioned above – we don’t want to assume or require you to add “mood” data.

Since we do add another validation in the form of a 95% confidence interval, the platform would predict that your weight should have been anywhere between 9.5 to 10.5 pounds lower than it was during this period. If there were other variables influencing your weight around this time period, even if your expected weight during this period would be higher or lower, during certain days, the platform 
would be able to recognize any variable, thus, attribute the increase of 10 pounds to the fact that you introduced Pizza to your diet, and not to weekend, holiday, and whatever other variable in the picture.

The measurement and modeling would look something like this:

From analogy to technology

If we translate the variables from the analogy to our actual technology – the variables would translate as follows:
 

And if we try and answer “What was the incrementality of the campaign you launched on TikTok?” using our methodology, rather than analogy, it would look something like this:

Our platform will orchestrate several data modeling to apply your own performance trends over weekends, weekdays, weeks of the year, months of the year, applying the impact of holidays, special days. 
The platform will also create a lot of micro-analysis overlaying your marketing activities vs. performance, but most importantly for our platform would be to score the influence of your operational marketing activities log over performance changes. i.e. how does your performance change when you increase bids on Facebook? Swap creatives on google? Start a new campaign on Twitter?
How does performance changes against various campaigns combined with different level of ad spend ?

The measurement and modeling would look something like this:

Bonus: Why legacy incrementality measurement is no fun

Legacy methodologies would require you to do an experiment to measure the value of Pizza to your weight. In our funny analogy, that would mean one of these two options:

Option 1 (regional geolift test)
• Eat pizza for 2 weeks, 
• Stop eating Pizza 
• Compare the difference in your weight, regardless of your exercise habits, the time of year, the weather, any other change your diet.

Option 2 (geolift test)
• Stop eating anything for 2 weeks
• Only eat pizza for 2 weeks
• Stop eating anything for 2 weeks
• Go back and measure the weight gain you had after starving yourself for 2 weeks

The biggest downside of legacy methodologies of incrementality measurement is that the opportunity cost is huge. In the example above, the opportunity cost would be starvation, or the delusion that other variables have a small or not at all influence over performance.