You Might Like These Articles
Did you Like this Article ? Share It!
Advertising measurement was not invented with digital marketing. Advertisers have utilized methods of measurement for the past 100 years.
Today, Advertising is a big-data world. There are almost a trillion ad requests available per day, billions of ad clicks, and quintillions of 1st data signals tracked and digested by the thousands of Advertisers globally.
This access to data allows for the creation of incrementality testing as a software, so that marketers do not need to run manual experiments such as “stop Germany for 5 days”.
Measuring Value, Not Traffic
Incremental return over investment takes a more holistic approach to measurement, always monitoring the results across the board, while taking into account the results as reported by last-touch attribution and media cost data.
This approach looks at the conversion changes caused by paid marketing, to come up with an understanding of the real incremental sales lift and incremental ROI.
Measuring incrementality requires a continuous layer of prediction analytics overlaid above every comparable combination. Creating a synthetic cohort allows our platform to monitor the difference in difference and come up with digestible insights to marketers.
E.g. “The ROAS for ‘New Vendor’ is cannibalizing Organic traffic. Stopping the activity with ‘New Vendor’ will lead to positive incrementality”
Measure Your Advertising Incremental Results
The Use Cases That Matter
We spoke with many growth teams from various industries and needs to come up with the 6 use cases our platform provides answers about:
Each of these use cases works in reverse as well. Which means that if you as a marketer lowered a bid and ROI increased, or if you paused a campaign and organics went up - the INCRMNTAL platform will provide you with these strategic outputs so that you are always unlocking the value in your marketing spend.
Data science, Algorithms, Statistical models are easily understood to anyone that has a PHD in Data Science, mathematics or Statistics.
But for the rest of us, a clear output with “what does this mean” was needed.
We’ve spent (and continue spending) hours speaking with customers to understand “how would you like to see the outputs?”
What we found was that customers wanted both:
A clear operational recommendation: “The results for “New Vendor” are not incremental at all. We recommend that you stop it”
The full mathematical model, showing what variables were considered (special days, overlapping activities, the prediction range) and the numerical calculation allowing customers to understand causality relationships to make better decisions.
Incrementality measurement until recently focused on segmenting audiences into a control group and showing those audiences with PSA or Ghost Ads, comparing the results of a campaign shown to the control group vs. the result of the general campaign.
This approach usually produced biased or inconclusive results, as there was no ability to know if the control group was “clean” and unaffected by other campaigns running.
Various other attempts to test incrementality were done by blacking out advertising all together for a period of time - but this approach had such high opportunity costs and only provided conclusive results for the time the test was performed - that most advertisers abandoned the idea of performing such tests.
Our challenge at INCRMNTAL was: How would we know if a user was going to perform an action, even if they were not advertised to?
The answer: we don’t
True Attribution Focuses on Incrementality
Our initial idea was: we will build “better attribution”. We wanted to build an attribution solution based on 1st party data, and apply machine learning to understanding the multiple touch points a user has with ads.
But this was a moot point - multi-touch is practically impossible in the mobile app ecosystem, as user data is becoming obsolete.
We also figured that attempting to help developers by offering a new measurement SDK is not helping the developers. No one wants to integrate another SDK.
Our research, had us understand that developers are not in need of “better attribution” - attribution as it is - is fine. But attribution does not provide measurement.
Making budget decisions based on attribution data lead to some major loses. Just ask Uber or AirBNB who reported that over 80% of their ad spend was redundant.
Once we established a few ground rules, we had our direction
We do not challenge attribution data
We are not offering to replace attribution
Incrementality testing is done in retrospect
Incrementality measurement does not happen for a single user
Onboarding via User Interface With No Coding or SDK
Our software needed to allow customers to onboard with absolutely no code or SDK integration. SDKs add weight to apps, and are almost always buggy, requiring multiple iterations for integration causing breakdowns and bugs.
As we required no PII data, no device identifiers or cookies to conduct incrementality measurement - we were able to simplify the onboarding process to support any data storage tools customers might already be using, such as: Google BigQuery, Amazon S3, Attribution Platforms, or even dump aggregated CSV files into drive and have our software fetch it from there.
The software allows marketers to map their data to the data it needs to calculate incrementality.
Causal Inference, Different in Difference
Once we established our ground rules, the answer was found in data science and statistics with Causal Inference and Difference in Difference.
Causal inference is the process of determining the independent, actual effect of a particular phenomenon that is a component of a larger system. The main difference between causal inference and inference of association is that causal inference analyzes the response of an effect variable when a cause of the effect variable is changed.The science of why things occur is called etiology. Causal inference is said to provide the evidence of causality theorized by causal reasoning.
Difference in differences is a statistical technique used in econometrics and quantitative research in the social sciences that attempts to mimic an experimental research design using observational study data, by studying the differential effect of a treatment on a 'treatment group' versus a 'control group' in a natural experiment. It calculates the effect of a treatment (i.e., an explanatory variable or an independent variable) on an outcome (i.e., a response variable or dependent variable) by comparing the average change over time in the outcome variable for the treatment group, compared to the average change over time for the control group. Although it is intended to mitigate the effects of extraneous factors and selection bias, depending on how the treatment group is chosen, this method may still be subject to certain biases
Incrementality Testing in Advertising
Applying causal inference into Advertising was the real challenge. Advertising, and specifically, multi-platform, high throughput, high scale, global, competitive and highly volatile, environment with no constant makes approaching causal inference an extremely challenging task.
You may say that we had an apple fall on our heads when we found our “how”. A simple, yet obvious, constant in every market research call we had with Advertisers across the globe and across various verticals.
From here on, it was an “easy” task, spending the next year running data experiments, developing anomaly detection, developing statistical models and algorithms, and developing a an AI brain that can interpret the algorithmic outputs to simple outputs.
We also felt that it was important that our dashboard will not be another dashboard. Marketers have enough dashboards to look at: Attribution, Media Costs, Campaign Optimization, Demand Side Platforms, Internal BI platforms and so on.
We built our platform as “Tickets” where the software allows customers to measure the incrementality of campaigns, channels or individual marketing activities such as: Price Change, Offline campaign activities, as well as creative.
Our software pushes tickets to customers.