GeoLift experiments managed to gain the nickname “the golden standard for incrementality measurement”, as the most precise method to measure marketing incremental lift.

In principle, the method is very simple to apply:
1. Split a country into many tiny regions
2. Show ads to users in one region, while holding off ads in another
3. Compare the performance of the two regions
4. The difference in performance between the regions will be the incremental lift 
Sounds great, works great, in theory.
 

What is GeoLift testing?

GeoLift testing is a method to measure marketing incremental lift by splitting a geographical area into test and control regions, running localized ad campaigns, and comparing performance to gage the incremental value of Advertising. The idea in GeoLift tests is to understand the causal relationship between advertising activities and marketing results. 

The GeoLift theory fails in practice

The GeoLift experiments theory crumbles in face of the reality, given multiple factors and variables that are very easy to be in denial of:

• User Data Geolift testing require access to users’ location data, which raises significant privacy alarms, forcing advertisers to rely on platforms 1st party data for testing.
  Being unable to synch users targeting amongst different platforms such as: Google <> Meta means that leakage will happen.

• Location data is extremely imprecise and unreliable. If you think I’m exaggerating – just go ahead and check your own location using this link, and do the same using your mobile phone on data. 
  Did you get the same location? Was it correct?

• Spillover: Pausing a campaign in a region, will simply cause other campaigns to fill the vacuum. Controlling against spillage is like trying to fill a cup of coffee with Latte on the one side, and Americano on the other.

• Cost and Resource Intensive: Conducting GeoLift experiments requires substantial resources, including access to location data, technology infrastructure, and data analysis expertise.

• Sample Bias: GeoLift tests rely on a subset of data, and may lead to inconclusive or simply wrong conclusions.

• Waste: Showing ghost ads, or PSAs, to audiences still means paying for the inventory, which translates to wasted ad spend.
  Some GeoLift testing experts will go as far as to suggest that you hold off 50% of the ads for 5 weeks – which means that you will need to sacrifice 50% of your ad spend, for the sake of measuring the effectiveness of your spend.

• Seasonality: Unless you have control over weather, and can put a pause over the holidays - running a 4-5 weeks GeoLift experiment test will not account for the impact of seasonality. Just imagine running a GeoLift test in December and comparing the results to January…Ho Ho NO!

• Measurement Lag: Analyzing the impact of location-based marketing can take weeks, which may not align with the need for real-time marketing decisions.
  And worst of all - while conducting the test, marketers will need to not make any other changes in their campaigns. Show us the marketer who would be willing to:
  1. Not change creatives for weeks for the sake of an inconclusive test
  2. Not make changes in bids or budgets 
  3. Not start or stop any other campaigns or channels

Why do marketers keep using GeoLift  ? 

GeoLift tests feel tangible. Our biggest beef with planned experiments is that they give the sensation of tangibility: 
“I stopped and restarted marketing, therefore, I know what to expect” 
But these are as tangible as the placebo effect in pharmaceutical trials.

In marketing, given the lack of ability to control the variables – both marketers and experiment designers will convince themselves that a planned experiment gave them the outcome they expected to see.

And if not, that would just mean that the test is inconclusive, therefore, another test is required until the expected result is reached. 🤔

This is not insightful. And it’s definitely not an effective means of measurement any more. The Gold Standard of GeoLift tests has well and truly lost its sheen. 

Measuring Incrementality without Experiments

Recognizing the practical limitations of the GeoLift experimental approach, we had to re-think incrementality altogether.
INCRMNTAL went for a completely new and innovative approach to measuring incrementality in an always-on environment. Our method doesn’t require our customers to run any planned experiments.

When starting with a whiteboard, we came up with an assumption that turned out extremely valuable to how we measured incrementality: 
Our assumption was that marketers make a lot of day to day changes as part of routine optimizations. Marketers tend to change creatives, optimize bids, targeting, budgets, scale campaigns up or down regularly. Often marketers will also test new channels, pause channels or campaigns. And beyond marketing – companies will launch new product features, launch promotions, and make changes in their products. 
Each and every one of these changes – can be seen as a micro-experiment.

Here is an example of a mid-sized Advertiser dashboard, showing the # of activities our platform logged:

Luckily, our assumption was correct. Our customers make a lot of changes. 
(if you challenge this statement, just head over to Meta ads or Google ads change logs and see for yourself)
Accumulating all of these changes, and building smart algorithmic models to learn these, allowed us to build measurement models allowing marketers to measure the incrementality of each and every marketing activity a marketer may want to measure.
Questions like: “What is the incrementality we gained by increasing the bid on a specific campaign?” “What incrementality did we gain by starting a new channel?”

Thanks to our proprietary technology, INCRMNTAL stands alone as the provider of a true always-on measurement solution. Marketers can measure campaign incrementality without the necessity to stop campaigns or conduct any A/B tests or GeoLift experiments.

Here is how INCRMNTAL customers see week over week incremental performance without needing to perform any GeoLift test:

If you would like to learn more about INCRMNTAL, feel free to visit our platform page, or schedule a live tour of our platform where we would love to show you a real example of how the platform works.