Media Mix Modeling (or “MMM”) was first introduced by Neil Borden in 1949, as an econometric method of creating a retrospective over Advertising spend and Sales results, to create media plans, or a Marketing Mix. While the world we live in today only vaguely looks like the world in 1949 , the fundamentals of MMM have not changed.

Media mix modeling (MMM) is a powerful method, which may be used by marketers to understand the returns their advertising spend yields across various channels. It allows companies to allocate spends efficiently, maximizing the return on investment (ROI).

Unlike simpler methodologies such as Last Click Attribution, which work without any special setup or settings, an MMM will not function well without substantial Calibration.

In this article, we will explore what calibration in media mix modeling means, the typical process of calibration, and the challenges with MMM calibration.

What is Media Mix Modeling Calibration?

MMM calibration is a critical component of making an MMM tool useful. Without calibration, MMM will potentially plot out complete non-sensemaking outputs. Calibrating an MMM is more than just a step, but a process which takes patience and perseverance.  
MMM Calibration ensures the model's accuracy and reliability.

The Calibration is a process where an analyst will align the MMM model with the actual observed data, in order to make the MMM practical, and reliable. The main objective of the Calibration process is to fix any discrepancies between the predicted results and the actual results, fine-tuning the MMM models.

For example: If an MMM output claims that a channel representing 2% of ad spend is accountable for 25% of revenues, while the marketing analyst has previous knowledge that this is incorrect – correcting such a discrepancy will allow re-running the model to get more reasonable outcomes. 
And here comes the biggest concern in MMM calibration. We will expand on this down below.

But first, lets’ answer the question:

How to Calibrate a Media Mix Model?

The calibration process in media mix modeling involves several important steps:

1. Collecting relevant and accurate data: Gathering historical data, including installs, revenues, sign ups, sales, marketing spend (by channel), and external variables such as economic conditions, seasonality, holidays, app featuring, in-app events, and any other variable that might impact performance. The data is typically collected over at least 52 weeks, to capture long-term seasonality.
2. Building the MMM model: Using the data collected, an analyst will construct a media mix model, using ad stock and saturation functions to come up with a Beta (β). This process uses a regression equation.
3. Generating predictions: The MMM model is used to generate predictions, estimating how certain changes in channels will impact marketing performance. These predictions are typically the output marketers seek out of an MMM.
4. Calibration: After reaching predictions, a marketing analyst will compare those to actual results, identifying discrepancies where the model seems far off from the expected results. Calibrating the model will require a marketer to conduct incrementality measurements. 
5. Validating the calibration: Ensuring that the calibration was correct, re-running the MMM after calibration should provide more reasonable and reliable results.

MMM calibration needs to be done regularly, forcing the marketer to go back to the first, second, or third step frequently, going through the same cycle. 
Introducing a new channel, new medium, or making a change in the user funnel will cause MMM predictions to be non-viable, focusing the marketing analyst to need to re-build the MMM and run the calibration process from scratch.  

Challenges in the MMM Calibration Process 

Calibration in media mix modeling can be a complex and challenging task for several reasons:

1. Data Quality issues: The quality of historical data is crucial for accurate calibration. Missing or inconsistent data, changes in method of attribution (i.e. IDFA > SKAD), and missing data will lead to inaccuracies in the calibration process, influencing the reliability of the model. 
2. In consistent results: Different media behave differently. For example: Google today represents Search inventories, Video, Display, and Programmatic. Each of these sub channels can produce completely different results and user engagement. 
   MMM traditionally expects a linear relationship between ad spend and performance. Once the marketing analyst is required to calibrate the relationship between ad spend and results, the model is prone to human biases and mistakes. 
3. Media lag effects: Marketing activities often have a lagged effect. For example – new campaigns on Meta Ads go through a learning phase, which often will not produce incremental results right at the start. TV Advertising tends to have a lagged effect, and a halo effect over marketing performance. Including these and other effects is challenging but important for the MMM calibration process. 
4. Incorporating external features: variables such as seasonality, app features, holidays, competitor activities, changes in the app store, weather, and many more may or may not impact marketing performance. Selecting which features to include and which not on its own is a delicate and time consuming process of trial and error. 
5. Utilizing priors and posteriors: An often used shortcut marketers utilize to calibrate MMM is by using attribution data as Priors. 

Priors in principle help calibrate an MMM model by informing the model of the exact results from each channel, however, utilizing wrong priors will cause an MMM model to produce errors and nonsensical outputs. 
We wrote an article specifically about the topic of Priors in MMM. 
 

Conclusions drawn about MMM calibration

MMM Calibration is a critical component of the media mix modeling process, ensuring that marketing decisions are based on accurate and reliable predictions. 
A well-calibrated model helps marketers allocate their marketing spends more effectively, and efficiently. However, the calibration process carries several challenges, from data quality issues to external factors, and others. Overcoming these challenges is essential for successful media mix modeling and, ultimately, for achieving marketing success.