Media Mix Modeling (or “MMM”) was first introduced by Neil Borden in 1949, as an econometric method of creating a retrospective over Advertising spend and Sales results, to create media plans, or a Marketing Mix. While the world we live in today only vaguely looks like the world in 1949 , the fundamentals of MMM have not changed.
Media mix modeling (MMM) is a powerful method, which may be used by marketers to understand the returns their advertising spend yields across various channels. It allows companies to allocate spends efficiently, maximizing the return on investment (ROI).
Unlike simpler methodologies such as Last Click Attribution, which work without any special setup or settings, an MMM will not function well without substantial Calibration.
In this article, we will explore what calibration in media mix modeling means, the typical process of calibration, and the challenges with MMM calibration.
MMM calibration is a critical component of making an MMM tool useful. Without calibration, MMM will potentially plot out complete non-sensemaking outputs. Calibrating an MMM is more than just a step, but a process which takes patience and perseverance.
MMM Calibration ensures the model's accuracy and reliability.
The Calibration is a process where an analyst will align the MMM model with the actual observed data, in order to make the MMM practical, and reliable. The main objective of the Calibration process is to fix any discrepancies between the predicted results and the actual results, fine-tuning the MMM models.
For example: If an MMM output claims that a channel representing 2% of ad spend is accountable for 25% of revenues, while the marketing analyst has previous knowledge that this is incorrect – correcting such a discrepancy will allow re-running the model to get more reasonable outcomes.
And here comes the biggest concern in MMM calibration. We will expand on this down below.
But first, lets’ answer the question:
The calibration process in media mix modeling involves several important steps:
1. Collecting relevant and accurate data: Gathering historical data, including installs, revenues, sign ups, sales, marketing spend (by channel), and external variables such as economic conditions, seasonality, holidays, app featuring, in-app events, and any other variable that might impact performance. The data is typically collected over at least 52 weeks, to capture long-term seasonality.
2. Building the MMM model: Using the data collected, an analyst will construct a media mix model, using ad stock and saturation functions to come up with a Beta (β). This process uses a regression equation.
3. Generating predictions: The MMM model is used to generate predictions, estimating how certain changes in channels will impact marketing performance. These predictions are typically the output marketers seek out of an MMM.
4. Calibration: After reaching predictions, a marketing analyst will compare those to actual results, identifying discrepancies where the model seems far off from the expected results. Calibrating the model will require a marketer to conduct incrementality measurements.
5. Validating the calibration: Ensuring that the calibration was correct, re-running the MMM after calibration should provide more reasonable and reliable results.
MMM calibration needs to be done regularly, forcing the marketer to go back to the first, second, or third step frequently, going through the same cycle.
Introducing a new channel, new medium, or making a change in the user funnel will cause MMM predictions to be non-viable, focusing the marketing analyst to need to re-build the MMM and run the calibration process from scratch.
Calibration in media mix modeling can be a complex and challenging task for several reasons:
Priors in principle help calibrate an MMM model by informing the model of the exact results from each channel, however, utilizing wrong priors will cause an MMM model to produce errors and nonsensical outputs.
We wrote an article specifically about the topic of Priors in MMM.
MMM Calibration is a critical component of the media mix modeling process, ensuring that marketing decisions are based on accurate and reliable predictions.
A well-calibrated model helps marketers allocate their marketing spends more effectively, and efficiently. However, the calibration process carries several challenges, from data quality issues to external factors, and others. Overcoming these challenges is essential for successful media mix modeling and, ultimately, for achieving marketing success.