Choosing the Right Model

Most SAAS-based media attribution solutions are delivered in a user interface, powered by a black box. Even if the powering is “glass box,” the attribution engine is a single model for all subscribers.

My colleague Nico Newmann, associate professor and marketing measurement advisor, asked an important question recently: How do you know your attribution has selected the right model?

Fun fact – there are probably hundreds, if not thousands, of advanced attribution models from which to choose. So how do you know?  And how do you know if the model was selected to provide the determination you wanted?

For example, a popular MMM methodology is a Bayesian model. The upside to Bayesian models is that they often provide a strong signal with little data. The downside to Bayesian models is that they require a lot of intervention, in the form of strong “priors.” A prior represents initial beliefs or knowledge about a parameter. Of course, priors can be updated with actual, observed data. Or they can simply remain a subjective estimate. In the case of the latter, a SAAS product powered by a Bayesian model makes a lot of assumptions about your brand’s data. That’s risky.

What’s the best solution?

Ensemble models. We like to say “let the data decide” which methodology is most accurate for our clients’ attribution, optimization and forecasting. And what our data often decides is that the answer is not a one-size-fits-all, but a strategic mashup of models for the most agile media mix modeling.