What if Performance Advertising isn’t Just an Analytics Scam?

If you haven’t read Rand Fishkin’s What if Performance Advertising Is Just an Analytics Scam? then I’d highly recommend you do.

It’s a near-perfect archetype of the sort of issues that get raised by someone who knows just enough about performance marketing to know the contentious areas, but not enough to understand their solutions.

Many of the claims in Fishkin’s article will be familiar with the criticisms raised, having heard them from CMOs, clients, and other stakeholders over the years. For those less familiar, I’m going to run through them, and some of the issues with Fishkin’s line of argument.

Selection bias

Fishkin starts his article with a quote from Brian Chesky, Airbnb’s CEO, which amounts to:

We cut our ad spend to $0 and our overall sales dropped by only 5%.

The article later references similar case studies from Uber, Bloomberg and Ebay, amongst others. In each of these cases, the brand in question drops their ad spend by some seemingly drastic amount, and sees little to no change in overall demand.

As impressive as these examples might seem, we have to consider the fact that no company would ever publicise their results had they observed the opposite. No CMO is going to go on Medium and write an article called We dropped our spend by X%, and saw our volume drop by X%. Even if they did write such a case study, it’s unimaginable that…