Correlation, Causation and the Marketing Report

Everybody knows correlation is not causation. Marketing reporting is built almost entirely on the confusion anyway, because the confusion does not arrive as a slogan — it arrives as a table that looks like evidence. Behavioural similarity can be observed without proving a cause; this glossary entry on interpersonal synchrony is a useful parallel.

Knowing the four shapes it takes is more useful than knowing the phrase.

1. Reverse causation

The outcome caused the spend, not the other way round.

Budgets are frequently set as a share of forecast revenue, increased when demand is strong, and cut when it is weak. So spend and sales move together because sales drive spend.

Where it shows up: any channel where budget responds to performance. Which is most of them, since automated bidding does exactly this within a platform — it spends more where conversions are already happening.

This is why platform-optimised campaigns look so good. The algorithm finds the people most likely to convert and shows them ads. The correlation is near-perfect and the causal contribution may be small.

2. Confounding

Something else caused both.

Seasonality is the universal confounder. Demand rises in the run-up to a holiday, marketing spend rises to meet it, sales rise. Every pair correlates and none of the relationships is what it appears.

Other common confounders: a price change, a competitor's outage, a product launch, PR coverage, a change in distribution, a weather event, a payday cycle.

The check: what else changed in the period? If the answer is "several things", the correlation supports nothing until they are separated. See answering "did the campaign work" honestly.

3. Selection

The people the channel reaches were already different.

Retargeting reaches people who visited your site. Branded search reaches people who typed your name. Email reaches people who gave you their address. Every one of these audiences was more likely to buy before the channel touched them.

So the channel's conversion rate reflects who it reached, not what it did.

This is the single most consequential form in digital marketing, because the channels with the best-looking numbers are precisely the ones with the strongest selection. Budget flows toward them, and total sales do not move. See what attribution actually measures.

4. Coincidence

With enough comparisons, something correlates by chance.

A dashboard with thirty metrics, sliced by device, channel and region, produces thousands of implicit comparisons. Some will move together for no reason.

Where it shows up: post-hoc explanation. The number moved, someone looks for a cause, and finds a metric that moved at the same time. That is not evidence; it is search.

The check: was this relationship predicted in advance, or found by looking?

What each one requires

They are not fixed by the same thing, which is why "be careful about causation" is not actionable advice.

Reverse causation requires an intervention you control — set the spend independently of performance for a period, and see what happens. Nothing observational resolves it, because the observational data was generated by the feedback loop.

Confounding requires either randomisation, which distributes confounders evenly, or explicitly modelling the confounders and hoping you named them all. The first is reliable; the second is what marketing mix modelling does, which is why it depends so heavily on specification. See mix modelling.

Selection requires a comparison group drawn from the same population. A holdout within the retargeting audience compares people who would have been retargeted against people who were. Comparing retargeted users against everyone else does not.

Coincidence requires pre-registration — deciding what you will look at before looking — and repetition.

All four are addressed at once by randomisation, which is why experiments are worth their cost. See incrementality.

Reading a claim

Three questions cover most of it.

Compared with what? If there is no comparison group, the claim is descriptive.

What else changed? If several things did, no single one is isolated.

Who was in this group, and how did they get there? If they selected themselves, the group is not comparable to anyone.

Language that keeps the distinction visible

The wording in a report shapes how it is read, and the shift is small.

Instead of "this channel drove 400 conversions" — "400 conversions had this channel in the path."

Instead of "email converts at 8%" — "people who received this email converted at 8%; the comparable rate for people who did not is unknown."

Instead of "the campaign increased sales by 12%" — "sales were 12% higher than the preceding period, during which the campaign ran and two other things changed."

This sounds pedantic and it is not. The first version of each supports a budget decision it cannot justify. The second is the same information without the implied claim, and it costs a few words.

And a note on internal politics: the person who has been reporting the first version will experience the second as an attack. Introducing this language gradually, in ordinary reporting rather than in a difficult moment, avoids most of that.

When correlation is enough

Not everything needs a causal answer, and treating every question as requiring an experiment is its own failure.

Monitoring. A drop in a correlated metric is a useful alarm regardless of causation.

Low-stakes decisions. Which of two subject lines. The cost of being wrong is small.

Generating hypotheses. A correlation is a good reason to run a test. It is not a substitute for one.

Where the direction is unambiguous. Site downtime and lost sales does not need an experiment.

The threshold is the size of the decision. Above a certain budget, a causal answer is worth the cost of getting it. Below it, correlation plus judgement is proportionate. Deciding where that line sits, explicitly, is more useful than treating every number with the same suspicion.

The summary

Four shapes: reverse causation, confounding, selection and coincidence. Naming which one you are facing tells you what would fix it.

Selection is the most consequential in digital marketing, because the best-looking channels are the ones with the strongest selection.

Randomisation addresses all four, which is what makes an experiment worth its cost.

And change the wording in the reports. "Had this channel in the path" instead of "drove" costs three words and removes a claim nobody can support. For a methodological discussion of causal inference, see this peer-reviewed overview of causal inference.