Measuring marketing honestly

Marketing measurement has one hard problem: you can see what happened, but not what would have happened otherwise. Everything else follows from that.

Attribution is bookkeeping, not truth

Attribution assigns credit for conversions that occurred. It cannot tell you which of them would have happened anyway. Last-click attribution systematically over-credits whatever comes last — usually branded search, which is the channel that needed you least.

Multi-touch models spread credit around with rules someone invented. They are more plausible, not more true.

Use attribution for what it is good at: spotting change, comparing like with like over time, and settling operational questions. Do not use it to decide whether a channel works.

Incrementality is the real question

Would this outcome have happened without this spend? Only an experiment answers it:

• Geo holdouts — run the channel in some regions, not others, compare.
• Audience holdouts — withhold from a random slice.
• Switch-off tests — turn it off, watch what happens, turn it back on.

These feel expensive because you are deliberately not spending. They are cheap compared to funding something for two years that was never incremental. Brand search is the classic case: much of it converts with or without the ad.

Media mix modelling, briefly

Regression across spend, outcomes and external factors, at a level above individual users — so privacy changes do not break it. Useful at scale, needs a lot of history, and is sensitive to how it is specified. Treat its output as directional, not precise.

Lag and proxies

Slow channels need proxy metrics you can read sooner: branded search volume, direct traffic, share of voice, self-reported attribution at signup. None is the outcome, all move earlier than the outcome. Say plainly that they are proxies.

Self-reported attribution — "how did you hear about us" — is crude, biased toward recall, and often the only thing that sees dark social at all. Collect it.

Report the decision, not the dashboard

A useful marketing report answers: what changed, what we think caused it, what we are doing about it, and what would change our minds. A dashboard of every metric answers none of those.

What trips people up

• Reporting platform-attributed conversions as revenue. Every platform claims the same sale, and the total exceeds what you sold.
• Killing a channel on last-click.
• Never running a holdout because it feels like turning off growth.

You have got this when

Asked whether a channel works, you reach for an experiment rather than a dashboard — and you know which of your current numbers are proxies.