LiftMarketing measurement

Analysis


Reporting Cadence: How Often to Look at What

Looking at a noisy metric daily guarantees reacting to noise. Matching the frequency to the signal, and why most reporting is calibrated wrong.

Most reporting is calibrated to the calendar rather than to the data. Weekly because the meeting is weekly, monthly because the month ended. The broader study of how people use and interpret time is known as chronemics, which provides useful context for reporting cadence.

The right frequency depends on how quickly the metric carries signal, and looking more often than that guarantees reacting to noise.

The rule

Look at a metric no more often than it can meaningfully change.

A metric with high day-to-day variance, examined daily, will produce apparent movements constantly. Each one invites an explanation, someone provides one, and an action follows. That is a loop that generates work and destroys signal.

The variance is the constraint, not the reporting schedule.

Working out the right frequency

Plot the metric daily for three months and look at the spread. If daily values routinely swing 20% either way, a daily comparison tells you nothing that is not noise.

Compute the range of normal variation. Then any single observation inside that range is not information, and the reporting should not present it as though it were.

Match the cadence to the decision. If the budget decision is monthly, weekly reporting on it generates discussion without a decision attached.

Faster for things you can act on quickly. Site errors, a broken campaign, a tracking failure — these are alerts rather than reports and they should fire immediately.

Slower for things that move slowly. Brand metrics, retention curves, lifetime value. Reviewing these monthly is theatre; quarterly is honest.

The rough allocation

Immediate, as alerts: conversion volume dropping, a channel going to zero, tracking failures, data freshness, site availability. These need no schedule; they need thresholds. See dashboards.

Weekly: operational performance where you can act within the week. Campaign pacing, spend, obvious anomalies. Presented against the normal range, not against last week alone.

Monthly: channel performance, cohort curves, the reconciliation ratio, progress against targets.

Quarterly: incrementality results, mix modelling, segmentation review, definitions review, and the question of whether the measurement approach is still right.

Annually: the audit of what is collected, what is retained, what nobody uses.

Why weekly reporting misleads specifically

It is the most common cadence and the worst calibrated.

Week-on-week comparison is noisy for most marketing metrics, and it is presented as though a change were a signal.

It invites narrative. A number moved, a meeting requires an explanation, and someone constructs one. The explanation is frequently wrong and it enters the organisational record as fact.

It encourages tampering — adjusting in response to random variation, which increases variation. This is a well-established effect in process control and it applies directly.

The fix is not to stop reporting weekly. It is to report weekly against the normal range, so that most weeks the honest observation is "inside the usual range, nothing to explain."

A report where most weeks say "nothing happened" is a working report. One that finds a story every week is manufacturing them.

What to include at each cadence

The metric, its recent history, and its normal range. Always. A number without context cannot be interpreted.

A marker for known events — campaigns, deployments, price changes, banner changes. This answers most of the questions the report would otherwise generate.

An explicit "nothing outside normal" statement where that is the case. Saying it directly prevents the search for a story.

And at the longer cadences, the questions rather than only the numbers: what did we learn, what changed our understanding, what should we test next.

The meeting problem

Cadence is usually set by meetings rather than by data, and the meeting then requires content.

A weekly meeting will generate weekly analysis whether or not there is anything to say. The analysis is produced because the slot exists.

Options: shorten the meeting when there is nothing outside normal, rather than filling it. Or move to reporting by exception — a written update, with a meeting only when something needs a decision.

Reporting by exception is the higher-leverage change and it is resisted, because a regular meeting is visible work and an exception-based process is not.

The one habit worth adopting

Put the normal range on every recurring chart.

It costs nothing, it is a straightforward calculation from history, and it converts the reporting question from "what changed" to "did anything leave the range" — which is the question that has an answer.

Most of the failures above disappear when that band is visible, because most movements are inside it and everyone can see that immediately.

The summary

Look no more often than the metric can meaningfully change, or you will react to noise.

Alerts for the urgent, weekly for the operational, monthly for channel and cohort, quarterly for causal work.

Weekly comparison against last week is the worst common practice, and against the normal range is the fix.

A report that says "nothing outside normal" most weeks is working correctly — and one that finds a story every week is producing them. For a formal view of signal monitoring over time, see NIST control-chart guidance.