Cohorts, and Why the Monthly Number Moved

Monthly conversion rate fell from 3.1% to 2.7%. Something got worse. The same cohort logic can help interpret employee attendance tracking software, where changes in team composition can move aggregate rates.

Or the mix changed. A campaign brought a large volume of early-stage visitors, and the same customers behave exactly as before while the aggregate declines.

An aggregate metric moves when behaviour changes and when composition changes, and the monthly report cannot distinguish them. That is what cohort analysis is for.

What a cohort is

A group defined by when they entered, tracked over time.

Acquisition cohort: everyone who first visited or purchased in a given month. The most common and usually the most useful.

Behavioural cohort: everyone who took a particular action.

The key property: the group is fixed. As time passes you are watching the same people, so any change you observe is a change in their behaviour rather than in who is being counted.

What it separates

Mix effects from behaviour effects. The aggregate falls; each cohort's curve is unchanged; the new cohorts are simply larger and younger. Nothing got worse.

Cohort quality over time. If January's cohort converted 8% in its first 30 days and June's converted 5% at the same age, that is a real decline in acquisition quality — and it is invisible in a blended monthly number because the older, better cohorts are still contributing.

The shape of value accrual. How long until a cohort pays back, and where the curve flattens. See customer lifetime value.

The effect of a change on new versus existing customers, which are usually different and usually reported together.

Reading a cohort table

The standard form: rows are cohorts, columns are periods since acquisition, cells are the metric.

Read down a column to compare cohorts at the same age. This is the comparison that matters, and it is the one the monthly report cannot make. Everything in a column is like-for-like.

Read across a row to see one cohort's trajectory.

The diagonal is the current period, which is what the aggregate report shows — and it mixes cohorts of every age, which is precisely the problem.

Watch for the incomplete corner. The most recent cohorts have only had a short time, so their later columns are empty. Comparing a three-month-old cohort's total against a two-year-old cohort's total is comparing incomparable things, and it is a common error in cohort charts that show cumulative totals.

Where it earns its keep

Diagnosing an aggregate movement. The first thing to do when a monthly number moves unexpectedly. If every cohort is stable, the movement is mix.

Detecting acquisition quality decline early. Recent cohorts underperforming at the same age is the earliest available signal that a channel is bringing worse customers, and it appears months before the blended metrics move.

Evaluating a product or pricing change. Cohorts acquired before and after the change, compared at the same age.

Setting acquisition budgets. Payback by cohort is what determines how much you can afford to spend and how much working capital the growth requires.

The mistakes

Cohorts too small. Monthly cohorts on a low-volume business are noise. Use quarters, or accept wide uncertainty.

Comparing incomplete cohorts against complete ones.

Ignoring seasonality of acquisition. A cohort acquired during a discount period behaves differently by construction. Compare like periods where you can, and note it where you cannot.

Cohorting by the wrong event. First visit, first purchase and account creation produce different pictures, and the right one depends on the question.

Treating a cohort curve as a forecast. It is history. Projecting it forward is a model with assumptions, and it should be labelled as one.

Only looking when something is wrong. The value is in the trend across cohorts, which requires looking regularly.

The related tool: composition analysis

Cohorts handle change over time. The other half of the mix problem is change within a period.

When an aggregate moves, decompose it. Conversion rate by device, by channel, by new versus returning, by geography. Frequently the aggregate moved because the share of one segment changed, while every segment's own rate held.

This is Simpson's paradox in its practical form, and it is why an aggregate can move opposite to every one of its components. See metrics that mislead.

The routine worth adopting: whenever a headline metric moves outside its normal range, check cohorts and check composition before looking for a cause. Most of the time one of them explains it, and the investigation stops there.

Making it a habit rather than a project

One cohort chart in the regular reporting, not a special analysis. Cumulative revenue per customer by acquisition cohort is the highest-value single chart, and it is a straightforward query against an order table.

Update it monthly and look at the newest lines against the older ones at the same age.

Annotate it with acquisition changes — new channels, budget shifts, pricing changes — so the explanation is next to the divergence.

The summary

An aggregate moves when the mix changes, not only when behaviour changes, and monthly reporting cannot tell you which.

Compare cohorts at the same age, reading down the column. That is the like-for-like comparison.

Recent cohorts underperforming at the same age is the earliest warning of acquisition quality decline, months ahead of the blended numbers.

When a headline metric moves, check cohorts and composition first. Most unexplained movements are explained there, and the hunt for a cause never needs to start. A platform-level example is available in the Google Analytics cohort documentation.