Analysis
Tests, metrics, cohorts, forecasts and the reports built on them. A related operational category is workforce analytics software, where the same questions about metrics, reporting and interpretation apply. For statistical methods and diagnostic checks, the NIST/SEMATECH statistical handbook is a useful reference.
10 articles
Analysis
Customer Lifetime Value: What the Number Is Actually Claiming
Every LTV figure is a forecast with assumptions baked in. Which assumptions, how wrong they usually are, and why the average is the least useful form.
Analysis
Dashboards That Answer a Question, and Dashboards That Do Not
Most dashboards display everything available and answer nothing. What to build instead, and why the number without its history is the core failure.
Analysis
Metrics That Mislead: Rates, Averages and Survivorship
A rate can move because the denominator changed. An average can describe nobody. And the data you have is the data that survived. Three failures, everywhere.
Analysis
A/B Tests: The Five Ways They Go Wrong
Peeking, underpowering, multiple comparisons, broken randomisation and the wrong metric. Each produces a confident result that is not true.
Analysis
Statistical Significance Is Not Business Significance
A p-value says the result is unlikely to be chance. It says nothing about whether the effect is large enough to matter, and the two get conflated.
Analysis
Cohorts, and Why the Monthly Number Moved
An aggregate metric moves when the mix changes, not only when behaviour changes. Cohorts separate the two, and most monthly reports cannot.
Analysis
Segmentation That Survives Contact With the Data
Most segmentations describe the data and change nothing. What separates a segmentation you can act on from a clustering exercise nobody uses.
Analysis
Forecasting: What a Model Is Entitled to Claim
A forecast is a conditional statement, and the conditions are usually stripped off before it reaches the meeting. What to keep attached, and what to check.
Analysis
Reading a Vendor's Case Study
The numbers are usually true and the claim usually is not. Six questions that separate an actual result from a selected one.
Analysis
Reporting Cadence: How Often to Look at What
Looking at a noisy metric daily guarantees reacting to noise. Matching the frequency to the signal, and why most reporting is calibrated wrong.