The Problem With Prettier Reports
Automated reporting is one of the first places non-tech companies deploy AI — it's low-risk, fast to implement, and produces something executives can see. The output looks clean. That's the problem.
Dave Kellogg published a direct warning this week: smoothing metrics — rolling averages, trend lines, normalized views — masks the real signal. His argument is aimed squarely at board decks and QBRs, but the implication for AI-generated reporting is sharper than he states explicitly. When a human builds a dashboard, smoothing is a deliberate choice. When an AI tool generates it by default, smoothing becomes invisible infrastructure — and nobody questions infrastructure.
What AI Reporting Tools Get Wrong by Default
Most AI-assisted BI tools — whether that's Looker with AI summaries, a GPT-connected data layer, or a purpose-built RevOps dashboard — are optimized to present trends clearly. Clarity, in practice, means reducing noise. Reducing noise means suppressing variance. And variance in pipeline, churn, or conversion data is frequently the earliest signal you have that something structural is breaking.
A single bad week in outbound conversion gets averaged into a rolling four-week view and disappears. A cohort of accounts that churned in a cluster gets absorbed into a monthly retention rate. The AI-generated summary says "retention remains stable." It is not lying. It is just not telling you what matters.
This is not a data science failure. It is a workflow design failure — one that happens at the point where someone configures what the AI is supposed to produce.
The Fix Is a Reporting Standard, Not a Tool Change
You do not need different software. You need a standing rule: raw period-over-period data must appear alongside any smoothed or trended view, and the smoothed line is never the only version your leadership team sees.
In practice, that means:
- QBR and board templates should include a raw weekly or bi-weekly data column, not just the rolling average chart.
- AI-generated summaries should be prompted to flag outlier periods explicitly before offering a trend narrative.
- Whoever configures your reporting AI — internal RevOps, a fractional leader, an analyst — needs an explicit brief that variance preservation is a requirement, not a nice-to-have.
The operators winning with AI-assisted reporting are not the ones with the cleanest charts. They are the ones who designed their AI outputs to surface discomfort, not suppress it.