For most of my career, how a report looked told you something about how it was made. Sloppy formatting, inconsistent numbers, a chart with no axis label. Those were signals. Not proof, but a reasonable reason to go and check the work before you acted on it. The reverse held too. A clean deck from a vendor, a tidy benchmark from the 20 Group, a confident page from a consultant, each earned a little trust before anyone had checked a single figure.
That shortcut no longer works, and I want to explain why it matters more than it sounds.
Two kinds of quality
Any piece of analysis has two qualities that have nothing to do with each other.
The first is presentation. Is it coherent, well structured, easy to follow, and does it look like someone competent made it. You can judge this in thirty seconds.
The second is whether it is true. Are the numbers right. Are the sources real. Do the conclusions follow from the evidence, or are they sitting next to it. You cannot judge this by looking. You have to go and check.
Presentation is immediate. Truth requires work. So for as long as there has been a correlation between the two, people have used the first as a proxy for the second, and mostly got away with it.
The signal is gone
The correlation is now broken, and not only because of software. A vendor with a good designer and a bad method produces a beautiful attribution report. A benchmark that averages thirty stores that look nothing like yours arrives in a format that looks authoritative. And yes, a language tool will take a confident guess and dress it in prose that sounds like a finding. I have watched it happen to me. The document looks finished. Then you start reading closely and the data underneath is wrong, or an assumption got made in the second paragraph that the rest of the document is built on, and the whole thing is lipstick.
The point is not that any of these sources is bad. It is that a fundamentally incorrect piece of analysis is now indistinguishable, at a glance, from one produced by an excellent analyst. How it looks is no longer a reliable signal of how it was made. Anyone still using it as one will be fooled, and will not know by what.
Agreement is not evidence
There is a quieter version of the same problem, and it happens in your own head.
You walk in with a view. You ask for analysis. What comes back, from a vendor, a manager, or a tool, elaborates your view, improves the way it is expressed, and supplies supporting arguments you had not thought of. It feels like independent confirmation. It is not. It is your own premise, returned to you in better clothes. Nothing in that exchange tested whether the premise was true.
I have found that the more articulate the person asking, the easier this is to fall into. A smart executive with a strong framing will get sophisticated analysis built entirely inside that framing, and the conversation will feel exceptionally intelligent while adding no information at all.
So the question I ask of any source now, human or otherwise, is not whether it sounds right. It is whether it is making me harder to fool, including by myself. Does it challenge the assumption. Does it tell me something I did not already believe. Does it say, when it should, that the available information does not support a conclusion. If it never does any of those, it is not analysis. It is agreement with good formatting.
Observation, inference, verification
What replaces the lost signal is a habit, and it is a boring one.
Keep three things separate. What was observed: the actual record, the actual numbers, read at the source. What was inferred from it: the story that explains the observation. And what has been independently verified: the inference checked against something that was not used to produce it. Most reports blur the three into one confident voice. The good ones label which is which, and are willing to leave a question open when the evidence does not close it.
A confident error with the word possibly inserted every few sentences is still an error. Hedging is not the fix. The fix is knowing, line by line, which statements are supported, which are inferred, and which nobody has checked.
What this means for a principal
Stop grading reports on how they look. Pick two numbers in any deck and trace them back to the source yourself. Ask every advisor, vendor, and tool what would have to be true for their conclusion to be wrong, and notice who can answer. Be most suspicious of the analysis that agrees with you most fluently.
And when something looks finished, treat that as the moment to start reading, not the moment to stop.