The dashboard was green. Volumes up, drop-off flat, conversion holding. Everything you want to see on a Monday morning.
And it still felt wrong to me.
Not wrong in a way I could have argued in that room. Wrong in the way a number feels when it doesn't match the product you've been using every single day. If you've run a team for any length of time, you know the feeling. That's what this piece is about: when to trust intuition over data.
Key takeaways
- Your gut usually isn't fighting the data. Most of the time it's telling you something's off in the way you're collecting it.
- When they disagree, you've learned something either way. Either the product has a problem or the tracking does, and both are worth a day of someone's time.
- Tell your team where to look, never what to find. Otherwise all you get back is your own opinion, wearing a chart.
When to trust intuition over data
We're all trained to defer to the numbers, and most of the time that's right. But a dashboard isn't reality. It's a chain of small decisions somebody made months ago about what counts, when it counts, and who quietly gets left out. Any one of those can go stale, and nobody sends you an alert when it does.
And your gut isn't magic either. It's years of watching the same product, the same customers, the same team. When it goes off, it's comparing what's on your screen against everything you've seen before. That's not a feeling. That's experience doing its job.
So what do you do? Trust the data and let the feeling go? Or trust your gut and go digging into the numbers?
Here's how I started handling it. When my gut said a number was wrong, I stopped treating it as an argument with the data, and started treating it as a signal that something might be broken in how we were measuring it.
Not overruling the dashboard. Just going and checking.
The specifics changed every time, but the pattern almost never did. It usually went like this. A funnel looks healthy. Completion rates steady, week after week. The product manager presents it with real confidence, because the chart genuinely says what they're saying it says.
Except you went through that same flow last week and it wasn't smooth. Support has been noisier than usual. Somebody in sales mentioned customers asking the same question twice.
None of that is evidence. All of it is signal.
So you ask for a re-check, and two days later it turns out the success event was firing when the confirmation page loaded, not when the transaction went through. Which means everyone who hit an error after clicking submit was counted as a happy customer. The funnel wasn't healthy. It just couldn't see.
Where the tracking usually breaks
Once you've chased a few of these, you start to notice it's almost always one of four things.
- The event fires at the wrong moment. On page load instead of on success. This is the most common one, and the most expensive.
- A filter is quietly leaving people out. App users missing from a report someone built for web. One region, one browser, one payment method, gone. And nothing on the chart tells you.
- Somebody changed what a word means. "Active" or "qualified" got redefined for a perfectly good reason, and nobody told the people reading the trend line.
- The number you're dividing by moved. The top half is fine. The base changed, so the percentage is telling you a story that never happened.
None of these are exotic. They're boring, ordinary mistakes that survive for months because the chart keeps loading every morning like nothing's wrong. I've written more about this in The Performance Playbook, but the short version is simple. A system that confidently reports the wrong thing is far more dangerous than one that reports nothing.
How to ask your team without putting words in their mouth
This is the part most of us get wrong, and it decides whether any of it is worth doing.
Say "this number should be lower, go find out why" and you will get a lower number. Not because anybody is being dishonest. People want to be helpful, so they find whatever you told them to find.
What I did instead was name the area, never the answer. I'd tell the team I had a strong feeling about a particular issue and ask them to please check that angle too. I wasn't claiming I knew better. I was asking them to give my instinct and my experience some weight alongside the query they'd already written.
Three things make that work. Point at an area, not a result, so "something feels off between step two and three" rather than "that number is wrong". Say out loud that you might be wrong, because that's what lets an engineer come back and tell you the tracking is clean. And ask what experience they'd expect to see in the data. If the two of you can't agree on what good looks like, that metric was never going to mean much anyway.
Now, was I always right? No. Roughly one time in ten the data was right and I was wrong.
But do the maths. One day of an analyst's time, against a metric that quietly misleads a quarter's worth of decisions. That's worth doing every time, even knowing you'll occasionally look foolish in front of your own team.
What you can't afford is needing to be right. The moment being correct matters more to you than knowing the truth, you've stopped investigating and started arguing.
I see the same thing when I'm coaching
What surprises me is how often this shows up in work that has nothing to do with dashboards.
A client is telling me a story. And somewhere in it, I get a strong sense of something they haven't said. Something sitting under the words, that they may not have reached themselves yet.
I've learned to say it out loud.
But how I say it matters far more than the observation itself. I make it very clear that I'm not judging, that this is coming purely from curiosity. And I tell them plainly that they're free to reject it, or simply not answer. Then I let go of the outcome, because I'm not attached to being right.
Sometimes it opens something big. Sometimes they say "no, that's not it", we move on, and nothing is lost. That's the whole discipline. You offer what you noticed as a possibility, and hand the other person full authority to test it. In coaching that person is your client. In a review meeting it's the engineer who has access to the query.
Same muscle. Say what you noticed, hold it loosely, let somebody else check it.
What to do on Monday
- Take the metric you quote most often in leadership meetings. Ask an engineer to walk you through exactly where that event fires and what the base number is. Not what the wiki says. What the code does.
- Go and use your own product this week. Complete the flow your dashboard measures. Your instinct needs fresh input, and most of us run on a mental model that's two years old.
- Write your hunch down before you ask anyone, and date it. Then name the area, and say you might be wrong. After a quarter of doing this you'll know your real hit rate instead of guessing at it.
- Pick one word a month and check everyone means the same thing by it. Active, qualified, complete. You'll be surprised. Running that kind of review is fairly standard technology advisory work.
Frequently asked questions
Isn't this just confirmation bias?
It would be, if my hunch were the verdict. But it isn't. It's only the reason somebody else goes and looks. Confirmation bias is deciding the answer and stopping there. This is suspecting something and then handing the check to a person who can prove you wrong.
What if I'm too junior for my gut to carry any weight?
Then bring what you saw, not what you concluded. "I ran this flow four times and hit an error twice, but the chart says 98% success" isn't an opinion anybody can wave away. Specific and checkable beats senior, every time.