There have been a few times in my career when I’ve consciously said that things were going well, but that I had a little voice in the back of my head telling me something I was subconsciously trying to ignore. Fortunately for me these have been, in the grand scheme of things, pretty benign – mostly thoughts that I was working on the wrong thing, not that something more sinister was wrong – but time is all you have, and it’s better to spend it working on the right things.

I once spent weeks working on an automated anomaly detection program, whose scope grew from specific insights for a specific data set to a general-purpose framework for large-scale data. I got advice early on that this wasn’t such a good idea - since when you have a million slices, thousands of p<0.001 “anomalies” will show up - but I didn’t really address it and forged on. The silent reason that I was ignoring was that I really wanted to build this to have a promotion-worthy project, but fundamentally our team was one with a few too many engineers chasing too little high-impact work. I ended up moving teams shortly thereafter - but if I had just talked this out with my manager and skip level I could’ve found something more productive to do much sooner.

This also applies to business-level problems. On that next team, I once spent weeks digging into customer lifetime value (CLV) data, looking for ways to make us profitable. I came up with some ideas that sounded nice - improving CLV 25%, wow! But it all ignored the larger issue: almost every customer we had was negative CLV, and it got worse, not better, the more they used our app. I should’ve tackled that, head-on, drastically - maybe it wouldn’t have changed anything, but it was the best chance we had.

Listen to the little voice!