So TV channels called the election and Joe Biden is the winner . Some of us felt elated and others refuse to accept it – and a large spectrum of people in between those extremes . No value judgment from me on that in this post – I just wanted to draw some parallels with the corporate world that I am a part of .
1. We often don’t trust data
The (often unrealistic) gold standard in corporate circles is “data driven decision making” . The hard part is not about having data – it is about whether you trust the data .
Just as right leaning folks won’t trust data shown on CNN or left leaning ones doing the same with Fox News – corporate world has its issues . The search for “one source of truth” was on when I entered the workforce in the 90s and it is still on . The reality is the best case is you settle for a version of truth that you make peace with and you stop worrying about other perceptions at some point . This is why CFOs get frustrated when the CIO implements a costly data lake and fancy BI on top – and the sales leader still believes “Big Ken’s excel file” .
2. We often don’t understand what the data is trying to tell us
Election results have been coming in for five days and it was clear for the professional data scientists where this was headed . But did it make any difference to people who were watching who leaned left or right ? Hardly ! Right leaning folks looked reasons why it’s all false and left leaning ones tried to hold their breath and tried hard to contain their excitement .
This is true in corporate world too . You look at data and try to find a way to fit it to your “world view” . This is why many corporate leaders do more of the same expecting different results . We only see and hear what we like and filter out the rest . Taken to an extreme that also means we often only collect and measure what fits our narrative .
3. We tend to think in binary terms
Last election , the polling industry lost their face in a big way . They thought Clinton will win and Trump won instead . That led to Trump and Clinton supporters both stopping to believe the models from statisticians like Nate Silver . It didn’t matter that many professionals tried to explain that what the model says is the chances of each candidate to win . If Trump had only 20% chance to win on the last day of last election – that didn’t mean that Clinton was sure to win . It just meant that she had a much higher chance to win . But that’s not how we see it – we often think in very binary terms .
I run into this routinely at my clients – especially when having discussions on data science related projects . There are only a few people who instinctively understand what probability means and that it is not binary . The smart ones immediately start mitigating the risk in various ways – but often I have to nudge them in that direction .
4. We can’t easily abstract and rationalize
Data often doesn’t plot into nice line or curve . It will always have some outliers . When we know of ten votes that didn’t get counted or three dead people who seem to have voted – its natural to think that the entire election is rigged . We often cannot easily think through whether there are enough of such votes to have changed the outcome of the election .
Similarly we look at aggregated and/or filtered data and make decisions that might not be useful . So when we wonder how a red state turned blue when everyone we know is a Trump supporter in that area – we don’t often realize that there are significant variations between zip codes or even within zip codes . We also don’t quickly realize that the dozen people we know in Georgia doesn’t represent all of Georgia 🙂
This happens quite often in the corporate world . Most decisions are done using aggregated data . I will make one example from a few years ago where a sales leader decided to over invest in the west coast business because sales was booming and every director there got two extra reps . A year later – sales rose only modestly and profit dropped a lot . It was a simple case of only one small part of west coast business over performing and everyone else not seeing enough demand . When you don’t know the details – you can make terrible decisions and get confused when you don’t get results .
I will stop here – there are probably a dozen more parallels . My puppy insists I need to go play with him 🙂
And Vijay Says... 