And Vijay Says…

Pressure is a fact of life – especially so in businesses. You cannot eliminate it, but you can probably minimize it with good preparation and training and so on. But whatever you do – you are bound to end up in high pressure situations from time to time. It could be your boss promising more profit to the board or the market changing too fast and all your carefully laid plans going for a toss – there are many reasons why you may end up there.

The part you have a greater chance to control is how you handle it as a leader. And I think this is where a lot of traditional tactics that we seem to take for granted become counter productive. I want to highlight just three such “common strategies” and why they may not yield the expected results .

Centralized decision making 

The most common tactic when pressure builds is for people higher in the hierarchy to take direct control of execution – especially of what appears as low hanging fruit. You will start seeing memos that go “All travel needs to be approved by the CFO” , “All purchases over $100 need to be approved by the GM” and so on.

On first blush it seems like a real good idea. People will be careful about what they spend money on because no one wants to go to the scary CFO to ask for a $500 air ticket. And multiplied by hundreds of people – that is real cost that is saved.

But what is the reality ? The good sellers will switch to a competitor with less draconian rules. The next best will stay paralyzed thinking they can push their deals when the rules ease off. The CFO would rather spend $500 to get the $100K deal – if only he knew. But this set of people won’t always ask. They will wait. It proceeds along similar lines across all segments. CFO might win the cost battle for a short time and still  lose the larger business war over time.

The accounts payable clerk could have handled this $100 restriction – or automation could have, if the company had the IT skills needed. So on top of killing everyone else’s productivity – the CFO now has a lot more low value work to do. Is that the best use of a highly paid executive’s time ? You cannot manufacture more time for yourself even if you have the most powerful title !

What about the mid level managers who actually lead their teams ? They feel powerless, and for the most part their teams realize it in no time and lose respect for them. So now you have a morale issue and the good managers start floating their CVs in the market too.

A CFO I respect a lot once told me – about a decade or so ago –  “When the business knows what they are doing, my job is to report what they do. When the business does not seem to know what they do – my job is to get it to a shape where I can get back to reporting again”. I think my appreciation for his words have increased every year since then.

Cutting the workforce 

When the going gets hard and saving travel and stationary cost is not enough, leaders have to let go of people. It is a harsh reality of business. Many companies manage cost very effectively on an ongoing basis and even they will occasionally be pushed to cutting head count significantly from time to time.

The traditional wisdom is to cut the bottom rungs first. What gets ignored or forgotten typically is the difference in org structure to handle peace time vs war time. In peace time – you need the matrix and hierarchy to make sure you are investing sufficiently for future. So you will see roles like “Chief of transformation” , “Chief of culture” and so on – and with great conviction, the big bosses will put their best leaders in those roles. That is absolutely the right thing to do as well in that context.

War time is very different. You need your best leaders leading the charge – dealing with the market and your employees directly. If they truly are your best leaders, they cannot be hidden in internal roles managing spreadsheets and on vague ideas.

It is common to hear “all hands on deck” messages from the top at war time. But how many of these peace time roles are actually redeployed to the front line immediately at war time? How many of those senior and expensive people are shown the door if they don’t have the war time skills needed to keep the company alive ? And what saves more money – getting rid of peace time leadership roles at war time , or getting rid of a lot of lower cost less experienced people ? Same question about over lay and ops support roles – matrix requires significant operational overhead. When you are fighting to stay alive, does it really matter much how many ways you can slice and dice your results ?

In my admittedly limited exposure – I have always felt that most leaders are optimists. They think of all troubles in business as temporary – and hence will go away very soon. So why go through the trouble of redeploying etc when there is a less complex way that looks good on a spreadsheet ?

Over communicating 

One of the things that leaders often encourage their teams to do the moment pressure starts mounting is “Please over communicate”. This often happens after most of the critical decision making has already been centralized – thereby reducing the usefulness of the lower level managers . Pleas to over-communicate  is done with great intentions as well –  for example, if people can alert their bosses of important issues early – they can help solve it before it becomes a disaster.

But what really happens when leaders try to over communicate ?

Even when there is no real pressure, communication is not usually a real strength for many business leaders. When they start to over communicate under pressure – the team starts to wonder a few things – “Wow this sounds desperate – should I brush up my CV” , “Clearly you have no respect for my time – and I seriously doubt you understand what my job really is” , “I lost you after the first five minutes – can we get to the point?”, and so on. The reason is simple – the more senior you are, your only way to over communicate is via some “one size fits all” strategy. When people are under pressure – they need clear instructions and specifics, the exact opposite of “one size fits all”.

When the bosses talk in generalities – what are the chances of the lower level employees to go back to them with specific topics ?

So what would be a sensible approach for leaders to handle pressure in their business ?

1. If you are going to war – declare it explicitly so that all your troops hear it loud and clear .
2. Define what it takes to win the war. Delegate battles effectively and stay focused on the war.
3. Act decisively to get rid of the peace-time org structures and redeploy troops for war.
4. Treat those who are not fit to be warriors with extreme kindness – and try to make it up to them when peace-time returns
5. Take the lower level leaders into confidence and empower them – centralize strategy as needed without choking information flow, and decentralize execution and communication.
6. Keep everyone posted as needed – focus on specifics. Resist the urge to over-communicate especially in “one size fits all” fashion.

“Football is two things. It’s blocking and tackling. I don’t care about formations or new offenses or tricks on defense. You block and tackle better than the team you’re playing, you win.” – Vince Lombardi

Just like in football, there is a perception that blocking and tackling has no glamor and it is beneath us to focus on those activities if we were to be successful. Not a month passes without me getting an SOS ping from someone with “Save me from this madness – I don’t want to be stuck in creating decks, business cases, managing utilization and creating pricing sheets. I want to be in the strategic stuff”. I have some empathy – I have made such calls too when I had their roles in the past. And one such call today is the reason I started typing this post 🙂

The main reason to hate the blocking and tackling work is because we generally don’t know why we do these “boring” tasks at all when we start out.

My favorite way of getting people (usually my younger colleagues) to see the WHY aspect is to bring them with me to a meeting where their work is used. A few years ago, a young senior manager came to me for some advice and told me how much she hated the mindless work around pricing deals. So I asked her boss to have this senior manager present the deal on his behalf for the deal review next time.

Our CFO asked her “Why should we make this investment instead of putting the money in an index fund”. She did not have a good answer, but her boss did. He walked us through the fundamentals of the analysis she had come up with and showed why it is a better deal than alternate ideas we had suggested.  She knew the mechanics of pricing very well – she just did not know the context of how her work was used by others for their decisions. Now she is masterful in how she crafts business propositions and is well set for an amazing career. To her credit – she now explains the WHY aspects of the work to the people in her team when she assigns them such work.

People who grow up learning to block and tackle well will have some advantages in business that are hard to learn later in life. I learned this from the head of manufacturing of a car company some twenty years ago when I was a young consultant collecting business requirements for an SAP implementation project.

He used to let me sit in his Tuesday morning staff meetings and for every problem that came up – he would ask “Should we make this problem go away, or should we solve it?”. That was a test to see who knew WHY the problem exists and hence can explain how to make it go away by making upstream changes. Over the course of several such meetings, I realized that people who passed the test were the ones that got better roles and more money and so on. It was counter intuitive for me on why he thought less of the people who just thought about ways of putting in a solution for the problem without questioning why the problem came up in the first place. Those were the people who got a lot of extra coaching and formal training. The irony was that often the solutions proposed by this second set of people were the “requirements” I captured for the ERP implementation 🙂

I am a big fan of delegation and have written about my thoughts on how to do so effectively a few times already.  As I look back at my own career, a good part of my success over the years happened because my bosses felt comfortable delegating more and more to me. And the reason they had that confidence in delegating to me was because they knew I had a good command over the blocking and tackling aspects of the job, and they were positive I can make “some problems go away for good” because I have a first principles understanding of the issue. Now – the honest truth is that I did not always have that understanding . There is a little halo effect that you earn for yourself when you are good at blocking and tackling – and that buys you a bit of extra time and room for mistakes to make the problems go away.

Blocking and tackling are less boring when you understand the WHY aspects . But that does not mean that boredom and grief will not return if all you do is block and tackle. When you are really good at blocking and tackling – you become good at solving problems the RIGHT way . The next logical step up from there is solving the RIGHT problems.

The people who have the most fulfilling time at work are the ones who can do both – identify the right problems to solve, and then solve it the right way. The reality is that majority of leaders become good at only one of these two things, and hence companies need multiple leaders working in tandem to bring both these much needed skills together . Personally, I prefer leaders who are awesome at execution to the ones who are hailed as strategic thinkers. To be more precise – I believe it is a step in the wrong direction to separate execution from strategy. There is no such thing, in my opinion, as a great strategy, poorly executed. Strategy that does not take into account the ability to execute is at best called marketing/PR/advertising. At worst it is just called a MISTAKE.

Yesterday evening, while flying from PHX to JFK, I had a chance to read this excellent blog by Arun Murthy . If you have not read it yet – pls read it first before you read my rant below. As always – these are strictly my personal views .

First – I think Arun is probably the best person to write a blog like this and needs to congratulated for doing so. He has been part of the Hadoop story from the very beginning and continues to play a significant part in shaping its evolution. It also felt that it came straight from the heart – and extra points for starting lists from 0 and not 1 🙂

My own point of view was developed over the last couple of decades as a data geek who worked on a range of technologies on data management as well as analytics for a wide variety of clients across the world.

Data is largely an unsolved problem in the large enterprise world. Just when you think you have it under control, you realize that the problem got bigger and more complex. And you also realize the tech has improved and now you have more options on how to solve the bigger problems. This is one reason why most techies eventually use philosophy as a framework for explaining the evolutionary nature of their work.

It is hard to predict which way an exploratory project will go. This is great for developers as it gets their creative juices flowing. But that is not how enterprise CIOs think. They value high doses of stability and predictability , and very low doses of complexity. They very well know that the pundits will use terms like “legacy thinking” to shame them all the time. But their world comes with flat and declining budgets and there are always a lot of lights to be kept on. Within all those constraints – the good CIOs try to foster as much innovation as possible. And of all the innovation they have sponsored in the last decade or so – Hadoop definitely was top of the list.

World was ready for Hadoop . Classic datawarehousing had been pushed to its limits. Data warehouses became data dumps. Cost of maintaining those datawarehouses started driving everyone nuts. And Hadoop promised a solution for all these and more. Plus the open source nature gave all the geeks even more incentive to introduce it in their shops.

In my opinion, 4 things led to Hadoop’s alleged demise

1. Too many options for clients to choose from
2. Unskilled people implementing it
3. Multiple changes in market positioning 
4. High operational complexity

Pretty soon – everyone ran into challenges. MapReduce was no longer sufficient to do most of what enterprises wanted to do. No worries there – Spark etc came up just at the right time and took over. The world realized that you just cannot run away from SQL even if you criticize it heavily. So many different SQL on Hadoop projects came into being and that did not always work in the way traditional IT shops expected. IT shops are not used to having tremendous choice in solving problems. When Cloudera and Hortonworks proposed different solutions to a problem – be it SQL, be in security or whatever – it became very confusing for the people who were trying to implement a long term solution in their shops. In short – “Hadoop is a philosophy” started getting interpreted as “there are no real best practices here – just keep experimenting” by a lot of clients. Just to keep it brief, I am skipping the divergent direction MapR took – and that story did not end well either.

Then came the question of skills. For enterprises to adopt technology faster – you need a lot of people with that skill. Much like how SAP market got flooded with poor skills when ERP was hot – Hadoop market did too . That had a direct effect on the quality of implementations. Many clients are still struggling with tech debt caused by using developers and architects who did not have good fundamentals in data management. Net net – hardly anyone replaced any data warehouses , and data lakes became the new data swamps. To be fair – the growth of classic data warehouses have been significantly curtailed since hadoop became mainstream.

Arun has already explained the “What is Hadoop?” question in great detail. So I will skip that entirely.

What also did not help a lot was the positioning of hadoop companies changed over time – perhaps to sustain the insane valuations in private markets. It swung from data management to analytics and ML to managing everything in cloud. When you try to do everything – even if it is a great problem to solve – it is hard to execute to perfection, and it confuses clients a great deal.

The last point that made it difficult for Hadoop was operational complexity. Data management is a lot of fun for developers. But the moment it is in production – rock solid operations is what keeps it going. Even for RDBMS based systems – DBAs and other Ops experts with all the mature tooling still spend significant time managing their landscapes. Hadoop ( to be fair – most NoSQL DB too ) did not prioritize ops sufficiently. In my view at least – this was perhaps the biggest miss and one I think Cloudera and others should urgently address. Every client I know will be grateful if managing Hadoop was significantly simplified – especially between on-prem and cloud.

All this said – I don’t think Hadoop is dead , or that it will die. It will continue to evolve and world of data management needs that innovation and open source communities to thrive. But if the four points I raised are not addressed – I seriously doubt Hadoop will reach its potential any time soon.