And Vijay Says…

This is one of those topics where my employer IBM has a vested interest and consequently I need to state upfront that what I say below is strictly my own views from my limited view of the large enterprise world. My personal experience over the last couple of decades has been in the very large company space. I have not spent enough time in small and medium companies to know their situation first hand. So please take what I say with a pinch ( or pound ) of salt . Also to preserve friendships that I value, I am not going to make any statements on specific cloud vendors. 

For once, customers led and vendors followed . 

I know several commentators on social media call proponents of multi cloud as snake oil salesmen. What they conveniently forget in this case is that multi cloud is a reality and vendors woke up to it late and smelled the coffee. In the large company space I am familiar with (granted it is a small set – but also the set that everyone else eventually tends to follow in enterprise IT)  – I cannot think of even one that does not have several cloud vendors they work with at some scale. Most common pattern that I have seen is their own private cloud, one major public cloud and then a few others at negligible scale (often brought in via developers trying out new things). In a few cases, I have seen two public clouds used at scale – commonly as the result of some corporate M&A activity .

In short, multi cloud is a reality today – not some futuristic idea. As Eagles crooned – You can stab it with your steely knives, but you just can’t kill the beast. Every company struggles to manage it (Most cannot even monitor – let alone do some active management) and that is why vendors are on the job trying to solve it.

I don’t know why, but now I am now humming We didn’t start the fire 🙂

Multi cloud for Bursting / cost arbitrage – no one does it well…yet

While in theory – a lot of people would like to be in an ideal world where workloads can be sent intelligently to execute wherever it is cheapest to execute, that is still enterprise utopia. Even in the simplest case of your own private cloud and just one public cloud – seamlessly moving most of your apps from one to another is not easy to pull off . The two common use cases are when peak compute is required – like when month end closing happens, or thanksgiving sales happen , and when disaster recovery happens and you want massive fail over to happen to a different public cloud . You can do both  for well engineered applications today – but that is a very tiny part of the overall landscape, and won’t buy you much.

You can check out any time you like, but you can never leave !

(Sorry – the Eagles song just keeps playing in my brain in a loop).

In an enterprise with multiple clouds – the normal scenario is that data and algorithms don’t sit next to each other . One will need to move to meet the other when it is needed. Moving data is expensive – mostly because companies love to create data, and no one likes to delete anything . Unfortunately some data cannot move for legal or cost reasons, and some algorithms may not move because its vendor has no incentive to expose it elsewhere.

It is already true that you will need to do one-off extremely high business value solutions that need multiple clouds. It will only become more and more true in future. As an example, say five years from now (may be less)  – you may need a multi cloud scenario where a specialized solution that needs quantum computing from one provider, AI APIs from another and some good old algorithms from your own private cloud to manage enterprise risk.

The one off solutions usually don’t come with a lot of governance. So others will build little apps on all these clouds for other one-off solutions . Proliferation is more or less a given.

That was a long winded way of saying – multi cloud is very useful in adding significant business value ( since no one party will ever be the lone innovator) and hence you will use it. But once you are there – you are better off learning to live with it peacefully than just fight it all the time.

Containers are awesome – but only when used as intended 

Everyone wants to work like Netflix and Amazon. But the reality is that they all have massive baggage of applications that were built when cloud was not a thing. There is a massive fascination in the field when it comes to containers. I am a big fan myself. But if you look at how the move to containers look like today – you don’t need to be a genius to realize it is not used as designed. Most of the time there is no systematic refactoring done. Docker and Kubernetes are not silver bullets. I have lost count of how many times dev teams have overlooked security, performance etc in the quest to containerize everything.  Forget all of that – look at the docker image sizes in modernization projects and you will probably see a lot of them represented in GB and not MB ( and I bet you there are posters pasted all around the building saying lightweight images are what we are going for). If that is the trend –  the end result won’t look much better than just outsourcing your data center.

Got it – but what if we build our own management layer ?

Some companies have the ability to make significant investments in people and tech. They can of course attempt to solve the multi cloud management issues by building their own management layer. While it might help the first wave of hand picked applications to achieve utopian status – it is not without some long term pain.

The most obvious one is talent. There are only so many top engineers in the ecosystem and you need to recruit and retain them for a long time. And while top engineers love to build great new stuff, they don’t always fancy the maintenance of those cool things. So you will need to invest additional $ to keep it running and enhanced with high quality .

Then there is the problem of each cloud being unique. Over time better standards will evolve – but for now, if you want to build a framework yourself – you need to assume there is a relatively small set of common functionality that you can make use of. The rest you need to build into your custom layer, or offload to applications to do so themselves. I will leave it your imagination and level of optimism to extrapolate what happens next in each case.

So how does one proceed with multi cloud?

1 Invest in refactoring the tech, and be realistic on time lines. 

On an average – most large companies have only ten or twenty percent of their workloads that are cloud ready if you ask the question today. And that only means ready – it does not mean optimized for cloud. The best thing you can do is to invest the engineering effort to get your applications cloud ready.

2 Refactor the organization for the (multi) cloud world 

People are the make or break part of any transformation. Ask the hard questions – like can you manage DevSecOps that include your current private cloud and also one or more public cloud ? Have your governance process been refactored and will it be audit ready ? Have you budgeted for frequent re-skilling and attrition management? Does your platform team and apps team know what each other does and are there clear rules of engagement ? etc

3 Be a minimalist and work with the ecosystem 

As of today, I don’t think there is a pragmatic way to move to a low chaos multi cloud landscape without close partnerships with the ecosystem – including some partners you may fondly(?) call as “legacy”. On the other hand – you cannot work with everyone and get some place any time soon. So the practical way to do this is to choose a couple of partners to go deep with and try your best to influence their roadmaps and tooling to suit your goals. If you can resist the temptation – I would urge you to not go down the path of building your own management layer with great sophistication.

Parting shot

Every large company CIO that I know personally is a realist when it comes to cloud. Some may not admit their true beliefs in public – which is not uncommon, and not just about cloud either.  So I am an optimist when it comes to where multi cloud is headed.

I had a chance to catch up with an old friend this weekend whose retirement party I unfortunately couldn’t attend . He was a senior partner in a law firm .

He told me a story which I want to share here . This happened some 35 years or so ago when he was just hired .

As he walked into the office, he saw that all the partners were in a conference room with a very serious look on their faces . After an hour or so they called the office manager – a middle aged woman who had spent all her career in that firm – inside . She came back with a big file in her hand – and she was crying . She left office almost immediately , still crying . His first thought was that she was fired !

He ran into her at the train station that weekend and she told him what had happened . He bought her and her kids some ice cream and she explained the reason why she was crying that day .

She had an abusive stay at home husband who was blaming her for all his problems, and she couldn’t afford a divorce . Apparently one of the partners came to know about this from his secretary and he called everyone else into this meeting . At the end of it – all of them wrote personal checks of $1000 each (which was a lot of money at that time) for her , and gave it to her saying “You gave your whole life to this firm and you are family for us . We have this covered and we will represent you in your divorce case etc . Go take your kids on a vacation for a couple of weeks and come back and we will figure out what’s next”.

Some years passed and my friend was nearing his own partnership at the firm . The trouble was that while he was good at his job – he just didn’t get the kind of visibility to the three senior partners . He went to this lady for some advice – by then she was supporting one of the senior partners as his secretary .

She pretty much coached him through the process – including apparently what to wear , what restaurants to eat, what jokes to crack and so on . She also helped him get some opportunities to be part of important work that got him the right attention . In two years he made partner .

She retired soon after – and at her farewell party, she told his fiancé (and now wife ) “I will never forget your boy friend buying ice cream for my daughters as we waited for the train – it’s the nicest thing anyone had done to them till then”.

A few days ago, my friend Frank Scavo wrote about the pendulum starting to swing from commercial software to custom built software .  Today morning, I listened to his podcast with another common friend Den Howlett .

I would highly encourage you to read Frank’s blog and listen to the podcast. The movement to DIY is real and I am seeing it in the field too. It is something I have wanted to write about and now is as good a time as any. I would like to focus on just a few aspects I have seen as critical to the success of custom building software at scale.

Do you have the right BRAND  to attract the top engineering and product talent ?

There are not a lot of talented software engineers to begin with. Attracting those few good people – be it a fresh CS grad or a veteran developer or product manager – to your company takes a lot more than throwing a lot of money at the problem. Your company may be the best retailer, or bank or automobile manufacturer on the planet.

Do these engineers you are trying to hire know of you as a place that works on cutting edge technology – and all that comes with it like great techies as your colleagues, modern work spaces, a thriving tech community and culture , cool perks and so on ? Do you have credibility in the market as an employer who invests in training people to keep up with the changing market ? Building that brand is real hard work and it takes significant time and investment . It is not a commitment you can make lightly.

Do you have the endurance to move from a project culture to a product culture ?

I have already posted my misgivings on IT shops using agile and scaled agile , so I will spare you those here. But in general – what sets apart professional software shops and rest of the world is the focus on products as opposed to projects. That is a culture shift that won’t happen without a lot of painful – and continuous – change management effort. Hiring has a direct influence on the culture .

The two largest pools of talent are other IT shops (commonly people jumping for better roles and $$) and SI companies ( commonly people who may even be willing to take a small pay cut if need be, but want to get out of their road warrior lives)  – both largely made of people who come from a project oriented background. May be ten years from now, this won’t be an issue – but today it is a definite issue.

One more thing while we are at it – If your team is global, are you willing to make the remote teams self sufficient and fully empowered and not just order takers from HQ  ?

Will your cutting edge team have the staying power?

If you are modernizing your IT function to enable a largely DIY culture – your economic model will probably need teams to be self sufficient on dev and ops, as opposed to one team having all the fun developing cool stuff and another doing the grunt work supporting it and answering the pager.

Will your software engineers hang around long enough if they are woken up at 3 AM by their buzzing pagers ? Do they have the skills to continuously automate ( the SRE types ) ? As you build follow-the-sun teams across the world – will your engineers have the skills and patience to work with people who are across time zones ? As you become more geographically distributed – will they be ok being further removed from the users for whom they are building software ?

Do you have a stress tested strategy for talent pipeline and IP protection?

Just as you want to build an awesome team, your competitors want to do exactly that as well . It is a lot cheaper to pay more $$ for your top tech talent than identifying them early and grooming them for a long time. Even if you are an amazing employer that checks every box, you have to assume a certain level of attrition.

Hence there are questions to answer about the supply chain for talent all the way from entry level to top levels in every geography that you operate. If there is anything more painful than attracting top engineers – it is in recruiting a top notch tech recruiting team and letting them do their job (often radically differently than what you have done traditionally).

Can you balance fun and discipline in engineering ?

Software development is not traditional engineering – developers are more artisans than engineers. On the good side, you will have plenty to choose from their creativity. On the not so good side – only a handful of engineering managers can balance it and eventually ship on schedule. Every good engineering team loves to create and evangelize their frameworks – and what they probably love even more is tearing down everyone else’s frameworks. I am as guilty of this as the next guy or gal.

The battle of frameworks goes like this typically – a new manager notices his teams are using 4 different frameworks . He brings them together and after a lot of debate they agree on a common framework . A few PIs later they find that they now have 5 different frameworks.

Needless to say – you need strong engineering managers, and ideally they are not repurposed from another function to fill a role.

So, can it be done ?

Despite all these uphill challenges – all this can be accomplished and to various degrees I have seen some companies do a decent job with modernizing their IT based on a DIY set of principles. I personally don’t know anyone who is still not struggling with it. The ones doing it better than others are the ones with extremely strong CIOs with the full backing of their CEOs (which is not common), and their boards (even less common).