Big Data : Switching Between Top Dog And Fire Hydrant


Most of you know I am a huge fan of big data. In my mind – this is the new top dog in enterprise software world. Companies can leap into a totally new world of insights and in all probability will revolutionize how business world makes use of information. I have not met a customer this year who does not have plans to go for some big data initiative. And no wonder – the vendor side of the world is all excited too. It is heartening for me to see both customers and vendors excited at the same time – that should curtail the hype cycle for big data.

Big data is capable of changing everything for the better – notions of what a platform does, what an application does, and rapidly advance data science in the academic world.  The technology and the theory of big data is fast changing – both on open source side, and on commercial vendor side.. There are a variety of options available for customers now for all aspects of big data. And while choice is good – it also tends to increase confusion. It is important to make sure that the bets on big data are made with eyes open, and not based on hype.

Big data has real benefits – and while the technology is evolving , there is already plenty available to make good use of for many use cases – ergo, you can use it today. CFOs will always have budget to spend for making more money – or to reduce cost. But they won’t be happy to write a check with neither possibility. So lets try to keep big data in its top dog persona, and try to avoid the evil twin , the fire hydrant.

There are 2 general ways to use big data

1. Find answers to questions you already have by sifting through a lot of data

2. Keep looking at a lot of data, and see if you can spot something – without having a very specific question to ask upfront. Once you spot something, you start asking questions like in option 1.

Option 1 is mostly just an extension of traditional BI – but with more data, coming in at a higher speed and probably of more types than we have dealt with before. 

As I think through all those past BI projects (and the associated blood, sweat and tears) – I can say with some certainty that most of the data pumped into data warehouses were never used. Vast majority of the customers got most of their answers from less than 30% or so of the data available to them. Of course it can be argued that they probably did not even know that the other 70% existed. But these are big companies with excellent top line and bottom line and they believe they have a mature BI platform. So lets say 50% of the data is useful instead of 30% I estimated. Yet, if you ask for the requirements for the next project at these customers – I will bet you dollars to doughnuts , they will ask for everything just like they did in the past. The fact that they don’t use half their data is of no consequence for them. That is how the BI world rolls.  Now with big data – there is a chance that more of the previously unused data will be used to enhance the quality of insights

Any kind of big data solution that is built for a customer who follows the option 1 route needs very cheap storage, or a way to store useful information alone, or have a BI solution that can sift through a lot of unnecessary data quickly to find answers.  This also means BI practitioners need even more due diligence in figuring out requirements so that wastage is kept to a minimum. Easier said than done .

In our familiar world of data warehouses – there is some data duplication across layers for various reasons (performance, transformations and so on). When you think of big data – don’t assume redundant data just goes away magically. On the contrary, many big data solutions (including Hadoop) need redundant copies of data . Storage is getting cheaper – but you will need a LOT of it if you keep agreeing to user requirements like before. There are also trade offs between using SAN vs several machines networked together with their own (cheap) disks.

What about performance? You can of course get excellent performance – but that depends on the question. If a system needs to use massively parallel processing – the question you ask should be able to split the data it looks at into many chunks and look at all the chunks in parallel, and then add up the results.

If you ask ” how many icecreams of each flavor did we sell today?” – and you have a million transactions, you can easily do that chunking and aggregation. However, not all questions can be answered that way. What if I ask “What is the correlation of the brand of icecream sold and rise and fall of local day time temperature ?”. This question is hard to split into many chunks because there are more variables . So while it can be computed, it is a fairly serial process. Now of course you can try some other way of solving this by looking at a smaller set of data (whether you cook one pot of rice or one barrel, you check if it is done in both cases using just a spoonful ), Or if you knew upfront that this question will be asked, you can do the old data warehousing technique of pre calculating data in useful forms and wait for the question to be asked and so on.

Essentially – depending on the question you ask, you might need a combination of big data solutions (say like Hana and Hadoop) to get a good answer in an acceptable time.  You can reuse and build on a lot of skills you already have in the shop today. But walk into option 1 knowing all the trade offs you have to live with. And I haven’t even skimmed the surface of all the things you need to consider.

What about option 2 ? No predefined questions – but you look at data and see if there is anything useful there.  The good news is that the technology is already there to do this. Bad news is that you need a lot of hardware, consulting etc to get it done. Well there is one more thing to keep an eye on – in the wrong hands, it is fairly easy to bark up the wrong tree when you are dealing with this kind of big data. False positives and false negatives galore. You might be chasing solutions to non-existent problems. An interesting side question while we are at it – have you ever run into a data scientist who said he/she has enough data ? I have not – they all would like even more data.  I am told they exist somewhere 🙂

What about disaster recovery ? Better start getting used to recovering a few petabytes of data.  High availability is probably not a big issue since it is kind of part and parcel of the big data design in most cases. And of course option 2 has to deal with all the issues with option 1 – just that you might not know upfront of extent of the stuff you have to deal with.

Big data will be a fun ride – but keep your seat belts fastened low and tight across your lap for any little bumps along the way.

 

Advertisements

What do I want to be when I grow up ?


When I was three – I was pretty clear what I wanted to become . I wanted to be a Station master for Indian Railways , wearing a white uniform and controlling the huge big trains with green and red flags . My grandparents bought me a pair of flags and I used to wave them around in cars and trains .

Couple of years later, I wanted to become a trainer of elephants .
20130727-195627.jpg

Two things sparked it – first the movie Guruvayoor Keshavan , and second the fact that my grand father had couple of elephants at home , one of which practically was my mom’s pet before she got married. And then I saw my first circus and this career ambition of becoming elephant trainer got expanded to the next “rational” thing – I wanted to be a circus ringmaster! Someone who can train not just elephants, but also horses, lions , dogs etc do cool stuff . I never cared for the human artists who did dare devil acts – I thought that was boring 🙂

This fascination for training elephants continued for a while till I saw how elephants are trained by inhumane techniques . And to add to that, I also read about how elephants are hunted in India and Africa . That was it – I becane dead against the idea of domesticating elephants and other wild animals . They are wild animals who should be allowed to live free .

20130727-200611.jpg

After a brief consideration of being a policeman , I decided that finally what I “really” wanted to be was a doctor. That lasted till tenth grade when my biology teacher pretty clearly let me know that I had no future in that field . The only part of biology I could relate to at that time was genetics – pretty much every other part made no sense , and I gave up the idea of being a doctor .

Meanwhile , I had developed a BIG fascination for dog shows – both obedience and breed shows . And as luck would have it , I also had two amazing dogs in quick succession . A female yellow Labrador and a male german shepherd . With the former, I did some decent winning in breed ring, and with the latter I pretty much became a top contender in obedience . The dog show bug bit me so hard that my parents were seriously worried that I will drop outbid school and become a professional handler or trainer .

20130727-202155.jpg

Well , that didn’t happen – and I have an uncle to thank . He was a dog show judge who convinced me that if I become an engineer and earn a good pay – I can buy any dog I wanted . And so I became an engineer , and with my first salary I bought a German Shepherd from Germany .
20130727-202224.jpg

I found in three months flat that I hated the shop floor life . I ran away from that job and did my MBA . And halfway through I was sure that I didn’t like becoming an investment banker either . I wanted to be a programmer . And so I joined TCS, India’s largest tech shop . And few months later , I landed in USA through them .

The first half of my career, I shifted employers frequently .Either I got bored with repetition, or I couldn’t stand by manager . In the second half – I was happy staying with one employer for long. And from being a programmer – I went on to do other things like functional consulting , Business intelligence , CRM , Project management and sales . And finally when I was going through the partnership appointment process at IBM, I left and joined SAP to start afresh as an engineer again.

It is not that I didn’t enjoy sales – I was always making sales way above my quota when I had a sales role in past. I just never could decide if I am a seller or a builder . Hopefully it will become clearer when I grow up . There are only two things that have been consistent in my career – an undying love for enterprise technology , and treating people the best I can . The love for technology is natural – that has remained rather flat without ups and downs . The part of treating people well is something that I learned along the way, and continue to do. Wll there is another “constant” that I enjoy – work takes me to several different countries, and it is nothing short of fascinating to make friends with people of other cultures.
20130727-212259.jpg20130727-212242.jpg20130727-212212.jpg20130727-212312.jpg

Tonight I found my little daughter playing with our new puppy, Ollie .

20130727-205047.jpg

I snapped a photo and put it on Facebook and a friend commented that maybe my daughter will take Ollie to junior handling at dog shows . Nothing would please me more if she does .

That is what triggered this post – it reminded me of my own childhood, growing up with my dogs . And how my ideas of what to aim for in life changed every so often .

Who knows what is ahead when I grow up ? May be everything I learned along the way would finally make me qualified to be a circus ringmaster after all – or may be a full time dog trainer ? I certainly don’t want to be a station master at Railways anymore – I hear they don’t wave red and green flags anymore .

And I certainly hope that my daughter grows up chasing (and changing) her own dreams and balance it with realities of life . I can only hope that she will have a dog by her side throughout her adventures .

20130727-210547.jpg

Watching Leadership Transitions at IBM and SAP


I “grew up” in IBM – and one thing IBM does really well is how to transition responsibilities from one leader to another . Although not at very close quarters , I watched how smoothly Sam Palmisano transitioned over to Ginnie Rommety . And what I still remember the most from that time is Sam saying Ginnie got the job because she earned it fair and square , not because she was a woman . I had watched Ginnie with a lot of admiration as she moved from running global services to global sales and then on to CEO .

No one that I know – employees , customers or partners of IBM – ever mentioned to me that they were worried with the transition . There was no chaos or confusion .

Ginnie was well groomed for the job – and inherited an awesome team. She made several leadership changes too at all levels of IBM. Proof of the pudding is in the eating – and here Ginnie has had a mixed year with some bad quarters , unlike her predecessor who went from strength to strength (from a financial point of view that is ). Yet, I never heard someone asking – “what would Sam have done ?” .

As Sanjay Poonen recently wrote in his SCN blog – there is no success without successors . I have a faint memory of a teacher mentioning this in my high school class in the context of sports -so I am guessing this is a biblical phrase . I can’t agree more – the hallmark of a great leader is how little he will be missed when he walks away from his current role . Any one who knows the guy taking over from Sanjay – Steve Lucas – would get what I am saying here .

My managers in IBM groomed me well too ( and I am eternally grateful) for my next steps up, and I was always encouraged to watch out for my team . It was extremely gratifying to see my pal Gagan Reen take over the innovation team after I left and never missing a beat . And I know he is grooming others too . Words cannot express how cool it is to watch the cycle propel itself by paying forward .

This is not to say all transitions are smooth . I worked in a small company before that got acquired . It was anything but smooth . I went from a position where my boss and I could talk on any topic at any point in time , to me needing an appointment with his EA to get an audience . Long story short neither me nor my boss stayed there for long . I later figured almost all of the acquired team dispersed because the transition wasn’t well managed . This was before I joined IBM . I also heard the big company that bought us did not get much repeat business from old customers we had .

And then I joined SAP about seven months ago. When I was considering the offer , I happened to run into Sanjay Poonen . Sanjay invited me to sit in his leadership class for a day in Palo Alto, at the co-innovation lab area. In my entire career – there are only two classes that I found useful for me . Sanjay’s was one . The other was done by Bill Smilie at IBM ( a program called Cornerstone). It was clear to me that Sanjay and Bill were both very good with mentoring the next generation of leaders by practical examples . Neither one preaches – it is learning by open discussion . If anyone who reads this has an opportunity to learn from these two guys – don’t think two times about it , Just DO IT !

A big reason I decided to join SAP was the faith I have in the leadership team . I had known Bill, Jim, Vishal, Sanjay,Rob Enslin, Steve Lucas and many other leaders from before – either via IBM or via the blogger program at SAP. So I had no doubts that I will be joining a company that had tremendous executive bench . Of course, I had no clue that people like Sanjay and Jim will be leaving the company so soon to begin their next adventures .

But having seen how smooth transitions can happen – I figured quickly that SAP won’t miss a beat either . Sanjay and Jim are leaving SAP in great hands .

And there is plenty of continuity – development is under the best possible guy – Vishal . And Hasso is deeply involved in technology direction. These are no ordinary people – and combine vision and execution very well . Bill has been Co-CEO for a while now – and is more than capable of running the show solo . And the strategy that SAP executes to now was formed by the leadership team that included Bill . That should ensure stability of plans .Look at the transition plan – Jim will continue for almost a year. The CFO transition also has a long time line . In short – this is as well planned as it could be .

So as a relatively new employee – I am as comfortable about this as I was when Ginnie took over from Sam when I worked at IBM . In fact, I am even a little more comfortable – given I know these SAP leaders a lot better than I knew the IBM leaders . And it is not just the very top levels – SAP has solid leadership talent right below these folks as well.

And just as no customer expressed any concern about IBM leadership changes – I am confident no one will ask me about SAP leadership changes either .

Last but not least – Sanjay and Jim, I wish you the very best .

Future of SAP, IBM and Accenture – I am bullish


My pal Dennis Howlett wrote this piece on diginomica http://diginomica.com/2013/07/20/weekend-speculation-sap-ibm-accenture-2015/ where he speculates about the future of these three companies . Having worked many years at IBM , having competed with ACN all my life as a consultant and now working at SAP – my view is (surprisingly eh?) slightly different to Den’s . I am not saying Den is wrong , just that I have some different opinions. And in some cases, I agree with Den. As always – this strictly my personal opinion, and not my employer’s opinion .

Let me start with IBM . It is a company I admire a lot as most of you know. When other companies think of long term as 5 to 10 years , IBM invests billions every year in research that may not pay for 20 to 50 years . And that doesn’t change even when a given quarter is not doing well .

IBM is 100% ruthlessly capitalist – maximizing share holder value is key and there is little tolerance for not keeping a promise made to shareholders . They are also a highly ethical company – not even a remote case of unethical behavior gets tolerated there . If you think of IBM as an American company , there is some cause of concern on shrinking local workforce etc . However , if you think of IBM as a global company – it is not all that bad .

IBM misses its bets like every one else – but wins more than competitors on long term bets . They systematically got out of HW business before any one else realized what lay ahead. I have heard stories that they also passed on buying intel and Cisco when they were small companies . So yes – good and bad decisions . But there is no denying their ability to see farther than most . I do expect IBM to get rid of more HW business in near term . Not many people know that IBM does R&D for lots of semiconductor companies even if they are not a big semi player themselves . Smart bets I say .

What about services ? Lou G saw services as the next frontier and invested in it . Sam P and Ginnie R grew the services business into a powerhouse . I know there are public references of bad implementations etc – but having worked there for long , I can assure you there are many more successful projects than failed ones .

Services business is heavily competitive – and margins get thinner all the time . IBM has a partner model (like ACN ) which they inherited from PWC acquisition – although partners are just regular VPs , not real partners . This means there is a partner (or several ) at each customer holding and developing that relationships and selling and delivering work there . And then there is an over lay of service lines for each area like SAP, custom development etc . And then there are checkers and double checkers to make sure finance and legal and HR and so on are taken care of. So the over head is significant for maintaining that big organization . Even with periodic restructuring , it is a big cost . And since IBM is all about EPS growth financially – this will be a constant headache .

So I do expect IBM to make a lot of structural changes in services business to make it leaner , especially at higher levels . And since they already have shown that they are not averse to selling off businesses that have limited upside in future – on a stretch, I can imagine IBM even selling off a part of that business to an Indian consulting company who has cash, but does not have the ability to accelerate up the value chain organically . Of course that is an extreme stretch – but not impossible in my opinion . But hey, everything is ok when we speculate, right ?

Even while I was working there , IBM was getting into SaaS consulting . I do expect that to pickup – but in over all scheme, without structural changes it won’t align with their EPS targets .

Will they get out of SAP business ? Just like all good sales leaders , IBM leaders also play the opportunistic game well . Believe it or not – even with steady decline , SAP Business suite based work is still highly lucrative – and IBM has the ability to go after a global customer base . There is a bull market somewhere at any point. IBM has a definite opportunity they will capture to promote blue software via consulting . But it is a tricky game – the doomsday of consulting company is when customers suspect bias . When I was an account partner at IBM – I would do what is best for my client , even though I was paid a salary by the SAP practice . So if the SAP market is lucrative , there is no reason for IBM to get out of it any time soon. The only exception of course is if they sell off the non blue consulting practice as a whole . So generally low odds in my opinion.

IBM software is the one to watch . Steve Mills is a one of a kind leader – I have the utmost respect for him and his leadership team . Apart from waking up late to modern in memory computing, that team made very few wrong steps. They try to control the lower levels of the stack effectively – DB, middleware , security, App servers, ETL, Portal etc . Apps is not something they have a strength in . And they are an acquisition machine – probably the best at M&A in the industry . IBM has a big war chest in liquid assets – they can acquire a lot more companies , and I am sure they will do that .

Whichever way I look at it , IBM as a whole ain’t gonna fall flat on its face . It will suffer some transition pains to restructure , and I do expect them to suffer some pain to meet 2015 roadmap for EPS . Maybe one year after 2015 is going to be painful , but that is about all I think is worst case. What also makes me say this is the depth of executive talent on bench .

About Accenture – I know a lot about them too thanks to them being a major player in SAP consulting . At IBM, many of my managers and peers benchmarked only against ACN . They were the ones to beat . They are very similar to IBM services . excellent customer focus , ethics and skills .

They don’t have software and hardware businesses like IBM. They are also a bit top heavy due to the matrix partner model . But they sell services better than almost anyone else – and have good investment in people . Their relationships at C level will see them through any hardship in long term . They invest in cloud consulting and together with some structural changes , they should be ok . Just like with IBM services , it is possible that another company might buy a part (or all) of Accenture . Not being diversified outside services is a risk for them . And they have no reason in my opinion to ever get out of SAP business .

Finally SAP . Den says SAP can reshape SI business . I doubt that seriously. SI’s – other than really small ones – do not depend on SAP for most of their business . And SAP’s culture is collaborative , not dictatorial . Every time SAP comes up with something new , a lot of time , effort and money is spent making sure ecosystem partners are on board quickly . SAP is a software company first – so services are strictly done on a need basis .

Economics and elasticity of software and services are very different – and SAP has been generally better off letting ecosystem deal with most of its services business .

However , there are couple of important points in the post ERP world . Speed required to get value is in weeks now – not in months and years . And SAP technology projects like Hana do not cater well to packaged implementation approaches that worked for ERP.

This is not a waterfall vs Agile problem although that is how popular opinion is formed . Waterfall won’t go away till customers change buying behavior and Agile becomes more predictable in services business. There is also an analyst push on outcome based projects – pay for result and not effort . This is easier said than done – I have first hand seen how many customers show resistance to gain share contracts . And “measure of outcomes” is usually hard to agree up on at contracting time.

The method of sales – especially direct sales – for services is a big eater of margins . For large ERP or maintenance projects , direct sales is unavoidable . But for a Hana project or a SFSF implementation , I doubt the big SIs can make it work inmost cases via direct sales . I expect them to start using a web store for some types of services one day soon . Maybe they use SAP Hana market place , maybe their own . But I do expect sales model of newer solutions to change in very near future – like within 2 years or so. It’s not just web stores – talent sourcing , billing etc all will change for a part of the services business.

Just like cloud vendors make every effort to discredit larger on premises vendors , some smaller SIs and independent consultants also tend to discredit larger SIs . All sizes of SIs have a segment of market they can serve better than others . I am all for a fair mix of independents and big SIs – but it is rather hypocritical to point fingers at each other . For deep tech expertise – a freelancer might be a good bet . But that alone doesn’t make a project successful . So – I expect no big disruption from small SIs and freelancers for majority of Services business . Pie is big enough to share for everyone .

However I do not expect even a third of the global services businesses to move to cloud by 2015 . Real customers are orders of magnitude more conservative than what is reported on social media . Enterprise software consulting business is a marathon or maybe a steeple chase occasionally – I don’t expect it to change to a sprint by 2015.

As world moves to SaaS more and more – the demand for integration will open up a very lucrative SI model . And to add to that will be the big data movement . So this disruption is just a shift of business from one service line to another for SIs . As my ex manager used to tell me – it is all green dollars whether it is ERP or Hana or SFSF or integration .

Ok that was a LOT of typing on a small phone screen . All I wanted to say was that I don’t see any of the SI companies – certainly not IBM and ACN – take a big hit long term , except a few bumps along the way . And for SAP – I have full faith in Vishal , Bill and Jim to match execution with the strategy they have laid out .

Phew …my thumbs hurt with typing – now I need a beer 🙂

In The Enterprise Software Race, No Horse Ever Loses For Being Slow


Another random rambling – this time caused in part by a twitter conversation today morning, and a few analyst reports I got around to reading at lunch break. And finally I saw this past homework sheet that my daughter filled last year , where she says she wants a job like the one her daddy has.

Image

I work in this great industry, and yet I continue to be amazed (and amused) by the logic of big and small companies that make our industry. An entire set of companies operate as if the forces of market just don’t EVER apply to them.

For software, supply is a non-issue in most cases. You build it once, and you sell it many times. And one way or another, you get customers to pay for maintenance and enhancements – either explicitly in case of on-premises companies, or built into subscription costs for the cloud companies. This model worked great for many years because only a few companies were competing in each segment – be it ERP, Databases, CRM or whatever. The threshold of entry was always high enough so that supply side remained a vendor advantage – Supply won over demand , believe it or not.

Software was not the only industry where this happened – even in the world of tangible goods, we have examples to learn from. Look at semiconductor companies. Till very recently – they defined demand of their customers based on their available capacity. Customers were told how much they will be allocated – and customers could not do much about it given the capital needed to stand up a new fab was beyond the financial strength of  most companies. Well, these companies eventually had to turn (painfully) to pure demand driven companies – and the same is bound to happen to software companies too at some point. In a few cases, it is already happening.

Along came cloud, and it made it easy for anyone to be a SW player. When I passed out of college – it was hard to start a SW company given there was no cheap way to access internet or buy hardware. Many good ideas never saw the light of day and many bright students shelved their ideas and took employment at big companies to execute on someone else’s vision. That is so not the case now – it costs very little to start a new software company today in comparison – at least from a “production” point of view.

Challenge today is that of demand (as in top line revenue) – not supply. Since threshold of entry is low, everyone and his brother is now in the SW game. And not surprisingly, the quality of ideas is poor in many cases, and there are no “easy” customers to find . So what is one to do – you hire the best sales person(s) you can afford and ignore the cost of selling to “buy” market share.

Profits be damned – it is all about growth. The trouble with this model is that you probably can never take your foot off the high sales/marketing expense pedal since competitors are also trying the very same strategy as you are.It is a mega race downhill.

Its not just high sales and marketing expenses that come with accelerated growth. To “buy” market share – acquisitions cannot be avoided either. The problem is that the owners hate to dilute their stock – so in most cases, debt is the only viable option. So between debt and the high expenses – the question is how long can such a company do its business . Debt is cheap for now – so probably they can all keep afloat for a while, but sooner or later, they will have to pay up. And it remains to be seen how many will be still viable and solvent at that time.

Then there are the companies who care only about profit ( EPS for the public companies ). They have the same challenges to generate demand – but instead of taking the harder route of increasing topline to create more profit, they play with other levers. They squeeze costs out of their system ( move to cloud, use labor arbitrage etc), they buy back shares aggressively and so on. Again, at some point – you run the risk of cutting into the bone.

Then there are the companies who try to take a more balanced view on top line and bottom line together. But they are in a fight of their life too – since companies who are aggressive on top line and companies who are aggressive on bottom line both have “trained” their customers on what to expect. So these “middle of the road” companies usually will have to resort to tactics unnatural to their DNA to fight it out with the others.

Customers “might” win in the short term when there is intense competition on vendor side, and that is a good thing. But in the long term, the very existence of some of the vendors might be questionable. So over the long term, do even customers win? or do customers also join this downhill race , maybe without even knowing it?

In short it is a mess – or I guess the more PC way of saying it is “It is an interesting scenario”. The extent of this mess is clear in the communication style of the leaders who run some of these companies. Every one touts they have the right strategy – yet very few seem to execute to it over time.  How fast can they drive forward while constantly watching the rear view mirror?

When I worked in the UK more than a decade ago, my friends got me hooked to watching horse races. Did you know that a horse never loses a race because it is slow ? That is how I understood it reading the coverage of races in newspapers and magazines. A horse  might lose because of bad training, bad surface, unfair handicap and a hundred other reasons. But NEVER will a horse lose because it was slow. If you watch our beloved industry from outside, I think you will feel kind of the same way about the companies that make this industry.

I do believe that the forces of market will eventually trump everything else – and I wonder how many vendors will stand when that happens and how many will bite the dust. Maybe it is time for this industry to get off the high horse called “strategy” and focus a little more on good execution.

And I hope this happens before my kiddo starts looking for a job – and yes it would be quite an honor if IBM (or SAP) employs her.

 

 

What Does BI Mean To You?


Totally an unstructured rant – be very warned ! 🙂

Over the last few weeks , a lot of discussions happened on BI between me and some friends . Essentially – why is it that BI projects seem to stay the same way it was always done even while tools became a lot better ?

To begin with – tools have become better , but not to the level where a user can use it in self service fashion in most cases . So, education is probably the number one issue in the context of BI. Education comes in many flavors – and the easiest one in my mind for a user is the actual tool training . So lets move past that.

There are a few things that have not changed from a BI project POV in several years

1. Users do not understand the data available to them

This needs a lot of bottom up and top down effort to fix . I think top down governance will hit a wall sooner than most of us estimate . Data needs to be explained for each user with real life examples, but that approach is hardly scalable.

BI project plans need to budget time and resources to fix this – but having seen no change in many years , I guess the inertia is just too much

2. Users do not understand what reports are telling them , and most of them have limitations in explaining requirements to IT or consultants

I have done elementary statistics type classes to explain data to users in some past projects . I have a feeling this needs a deeper look to see if such an approach needs to be standard in BI project life cycle.

Users have unrealistic expectations of BI in many cases. This movie is now playing again with predictive Analytics . I see a lot of users and consultants having limited idea of what a predictive model is telling them and what it’s limitations are . No wonder several of them crash and burn .

3. IT and consultants takes a one size fits all approach for BI , and spends little effort in acknowledging BI is personal , not generic

This won’t change till BI is no longer treated as a side kick for ERP . People personalize ERP quite a bit ( at least compared to BI) , and that rigor needs to get into more BI projects .

There are a dozen more things , but I feel better already after typing 3 to get it off my chest

Sorry – had to rant . Will try to not do this too often .

BW on Hana , How About In The Cloud ?


I have been talking to customers on BW on Hana for a good while now – about their plans to make use of BW on Hana, and what roadblocks exist for its adoption etc. Here are the top three concerns that I hear the most

1. Projects to migrate to BW take a long time to complete, delaying value realized by business
2. Due to the length of the project and the cost to acquire HW, SW and Consulting services – the initial investment to be made looks high for some customers
3. Is it possible to try a few scenarios in BW on Hana quickly before going all in with a full migration?

Last year, several customers asked me why they should consider BW on Hana. These days, I hardly hear that question – I get more questions on “HOW” they should go about it . And to help customers and partners with this , my team and I have been traveling across the world doing hands on sessions on BW on Hana as well as BI on BWoH. Several other teams at SAP have also been doing similar enablement sessions, so I think the “HOW” part will be adequately taken care of too.

But we still need to better address the 2 questions on length of migration projects as well as the initial cost. And this why I wanted to ask your opinion of putting BW on Hana on cloud, on a subscription basis. There are many potential deployment options like I scribbled in these pictures on what systems can move to cloud.
BW on hana alone in cloud

BW and BI in cloud

everything in cloud

I I really have no idea whether SAP can actually do any of these ( it needs to make sense on technical and business dimensions). So please do not treat this as an official SAP statement. My intention is strictly to present my thought process to the ecosystem and get some feedback from all of you. And of course , this is just an “additional” deployment option, not a replacement of on premises BW on Hana which I fully expect to continue.

Back to the 2 common concerns – how do we solve that? I think a cloud option is probably one worth considering.  Here is what I think might work

1. SAP or a Partner acts as the cloud provider and signs a subscription contract with customer via SAP Hana Marketplace

2. Customer gives a copy of current BW system to the provider

3. Provider imports and migrates the copy into their cloud, and does testing. Maybe customer can do some user testing too.

4. With necessary signoffs, the solution is cut over into a productive instance on a cloud that customer can access

5. Any extra data/ transports etc that are needed are applied, deltas initialized, BI tools connected etc

6. Customer starts using the productive instance of BW on Hana with some agreed up on SLA

7. Customer starts paying the monthly subscription, with some provisions for data growth, additional BI clients and so on in future

I probably made it sound simpler than it is – but hopefully the general idea is clearly coming across to you.

I think there are many advantages in the cloud model

1. Customer deals with one provider alone – not one each for HW, SW, Consulting and Support.

2. Since one provider has full control over end to end experience on multiple customers – economies of scale should kick in and result in less time to do migration, and less cost than customer doing all the steps individually via multiple partners.

3. With subscription pricing, there is no big initial outlay for customers . Monthly payments are predictable for budgeting reasons etc .

Of course there are some trade offs too that need to be considered

1. There might be some reluctance to move a data warehouse to cloud for fear of security. However, this reluctance is definitely coming down – and VPN and other security measures can be put in place .

2. Network can become a constraint for loading and reporting. However, given the speed of BW on Hana on both reporting and loading are faster than classic BW, some of this can definitely be compensated. Also, not all loads are real time into BW . There is of course the option of moving the source systems also into the same cloud and reap even more benefits.

Now the million dollar question(s) – what do you folks think about this? Do you think this will help more customers derive value from BW on Hana ? What other factors should be considered from the customer’s perspective ?

And about the third concern of how to get a jump start on BW on Hana without going all in – Do you think customers will like to get a Hana One Premium type system for BWoH that they can run in parallel with existing BW systems for just a few scenarios ?

Lets discuss here in the comments section, and pls fill out the polls below