Big Data : it’s about time we stopped putting the cart before the horse 


  
It rained heavily in Chandler yesterday and our front yard looked a whole lot nicer than the usual dusty appearance . That somehow made me think of the new look and feel of the world of data I live in as a professional .

For those of us who grew up implementing big data warehousing projects , it should not be a shock really to look back and figure out that most DW projects started because business had an analytics problem , but in the end 90% of effort was spent on the plumbing – the management of data (ETL, data modeling and so on) and only 10% on actual analytics (or even just basic reporting in many cases)  . 

This is true not just in design and build – it’s the case with supporting and maintaining the Data warehouse too . Companies have spent countless dollars on DW implementations and no one is truly happy about it . Yet, no one I know has any plans of fully replacing their DW implementations either (which of course is the right thing to do ).

Along came “big data” promising to make life better for everyone and setting very high expectations . Vast majority of customer executives that I speak to think of big data as an answer to their analytics solutions . Even amongst the CIO community , very few realize that most of the conversation they have heard is about the data management aspects ( 3V model is familiar to everyone and it’s about data management , not analytics). So in the past few years , I have seen several of my clients jump into big data initiatives to accelerate the realization of their analytics needs . 

The fall from grace is rather rapid – mostly because of unrealistic expectations . To begin with the minimum requirement for big data projects in many cases is to meet the SLAs of their existing data warehouses and data marts . It doesn’t take too long to realize that ain’t gonna happen . 

Then comes the dejavous realization that big data projects also need most of the time spent in ETL just like data warehouses did in past . Usually this leads to a quick reduction in scope of the projects – usually by eliminating some sources of data that are more complex or less clean , and of course this means analytics is compromised too .

Finally the reality of “data lakes need a lot of curation” kicks in . No company has enough man power to curate all the data that it needs for analysis . And at some point , the data lake just becomes a data dump with the idea that “curation can wait while we figure out what we need to analyze”. That is rarely practical – data scientists won’t always know the context of the data unless an expert curated it beforehand . And the world doesn’t have enough data scientists today to make them do data cleansing for most of their time .

Till such time as AI/Cognitive capabilities take the stress of curation away , I think analytics will continue to get short changed and the promise of big data ( and specifically data lakes) giving powerful analytics for busines users will not exactly work as advertised . 

It’s not all gloomy though . Customers who start small with well defined analytics requirements have already started realizing benefits from their big data investments . They don’t take a “build, and they will come” approach . They just build intelligently as requirements come up and plan to have more comprehensive solutions down the road . They value business flexibility and agility over technical elegance . Many of them have taken the time to formulate a strategy and a roadmap on what they want to do – leading with analytics that satisfy specific business requirements and working back to data management , and not the other way around . 

Of course we need both – but It’s time we put the horse (analytics) in front of the cart (data management). 

Tesla – thanks , but no thanks !


I have no doubts that the future of automobiles is electric . Having seen first hand the experience of several friends and colleagues who bought a model S and/or a model T , and reading about the 180000 people who put down $1000 as deposit for a model 3 – I am sure Tesla will be a top auto maker in future . But despite all my excitement – I have no reason to jump in and buy one in near future . I am going to sit and watch this play out from the side lines for a few years . 

  
Picture : http://icdn8.digitaltrends.com/image/tesla-model_x_side-3-970×546-c.jpg

 I am VERY thankful to everyone who has bought a Tesla already or will buy one of the next several ones to come out . You definitely are subsidizing my potential future tesla and you have my gratitude . 

It’s an engineering marvel for sure . When it comes to design – I love everything from the looks , to the lack of mechanical parts (I am a mechanical engineer by training and Automobile engineering was my most favorite subject in college ), to the “software rules” paradigm, to the autopilot , to the safety features (and promises)  to the amazing dashboard display …well I like it all !

Musk’s salesmanship is a sight to behold – especially on the big stage for new product intros . 180,000 people parting with their cash (granted, it’s refundable) – several of them even without seeing the car first – is a feat only apple can boast of today . That is 180 million dollars of interest free money . What is not to like ? 

So why am I hedging my bets and not buying one yet ?

1. I can probably afford one , but Tesla is still expensive in my mind – especially when I think of how much lower the prices can get once it gets mainstream in future 

2. I live in AZ – it will take some time to get the charging network established throughout the state 

3. By my rough math – Tesla sells as many cars a year historically as say an established company like GM does a week . Scale is very capital intensive in this business and I am not sure if Tesla will have a sustained advantage when all the established car companies jump in and start delivering at scale 

4. Tying the affordability of the lower end model 3 car to a government decided tax break makes me wonder if that is a good idea . Granted – Musk will have the best lobbyists working for him , but I still don’t feel comfortable on this line of thought 

5. Tesla will need to sell more X and S models to gain their big margins to make model 3 work . That is largely the top end market that Porche and Merc and others have a stranglehold of today . If these German/American/Japanese luxury brands come out with truly great electric cars – I don’t see Tesla managing to sell a lot . 

6. Apple /Google and their car projects – that is a wild card too. No idea if it’s going to be a real car or if Apple/Google  does software and car companies just OEM it . Tesla is not the only company to have top talent on software 

7. Given the lack of driver loyalty , I am no longer sure if Uber and other companies like that will thrive long term . I really wish they do – and if they do , I can’t wait to get rid of car ownership for good . 

8. If Tesla does come up with a safe driverless car – then maybe it’s the uber like companies that will buy all of them . Again – I would rather not own a car at all in that case 

9. The car is green and I like that . But I am not convinced yet that the manufacturing process for these cars and their monster batteries are energy efficient . And I don’t really know how long it will take to make the process fully green . 

There are more reasons – but my flight is about to land and I need to power down . So no Tesla for me for now . But I hope I find reasons for my heart to over rule my head , or to find my head is just wrong and I should just buy one 😉

Services world is fun again


img_7085When I left the services world few years ago, the truth is – it was rather boring. Whether it was ERP or analytics or mobility, there were very few projects that were fun to sell and deliver. Yes – I did say fun , and with a straight face !. Don’t get me wrong – between then and now, it is no less difficult to sell and deliver consulting projects. The big difference is that the problems we get to tackle, the buying behavior of clients, and the technology options available to solve those problems all make it a lot of fun to work in.

A lot of credit goes to customers – they are pushing the vendors harder today and rightfully so. Many more leaders want to differentiate their companies to an extreme in a short time, and not in small increments. More than technology capabilities per se (that is a given – you don’t get a seat at the table without the tech chops) – they are looking for partners who can help them think through their roadmap and identify new opportunities. They are happy to take risks – as long as vendors are open to risks and rewards being shared . I am even seeing signs of procurement function getting modernized too in their approach to software and services purchases.

Sophisticated business solutions are complex by nature – except the customer has little to no reason to deal with the complex technology directly. Last week, I met with a large sports CIO and his IT staff is essentially 3 people. He had very little interest in what happens behind the scenes – he just needs some very sophisticated requirements to be satisfied. Conceptually what he wants is “cloud ” – except he never said the word, and even when I asked him point blank, his response was “I don’t really care Vijay – you can deliver any way you like as long as my parameters are met, for the money I have available to spend”. The CMO who was sitting next to him added “And if you show me how my firm makes more $$ by using your solution – I am happy to keep paying you more”. Not to be out done – the CFO got into his ask “And if these guys are happy with your solution – I want to sit with you and see your models on different capex and opex combinations on how we pay you”.

This conversation repeated more or less exactly the next day in Manhattan – where I met with another CTO/CMO/CFO combo team. The one difference here was that the CTO staff wanted full visibility into how underlying technology worked. Just visibility – like a car’s dashboard where you get to see if there is a check engine light, but need to schedule a mechanic to fix it. We also had a passionate debate on NoSQL vs Hadoop – and then they said “we know we don’t need to worry about that in a cloud model, but want to make sure you have thought through our workloads correctly”. Thankfully we had .

And to round off the week was a senior exec at a Telco – and all he wanted was to check is if his roadmap for digital transformation for his business can be executed in half the time, without changing any of the underlying technology. Next week we will do a series of design thinking sessions to figure it out.

There is never a dull moment. No two deals or projects look cookie cutter like the ERP projects I grew up with. Boredom is the farthest from my mind when I spend a morning explaining to a VP of BI why her expectations of a data lake needs to be different from the data warehouse she is used to, and then spend the afternoon with a COO showing how Watson can reduce subscriber churn. Tiring – yes. Boring – hell no !

Whether it is story telling, crafting the right financial model, figuring out the right tech solution or even assembling the right team – fun is back in the business. All the things we needed in the past – like deep relationships, trust etc continue to be as important as ever, just that the entire approach is different now on both sales and delivery. We spend a lot more time discussing the effectiveness of solutions – efficiency is a given and we don’t need to spend time and energy reinventing the wheel.

Sales always had elements of fun where new and interesting ways to pitch to a client was always an in-thing. What has really started to change now is actual delivery. Everyone is now pitching in – PMs, devs, ops folks, analysts, interns – to creating a storyline that makes sense of the bigger picture and how their piece parts fit in. It is such a great feeling to see how much less work it is technically when we make micro-corrections daily. The lines between discrete products and services that put them together is still very much there – but it is starting to blur to a large extent. Tactically – a new generation of product managers (probably better called solutions managers) are arising , and they bring together the world of products and services.

The one thing that is NOT FUN that can and should change is the need to travel to deliver solutions. Even with cutting edge video conferencing and other collaboration tools, customers still expect all consultants to show their faces weekly onsite. There are parts of our business where this is less of a problem – like our designers working out of studios and only visiting customer offices periodically. I am hopeful that services business will largely move into a model where there is less travel involved. The sheer amount of time and money spent in air travel is an absolute waste and that is the next thing that needs to transform in large scale.