The future of big data – consolidations , partnerships and mostly open source


In theory, I am on vacation this week – except that I have not managed to get out of email yet. And to top that, I caught up on a lot of reading and as usual, have some thoughts to share. As always – these are just my personal opinions, not that of my employer.

The good(?) and bad thing about big data is that it has no rigid definition – it is more or less what we want it to be at that point in time. I like to think of “big” in terms status quo of a customer. If the customer only reports out of their ERP data, and now wants to combine CRM and SCM data with it , or maybe combine social data with it – it is already BIG for them. For a stock exchange – they have already figured out how to manage lots of fast and furious data. It is hard to tell them data is BIG now – data was always big for them. It might be a bit “bigger” now, but you get the point. “big” is relative to status quo.

At the moment – there are very few big data projects (as a % of all projects ) that add outrageous business value. Its like a white tiger – a majestic animal with great abilities and all, but rarely seen in the wild. But they are not a myth and they really exist. I saw one just this week and took a photo. I know many friends who tend to look at big data as a white elephant – and I now like to think of the current situation of  big data as a white tiger 🙂

tiger

So where is it all trending to? I have some opinions/guesses – not backed by any scientific research. So take it with a pinch (or pound) of salt.

1. Vendor Consolidation

I definitely think there is some consolidation about to happen in 2 to 5 years time. This could take many forms – established firms like Oracle, IBM, MS, Intel, SAP etc could buy some Hadoop/NoSQL type companies. Some of the newer players could merge. The big ones have enough cash in their war chest and those that don’t have cash might not mind a bit of stock dilution to be successful in the brave new world . Big data is what is going to sell more analytics, more hardware and more databases in future – so the incumbent big players in those areas will have all the incentive to jump in as soon as they feel a big data start up is up for grabs.

Oracle might be the one exception to the rule here. A lot of big data startups were founded by people who can’t stand Oracle for some reason or other . So they might not sell to Oracle (but then we know what happened to peoplesoft few years ago, so who knows). End of the day, almost everyone has a price at which they will sell – and these big vendors might pay such a premium given the long term gains. Oracle, IBM etc have an enviable customer base, and have some very driven leaders – I won’t for a second under estimate them.

2. Partnerships

I am a guy that heads a team focused on partnerships, and if I didn’t think partnerships are an integral part of this big data story, I would not have taken my current job. I am firmly convinced that no one vendor can solve an end to end business problem for a customer. Most customers don’t want to spend a lot of time and money in integration work. While no integration is exactly seamless – customers expect a significant part of the integration to be available off the shelf.

Not only that, they probably don’t want the hassle of chasing multiple vendors for making a solution. So I expect to see a lot of OEM/reseller licensing in big data world where even competitors sell each other’s products. It is not new – IBM and Oracle, SAP and Oracle etc are all relationships based on co-ompetition. I think we will see this become mainstream across the board in less than 5 years. I also think that some companies will shy away from this model – and probably perish in the process.

Even the vendors who buy out big data startups cannot avoid this partnership thingy – customers will force the issue if they shy away.

3. Open source will become the norm for software business models

I will go out on a limb to say in 5 to 7 years, open source will be the normal way a company thinks of software. Even the closed source vendors will embrace it more openly (pun intended). The reason for this is simple – as the newer big data vendors take a stance that they don’t need to maximize any given transaction, and are willing to give customers more value than they pay for , it becomes harder and harder for closed source, maintenance revenue based businesses to keep up their current model. Even if they get bought out by closed source vendors – customer expectation will not change much I think.

So why should it take 5 to 7 years ? It takes time to make a dent in the universe 🙂

a. Open source big data vendors need time to mature their code base. They need to build tooling around their products that enterprises need. Remember the time before say Oracle 8i? I think open source vendors will go through that kind of evolution, except that they have to do it in 5 years instead of 20 .

b. Also, for the incumbent vendors to take notice of the new companies seriously – they need to be at least a $500M revenue (or a  trajectory to that kind of revenue) business . I saw this first hand with SAP Hana – Oracle and IBM did not say anything against Hana for a while till SAP started showing big numbers in hundreds of millions of dollars. I fully expect the same to play out against open source too.

c. Unlike ERP etc – I don’t think there will be one big mothership of a big data project at most customers. Instead, I think there will be a lot of parallel projects of smaller size. Some will go live and some will get killed as better use cases evolve. I think big data will drive a trend towards “disposable apps” in enterprise.  But it aint happening overnight.

And all this will take time to shake out – which is why I am pegging it at 5 to 7 years. But I have no doubts that open source will dent the universe !

MongoDB World 2014 – A quick recap


We just finished our big annual event – MongoDB World in NY. I am in equal parts excited and exhausted. My channels team totally rocked at the show – and I could not be more pleased with my gang.

IMG_4299

I have been here at MongoDB for about 3 months now – but I had no idea how awesome an event this would turn out to be . A big thanks should go to the able leadership of Meghan and Dina . And looking at my calendar – I think maybe the first one to thank would be my most efficient  assistant , Kaila Hecht , for getting me through the zillion meetings on schedule . Special thanks to Jen Seelin for a first class job handling AR/PR.

IMG_4298

I have had busy schedules at conferences as a leader in IBM and SAP, but this was at a whole different level of intensity.

We had about 2000 people attend – and it was our first multi day conference ever. More than a hundred people told me they could not believe this was our first big conference. I can’t blame them – I could not believe it either. On Tuesday afternoon, I walked out of a meeting with a large SI partner, and I found a really long time along the corridor. I was curious why all these people were lining up and stood on tip toes to see what was up. All I could see from where I was standing was the Men’s room. I was telling myself that we should have had planned better for bio breaks, and started walking towards the head of the line to tell people where the other rest rooms were – only to find that this was a line of people who could not get into the session for time series 🙂

We had a pretty good discussion with the customer CIOs on the first day, moderated by my pal Paul Cross and the most well respected Merv Adrian of Gartner

IMG_4291

As the dude running channels – it was gratifying – scratch that – downright THRILLING to see the kind of support our partners are giving us . Many of them had booths at the expo and unlike most tech conferences where people on booth duty out number visitors, here I saw plenty of traffic . For a first conference – how many companies will have representation from partners from North America , EMEA and APAC ? We did ! A huge thank you to all our partners !

I did have one regret – something I plan to fix next year. I could not listen in on most of the customer sessions. I can’t tell you how bummed I am to miss those. Next year, I am going to fix that and block some time for customer sessions. That is the most inspiring part of the whole event, and we had some rock star customer successes. I am used to conferences where majority of the content is produced by the vendor, and a small percentage of customers telling amazing stories. Boy was I thrilled – there was no shortage of amazing use cases. We had a partner from Israel who showed us how a telecom billing system was built on MongoDB. We had Citi and Goldman Sachs telling us about their MongoDB as a Service implementation. We had Parse ( now a part of Facebook) telling us how 275000 apps were enabled by MongoDB. And many many more .

What stood out clearly was the trend for some really big customers to think of MongoDB as a common platform for multiple projects, as opposed to a small little “toy” project. Enterprise software does not typically boast of tens or hundreds of thousands of apps – so this was very interesting for me to watch. There are a few different reasons for this. MongoDB being open source definitely helps in viral adoption. The price point for MongoDB paid subscription is low enough that cost pales into background in front of value. This is the opposite of typical enterprise selling motion – where a vendor tries to maximize every transaction. We don’t do that, and customers like it and use our technology more . And then there is the most amazing part of our story – our community.

IMG_4290

We have over 600 partners now in our partner program ! We have technology integration with  a broad range from exciting brand new start ups to household names in enterprise software like IBM, Teradata and SAP . As I have said before – it is not what we make , it is what we make possible . Partners are the ones primarily driving the “what is possible” – and they have been doing a fantastic job at that .

Our partnership philosophy is pretty simple – whether they are systems integrators, resellers, distributors, ISVs or IHVs, we want to enable and support them and get out of their way of scaling customer success. There are many things partners do that we can’t do by ourselves. A database by itself doesn’t have any direct business value – business value is delivered by the variety of apps we enable. Those are written by our partners and customers. So pretty much everywhere a partner can add value to a customer – we try to support them to the fullest extent, and get out of their way. The primary yardstick is to make sure we don’t compromise customer or user experience in the process.  We will succeed together – Our customers, partners and us – and we will do so at scale. If you are a partner reading this and want to know more about our partner program, please visit us at http://mongodb.com/partners

If you did not get a chance to watch the keynotes – you absolutely should watch the videos. Plenty of tech goodness there – document level locking , pluggable storage engine, MMS automation etc – , as well as a lot of explanation of the big picture vision, customer successes etc. You will love it. I had fingers and toes crossed for the live demos – but Eliot and gang pulled it off in style. For me in particular – I loved the fact that all keynotes were short and to the point. Understated , elegant and yet quite enthusiastic. Hopefully this becomes the trend in tech conferences. Same deal for tech talks – they were concise and had time for Q&A.

20140626-101726-37046663.jpg IMG_4297 IMG_4294 IMG_4293 IMG_4295 IMG_4296

I am going to take a few days off to recover, but I am looking forward to the next MongoDB World already 🙂

Random productivity stuff


I am always looking to find one more way of tuning up my productivity . Off late a lot of people have started asking me of my work habits , and I thought maybe if I jot down mine – maybe it will prompt a few more of you to chime in and we all gain a new trick or two

1. Sleep more

Counter intuitive as it may sound , I am more productive when I sleep well . So if I have to make a trade off between spending an hour prettying up a presentation or sleeping , I choose sleep every single time . I picked up this habit during business school and it works well for me till today

2. Don’t sign up for news letters etc on work email

If I need one issue of some magazine type thing , I use a hotmail id that I don’t monitor . If I am part of an email group that someone else is monitoring , I use an automatic rule to move to a folder in case I need to search later .

3. Delegate aggressively – both responsibility and authority

This needs trust and I have a simple policy on trust . I trust everyone in my team by default . Everyone has exactly one chance to break that trust . The second time it happens – I just stop trusting and won’t delegate to that person . Thankfully I very rarely have had a colleague break my trust . If you can’t trust your team , why should you lead that team any way ?

4. Move people to bcc the soonest you can when an email needs threading

If my colleague introduces me to someone on email and I need to set up time with that person – I move my colleague to bcc on my first reply, with a thanks . He just needs to know I am on it – he doesn’t need to know when I am doing the meeting and whether I need to go back and forth 10 times to set a time . It saves a lot of trouble for all the people who are drowning in email

5. Aggressively stay on top of email and twitter

Once you have the discipline of minimizing email , then be aggressive on making sure you read everything . I do it in stages – flagging as I go anything that needs more attention . Then I get to them throughout the day . No mail stays unread when I go to bed . The one exception is vacation – where I usually schedule a day or two at the end to catchup on mail before I return to work . I can type fast on my phone – so I don’t need a larger device for email .

Same deal with twitter – I respond in real time if I can when I get a notification that needs my attention . It takes a lot less effort than doing so in batches .

6. Keep all conversations as short as they can be

On internal emails , I rarely write more than couple of sentences and sacrifice the salutations etc . And if an email thread starts to bulge – I pick up the phone and call and get it sorted out quickly . Email warfare is the biggest productivity killer – and it is full of misunderstandings . Phone solves that easily .

7. Unless it is a customer related activity , don’t get into a string of back to back meetings

My internal calls are all on Fridays . And on other days , I take time off calendar to let serendipity do its wonder . I will talk to anyone who will talk to me at that time – and more often than not, good things come of it . There is no excuse for not having time – we always make time for things that are important for us .

8. Value agility over elegance

This is an easy trade off for me – if I can do something to solve a problem then and there , like making an email intro for a friend , I just type as I talk . It won’t be polished – but it gets the job done and helps me do more . The exception is with customer facing stuff where I am more careful , especially with prospects who need to be converted to customers . Slip ups can be costly in those cases .

9. Planning is everything , the plan is not everything

The process of planning is useful – we get to think through stuff carefully. But the plan we come up with shouldn’t be set in stone – because circumstances change . So when people say there is no budget for this change or that change – I don’t take it at face value . There is always something else that can be sacrificed . Budgets and plans are for business convenience – they shouldn’t be the tail wagging the dog . There is a limitation that this only works for things under your influence – it’s hard to convince others to view plans the way you do 🙂

10. Spend time with family

I have a crazy work schedule on week days . So on weekends , I try hard to not get pulled into work related stuff . I am not completely successful in turning off from work a 100% , but I am much better than say two years ago . There is no one else to blame – if I don’t set a boundary , work will set a boundary .

I am sure there are more things – but this is a good summary . So what are your tips ?