MongoDB for the SAP Community


One of the most common questions I get from my friends from the SAP ecosystem is how MongoDB works with SAP. So I thought I will pen some initial thoughts on how I see the technologies working together. 

NOTE : Please don’t look at this as an official roadmap or anything – this is just my personal view. Neither MongoDB nor SAP has agreed to do anything of this sort . This is just Vijay thinking aloud – and I am sure someone will set me straight if I am not making sense.

First, what does MongoDB not do with SAP systems ?

This is easy – MongoDB is not a database to run your ERP / business suite or BW type systems. MongoDB also doesn’t do native predictive analytics.That is what Hana (or other RDBMS systems) are for. MongoDB has no competitive position against Hana. Also SAP Hana Cloud platform has been using mongodb for some time already for content management. And MongoDB is one of the databases for SAP’s Hybris platform.

Now that we got that out of the way, lets see what can MongoDB do to complement an SAP landscape

1. Visualization

SAP announced a technology collaboration for Lumira to work with MongoDB at SAPPHIRENOW last month. Lumira is SAP’s agile BI solution . MongoDB holds data in JSON (BSON internally ) . The existing connector uses a JDBC protocol for the integration – and more native options are planned for future.

What is the big deal ? the big deal is that MongoDB holds richer data than what you typically see in rows and columns of a table. 

MongoDB holds data that has a hierarchical or nested nature – and visualizing that rich data ( for example, not only what region that customer is from – but details of what is the past history of sales in that region for customer, what other customers have bought in similar categories etc) is an amazing value add for a business user.

On top of that – given the dynamic schema, you can keep enriching the data (like campaign information at real time from a web site) and visualize the results immediately . 

2. ETL/EAI

I could totally see in future SAP data services and Hana cloud integration for ETL/EAI use cases where MongoDB is the source or target. MongoDB is probably one of the most  popular database for cloud applications already – and that trend will only increase. But those apps need data to be fed into it from somewhere, and I think data from there might need to get pumped into other systems. So an ETL/EAI type tooling from SAP around MongoDB is something I can imagine being a sensible thing assuming those sources and targets could be SAP systems.

3. As a datahub for integrating SAP and non-SAP applications

Consider a business application SAP would build on Hana for retail . The sources of information that need to pump in data to that app will typically be disparate in every sense – POS data, ERP data, IS-Retail data, Web store data, Click streams and so on.

One way to do it is to pump all this data into Hana and write code there to filterthe dataset that the app needs. This will make sense for cases where the schema is set in stone and not subject to frequent change. But that is usually not the case for retail like data – schema needs as much flexibility as possible. So a second way to design that will be to use MongoDB as a data hub to consolidate data from all the sources, and then from there provide the app on Hana with just the right data it needs to do its thing. 

There is no certified connector for MongoDB with Hana today. But that probably is not such a big deal for a project based solution once the Hana SDK for AFL (app function library in C++) is available. MongoDB has a C++ driver that can be hooked on to AFL  and this should solve the data exchange problem from a technology point of view.

Another option – a little bit of code on XS engine (the lightweight web server/app server in Hana)  can make it work too – since XS understands JSON well. Thomas Jung probably is the best contact to help with this – but I don’t think you will need a lot of support. 

And for non-Hana customers of SAP, you probably need to code in ABAP from the app server layer to make mongodb integration work. I have not opened an ABAP editor in a while – so my memory is rusty. But I think its only from 7.x that ABAP has native JSON support. So you might want to do some hack to check if Kernel supports JSON. Horst Keller might have published some standard way to do that. Or if your ABAP skills are not top notch and you would prefer to use PERL or something outside the app server that MongoDB has a driver for . I guess you can create a service via ICF and take an RFC route. Probably not the most elegant way, but it should work too. Maybe Thorsten Franz can enlighten us on the possibilities.

Independence – view from the corporate jungle


USA celebrating its independence day tomorrow . And coincidentally, I have been pinged by a lot of people with messages along the lines of “I can’t do anything independent where I work. What should I do?” type stuff. It is a question I have asked myself as an employee and as a leader/manager.

In primary school – I learned a poem in malayalam that roughly translates to “even if the bird is put in a fancy golden cage, its world doesn’t have any independence” . A lot of corporate peeps feel that way that they are stuck in a fancy cage. So what can be done about it ?

Table stakes
To begin with there are some basics – which might seem obvious and downright trivial.

1. Live within your means – or ideally, a little under your means.
2. Max out our at least match the employer contribution for your 401K if possible
3. Save a litte bit for a rainy day, invest wisely
4. Take care of your health (something I definitely need to work on)

Even if all you do is these 4 things – you should have a little bit of a safety net to take a few big swings when opportunity arises.

Then comes the core issue of employment.

Let me share a few principles from my own career 

1. Always keep your skills sharp and up to date. Even in terrible economic slumps, my skills helped me switch jobs. It takes effort and passion to do this. If you treat your job as a way to earn a living and your heart is not in it – you will find it hard to keep your skills up to date. It is a choice you make – at a minimum, keep it sharp or develop a new skill to get a job where your heart is. For me – programming was where my heart was. So right out of Business school, I chose to code instead of trading equities.

2. Loyalty in a perfect world is a two way street – but alas, the world is not perfect. So while it is a great trait to be loyal at work place, don’t let it blind you as an employee or manager. End of the day – there is a business to be run, and the sooner things are put in that perspective – the easier it is to make and accept decisions.

Loyalty is a complex subject. When I joined the workforce – my role models were people like my father who spent decades at the same company, rising through the ranks. It was a serious problem if your CV looked like you were jumping every few years. Yet – I switched employers quite a bit in my early years, and it only helped me.

3. Network naturally. I can’t emphasize “naturally” enough. I am not a fan of aggressive networking – it turns me off. Make friends with your vendors, customers, peers etc just like how you make friends in social life. Help connect people where you can. And at some point – usually sooner than you think – you will have a useful network. Don’t mistake this for buying followers on social media. Also – don’t wait for shit to hit the fan before you start networking. Start when things are good. But if you have not done that – start today and not tomorrow. Quantity of relationships is not a factor – you just need a few who matter.

4. Always have a few options – plan B, C, D etc. You can’t always make plan B’s when bad things are happening. You need to have other options figured out when things are good. If your skills and network is kept alive and thriving, this should be a natural side effect. Don’t let loyalty stand in the way of having an alternate plan – you don’t need that plan B triggered if everything continues to be good.

And when it is time for the swan song

Despite all your best efforts and intentions – you might still have to get out of a job. Maybe you have a jerk as your boss, maybe you want to be your own boss, maybe you want more money – all are valid reasons. But there is a good way to leave – and some terrible ways.

1. Always leave gracefully. Burning bridges don’t get you far – it is really a small world and you should try as hard as you can to not piss anyone off. If your employer doesn’t deserve you – and you think it is their loss, just walk away smiling with a polite thank you. Don’t give in to the temptation of mass emailing everyone on your last day, yelling at your boss and stuff like that. It only shows you as immature. Walk away with grace – go home and scream your heart out (leave your family out of the yelling part if you can). I broke this rule once many years ago – shouted at my boss, emailed a terse resignation, and banged the door on his face for not promoting me. It was stupid of me to do that – and I repent it.

2. If you can’t stand your boss – try to get another one before you quit. If you are good at what you do, there is almost always a way to make it work in another team at the same company. Pardon me for saying this again – but this needs those skills and network to kick in. If you can’t find another boss – then don’t sit there and rot, actively use your plan B, C etc.

2. Leave on your terms whenever possible. It is almost always better than being fired. The only exception I can think of is when being fired comes with a nice farewell compensation package from the employer. But for this to happen – you needed to have those Plan B,C etc figured out when things were good.

3. Keep in touch with former colleagues, bosses and customers and help them whenever you can. The way I look at it is kind of in a “good karma” kind of way. Help as many people as you can – and some day, someone else will help you too when you don’t expect it. If you expect reciprocity before helping someone – in most cases it won’t happen, and you will end up bitter. I can assure you – in my life and career so far, this has only helped me. It has not once hurt me.

4. Plan and re-plan your career periodically. When I came out of business school – I thought I could retire at Thirty five. My goal was to get a job that lets me do some globe trotting, stay in fancy hotels and have a home with a pool and buy as many dogs as I needed. Those things were all accomplished by thirty five – but I also learned that none of those were worthwhile goals, and that I could not retire in that time frame. It also taught me to not trust any plan – planning is important, plan is not important. I replan my career every year or two. I am sure things won’t work the way I have laid it out even today – but I don’t feel bad about it any more.

So that is it – those are probably the principles that helped me gain some independence in my career. I hope some of them will help your career too. Please share any tips you have too.

The future of big data – consolidations , partnerships and mostly open source


In theory, I am on vacation this week – except that I have not managed to get out of email yet. And to top that, I caught up on a lot of reading and as usual, have some thoughts to share. As always – these are just my personal opinions, not that of my employer.

The good(?) and bad thing about big data is that it has no rigid definition – it is more or less what we want it to be at that point in time. I like to think of “big” in terms status quo of a customer. If the customer only reports out of their ERP data, and now wants to combine CRM and SCM data with it , or maybe combine social data with it – it is already BIG for them. For a stock exchange – they have already figured out how to manage lots of fast and furious data. It is hard to tell them data is BIG now – data was always big for them. It might be a bit “bigger” now, but you get the point. “big” is relative to status quo.

At the moment – there are very few big data projects (as a % of all projects ) that add outrageous business value. Its like a white tiger – a majestic animal with great abilities and all, but rarely seen in the wild. But they are not a myth and they really exist. I saw one just this week and took a photo. I know many friends who tend to look at big data as a white elephant – and I now like to think of the current situation of  big data as a white tiger 🙂

tiger

So where is it all trending to? I have some opinions/guesses – not backed by any scientific research. So take it with a pinch (or pound) of salt.

1. Vendor Consolidation

I definitely think there is some consolidation about to happen in 2 to 5 years time. This could take many forms – established firms like Oracle, IBM, MS, Intel, SAP etc could buy some Hadoop/NoSQL type companies. Some of the newer players could merge. The big ones have enough cash in their war chest and those that don’t have cash might not mind a bit of stock dilution to be successful in the brave new world . Big data is what is going to sell more analytics, more hardware and more databases in future – so the incumbent big players in those areas will have all the incentive to jump in as soon as they feel a big data start up is up for grabs.

Oracle might be the one exception to the rule here. A lot of big data startups were founded by people who can’t stand Oracle for some reason or other . So they might not sell to Oracle (but then we know what happened to peoplesoft few years ago, so who knows). End of the day, almost everyone has a price at which they will sell – and these big vendors might pay such a premium given the long term gains. Oracle, IBM etc have an enviable customer base, and have some very driven leaders – I won’t for a second under estimate them.

2. Partnerships

I am a guy that heads a team focused on partnerships, and if I didn’t think partnerships are an integral part of this big data story, I would not have taken my current job. I am firmly convinced that no one vendor can solve an end to end business problem for a customer. Most customers don’t want to spend a lot of time and money in integration work. While no integration is exactly seamless – customers expect a significant part of the integration to be available off the shelf.

Not only that, they probably don’t want the hassle of chasing multiple vendors for making a solution. So I expect to see a lot of OEM/reseller licensing in big data world where even competitors sell each other’s products. It is not new – IBM and Oracle, SAP and Oracle etc are all relationships based on co-ompetition. I think we will see this become mainstream across the board in less than 5 years. I also think that some companies will shy away from this model – and probably perish in the process.

Even the vendors who buy out big data startups cannot avoid this partnership thingy – customers will force the issue if they shy away.

3. Open source will become the norm for software business models

I will go out on a limb to say in 5 to 7 years, open source will be the normal way a company thinks of software. Even the closed source vendors will embrace it more openly (pun intended). The reason for this is simple – as the newer big data vendors take a stance that they don’t need to maximize any given transaction, and are willing to give customers more value than they pay for , it becomes harder and harder for closed source, maintenance revenue based businesses to keep up their current model. Even if they get bought out by closed source vendors – customer expectation will not change much I think.

So why should it take 5 to 7 years ? It takes time to make a dent in the universe 🙂

a. Open source big data vendors need time to mature their code base. They need to build tooling around their products that enterprises need. Remember the time before say Oracle 8i? I think open source vendors will go through that kind of evolution, except that they have to do it in 5 years instead of 20 .

b. Also, for the incumbent vendors to take notice of the new companies seriously – they need to be at least a $500M revenue (or a  trajectory to that kind of revenue) business . I saw this first hand with SAP Hana – Oracle and IBM did not say anything against Hana for a while till SAP started showing big numbers in hundreds of millions of dollars. I fully expect the same to play out against open source too.

c. Unlike ERP etc – I don’t think there will be one big mothership of a big data project at most customers. Instead, I think there will be a lot of parallel projects of smaller size. Some will go live and some will get killed as better use cases evolve. I think big data will drive a trend towards “disposable apps” in enterprise.  But it aint happening overnight.

And all this will take time to shake out – which is why I am pegging it at 5 to 7 years. But I have no doubts that open source will dent the universe !