Vishal Sikka explained in his SAPPHIRE and TECHED keynotes that the general direction for SAP is to collapse the multiple layers that exist today, and move more and more of them natively into HANA. It sounded like a pretty neat idea to me when I heard it, and I did not think more about it for some time. Timo Elliot then mentioned the example of digital photography getting rid of obsolete layers and revolutionizing the industry. In his opinion, HANA will do the same.
Vishal and Timo are two of the most brilliant minds that I admire in the SAP world. But for some reason, a part of me started thinking – is collapsing layers a good idea generally? My education is in mechanical engineering, and not in computer science. So I started thinking along the lines of machine design to see if I can rationalize this somehow. And I failed.
I started an attempt to discuss with Timo about this on twitter, but it is hard to do it 140 characters at a time. And then my flight to PDX started boarding and I had to abruptly end the twitter session and run to the gate. Since I have some time now in the flight to reflect (and type), here goes.
Layers exist for good design reasons. Layers traditionally have been built for some specific purpose, and over time they get very optimized for that one purpose. As technology improves, and smart people get bright ideas – they can replace a given physical layer with something else that is better. But the layer in design time should still remain in most cases. Why? mostly because there are smarter people in every generation who will improve upon this, and Mr Murphy might show up unannounced.
Taking Timo’s example of digital cameras – the physical storage in films got replaced by memory cards. But there is still some storage layer that did not get replaced. Tomorrow something else will come up that is better than these cards, and this layer will further improve. Now – nothing stops a designer from combining thr storage with the body of the camera or the lens or something else. This might be pretty revolutionary – and might outperform other cameras too, and users might love it in short term. But the question is – with this design, can the next generation of designers add on benefits of advancements in each layer that collapsed? To net it out – does performance outweigh the need to maintain and enhance?
Another example is our family car. I love that car – but have had multiple accidents with it in the past. If the manufacturer of the car had decided to collapse layers – my insurance company would have probably paid off the car after the second time it had an accident, and I would have had to cough up more money to buy another car. It just would not have been easy or cost effective to repair it. I would have been pretty mad at this scenario since all but one accident was the other party’s fault. Fellow Indians who moved to the USA might resonate with this – when I grew up in India, we could get everything on my dad’s Ambassador repaired somehow. When the newer cars came up – the concept of repair was mostly rip and replace. It was not affodable for many of us to do this.
In a stable system, it is comprehensible that some layers could be collapsed. Example : BW has aggregates. With HANA, you can aggregate on the fly. So it makes sense to get rid of this layer once BW gets HANA as the DB. But here is the question in my mind – If and when HANA truly becomes big data friendly (petabytes instead of TB), will it help to have a logical definition of an aggregate that will tune the SQL to get a specific aggregation done more efficiently?
We can argue that most of the data in BW is in TB range, and hence this is not needed at this time. But that brings up another question on innovation – is it innovative if you have to constantly tinker with the design? or should innovation stand the test of time? Timo made an excellent point on twitter – innovation does not come in neat buckets. It is hard to argue with that, and I readily agree. But for innovation to be useful – I believe it needs to be moulded into a form that can work for an extended period of time. At a minimum, this should apply to anything that costs customers in the millions.
Even if I somehow convince myself that collapsing layers make sense in a stable system, since we have a good idea of what each layer does and what will happen in foreseable future – can I extend this and believe that this is applicable to newer systems? I am having a hard time convincing myself of this.
HANA is just about a year old or so. Revisions are coming fast and furious to the HANA software. And HANA has an aggressive roadmap to make it work for BW, ECC and so on in a short period of time. It means work needs to be done on many different things that make up HANA (stabiling the core, enhancing the SQL, scaling out and son on). In this scenario – will it be easier for SAP to develop and enhance HANA if more and more layers are brought natively into HANA ? May be it is – I don’t know. I have not worked in software product development. Hopefully some one from that background can explain to me why this is a better idea.
The closest I can relate to is a custom build engagement that I manage, which uses MSFT technologies. The architects followed a multi layered paradigm to build this. It handles mission critical information, and there is very little tolerance for error. We debated many times internally if it was possible to decrease the number of layers to improve coding speed, and performance. My chief architect held firm that each layer has a job to do and it will serve us well if we stuck to it. We agreed on it – and he was right. As soon as first release was done, we had to start on the next one, and there is just no way we would have kept up with enhancements and performance tuning aspects etc if we did not stick to the multi layered architecture.
Sure we could have gained some performance – but in the overall scheme of things, the solution seemed pretty good. But then – this is a project, and not a product. What works in projects might not be the right approach for product development. So I can’t make any firm conclusions based on this experience.
If I knew in my younger days that my career and interests will go this way, I might have skipped my MBA and done a masters in computer science or gone to a design school. Since I did not do that – I guess I have to learn this the hard way by debating this in public, and probably get electronically beaten up 🙂