Microservices – What have we learned ?

Yesterday, I shared some of my thoughts on serverless architecture  and ended up getting a lot of feedback and a lot of it went back to SOA and then logically on to Microservices. So I thought it may be worth penning some thoughts on this topic as well. Microservices are not a fad – they get used quite a bit today across big important companies, although perhaps not very consistently. Also, I think we have enough experience by now to calibrate our expectations compared to a few years ago.

What is microservices architecture?

I am sure there is an official definition out there somewhere. But a simple way to describe it is as a way to design a solution as a collection of services, each of which does one thing quite well independently. Each service is its own thing – own process, implementation, language etc . The collection of such services that solve an end-to-end problem is then orchestrated with a lot of automation capabilities to form an application.

Why is this a good thing compared to a “monolith” architecture ?

Separation of concerns is not a new idea in designing systems. Microservices architecture is built on this principle. The traditional way of building an application includes a front end ( html/js) , a database ( SQL/NoSQL/File/…) and App server to handle logic. When we hear people criticizing “monolith” apps – they are usually referring to the serverside logic built as a logical whole.

Monoliths are not bad per se – they can be designed and implemented in a modular way, and can scale with the help of good design using load balancers etc. Just that when it comes to scaling, testing etc – you have to deal with the whole even though only a small part needs to change. As cloud becomes more and more the default deployment option, the flexibility to scale and change quickly becomes a bigger need than in the past. Microservices is a very good way to deal with it. Many monolith systems will co-exist with the world of Microservices .

How micro is a microservice ?

This is one area where the wisdom from actual projects tend to throw cold water on the theory and philosophy of microservices. The tendency for many engineers is to go super granular in service definition . Almost without exception, everyone I know who started with this approach has regretted it and agreed to start with fewer services and then break them into smaller chunks over time. The operational overhead is quite significant as you play with tens of services – you now have to maintain and monitor several services, and at some point there is a performance penalty for too much communication across a bunch of services that all do one little thing.

Another interesting aspect is whether your system needs to behave more in a synchronous fashion or an asynchronous fashion. When you break the system into smaller chunks, essentially you are favoring asynchronous communication between them. Then if you need to make it work in a synchronous fashion – you may question your granularity decision quickly.

What about the product/project team?

I have seen several ways in which teams are organized , and have spoken to folks who worked in such teams where I had no direct involvement. There are a few consistent themes

1. The need to communicate frequently and freely is a make or break criteria, way more than traditional approaches. With great flexibility comes great responsibility !
2. One big advantage that comes with microservices is that each service can be implemented with a different fit for purpose language. And each service might choose a different database for persistence. While that is all great in theory, just because you can should not translate to you should. For large projects – too many technology choices leads to diminishing returns. Choose wisely !
3. There is practically no good way to hand off to an ops team when dev is over. Microservices forces a DevOps culture – or at least DevOps tooling for sure. Its probably a good idea to get EVERYONE in the team some training in tooling. You need different muscles for this world than dealing with a Tomcat cluster. The promise of CI/CD needs a highly trained, high performing team. I may even venture to say that best practice is to have the same team that builds the team to continue to support and enhance systems built on microservices. There are just too many moving parts to effectively transition to a completely new team.
4. There is no substitute for experience. There are not enough highly skilled folks around , so the ones you get need to carry the weight of mentoring their less experienced colleagues. Written standards might not be enough to overcome this. A common observation is two services looking at the same business object – like a vendor object that is of interest to an accounts payables service and a compliance service – and interpreting the semantics differently. Only with experience can you catch this early and converge.

Is it truly easy to make changes compared to monoliths ?

If you are a microservices fanatic, you probably are well versed in all backward compatibility tips and tricks, and hence your answer has to be YES. I will just say that there are some cases where you wish you were working in a Monolith, especially when faced with pressing timelines. A good example is the changes many apps will need due to GDPR  . When multiple services need new functionality – you need to wrestle with the best approach to get this done. Would you create a new service that others can call ? Maybe a common library? Maybe change each service and make local changes? Each has obvious penalties. No silver bullets – decisions taken in designing the app will dictate whether you buy aspirin from Walgreens sized box or Costco sized box 🙂

What about monitoring, testing, debugging etc ?

All the overheads on these topics that comes from distributed computing are in full force here. This is one area where the difference is significantly more noticeable than in the monolith world. Many of us are fans of doing Canary releases . You should have some consistent philosophy agreed on upfront for release management. Whether we admit it explicitly or not, lean and fast deployment has a tradeoff with testing effectiveness. Essentially you are relying more on your ability of montiring your app ( via all the services and messaging frameworks and redundancies) and making quick changes vs trusting impeccable test results . This is a significant change management issue for most technology teams and especially their managers.

So is microservices architecture a safe bet for future ?

There are plenty of public success stories today of microservices implementations – and conferences and tech magazine articles and youtube videos and all that. All Major SIs have expertise in implementing them. So in general, I think the answer is YES. However, I am not sure if microservices over time will be any less frustrating than monoliths in how they evolve. I probably will get some heat from my purist friends for saying this – but perhaps one way to smoothen the journey is to start with a monolith as we have done in the past, then as it evolves – perhaps have services that call the monolith’s APIs. And as you learn more, break down the monolith to a full set of services. I am not saying this because I am a non believer – I am basing it strictly on the talent available to do full justice to a pure microservices based architecture in a mainstream way. Just because Netflix did it does not mean everyone can. In any case – the mainstream pattern in large companies any ways is to start with their old monoliths and roughly follow the approach I mentioned.