Future of Software Development 


There are so many angles to this topic – and this is my third attempt in three days to organize my thoughts around it . The trouble is not that I don’t have enough ideas – it is that most ideas seem to contradict each other when I read them again .  Let’s see if the third time truly is the charm 🙂


1. Everyone will NOT  be a (meaningful) programmer in future 

I know that is not a popular position to take today – but that is where my mind is at now . We will need to water down the definition of coding significantly to make “everyone is a coder” be a true statement . If I can change the tires and oil of a car  , and program the infotainment system – should I be called an automotive engineer ? That is “roughly” how “everyone will be a coder” sounds to me now . 

Don’t get me wrong – I do think that understanding how code works is important for everyone in future . That doesn’t translate to everyone coding , at least in traditional sense . Very complex applications exist today in MS Excel – created by business analysts who are not traditional programmers . If we change the definition of coding to include that kind of development – I can buy into “everyone will be a coder”. The closer statement – though less sexy – would be “everyone will be a designer or modeler” !

2.   More code will be destructed than created 

World around us is changing extremely fast and that means we need lots of newer kind of applications . But the pace of change is such that no application can satisfy a customer for any length of time . Building better components and better orchestration mechanisms are the only way to deal with it . Neither concept is new – but the execution will kick into a higher gear . API designs will need a lot more flexibility than we are used to 

3. Performance will trump simplicity 

By simplicity – I mean what “humans” think of as “simple”, not machines . Code for tomorrow will be used more for machine to machine communication than for machine to human – by orders of magnitude . Creation of code itself might not need a lot of human help for that matter . And while maintainability and human readability are important today , it might get trumped by the need for extreme performance tomorrow  . For example – if both ends of an interface are machines , why would they need to communicate in text and pay for the overhead of things like XML/JSON tags that need to be converted to binary and back again to text ? 

4. You won’t need as much code in future 

A lot of code is written today because a human does all thinking and tells computers what to do in very prescriptive ways with conditions and loops and all that. When computers get to “general AI” – they will learn to think and adapt like humans – and won’t need step by step instructions to do what they do today . Less code will do a lot more than a lot of code does for us today . We may be decades away at most – we are not centuries away from that stage . Software will eat the world now , AI will eat software tomorrow 🙂

5. Software offshoring/outsourcing  will not be for development or maintenance – it will be for training 

It’s already possible for machines to learn from vast amounts of .  Some time in far future , machines will self train too . Till then – and that’s at least a decade or more – humans will need to train machines on data . And that will need to make use of local knowledge , labor arbitrage etc and hence will be an ideal candidate for offshoring and outsourcing ! 

6. Community of developers will be the only thing that matters  

Well – that is already true, isn’t it . I have forgotten the last time I have checked official documentation or real books to learn anything . I google or search on stack overflow to get most of what I need . I am not a full time developer – but looking at the communities that help me , I am sure full time developers do what I do , a lot more than I do 🙂 . A better way of mining this treasure trove of information is the need of the hour to get significantly more engineering productivity. 

7. More and more use of biological sensors 

Human bodies and brains are the ultimate computers and we are some ways away from mimicking human thought . In near future I expect simple sensors for all kinds of chemical and biological stuff ( how cool would smell be as an input , like touch is today ) that provide input to and also dictate how code should work . Text is fast becoming the most boring part of data any way 🙂

8. We haven’t even scratched the surface of parallelism 

What we call as massively parallel today in all probability will be amusing and funny to tomorrow’s programmers . The over heads of making parallelization work today is pretty high – and that will go away soon. A part of the problem is also that majority of developers don’t think of parallelism when they design code . I guess the near term solution will be for more primitives in popular programming languages (like for data access) to have built in parallelism . Note to self : get better at functional programming in 2017

9. Ethics and Privacy become core to all development 

A few things are happening together now

a) data is exploding and we are leaving our digital finger prints everywhere 

b) applications won’t stay around long enough to have ethics and privacy as a “future release” issue to be fixed

c) more and more software affects humans , but is controlled by machines with little human input 

d) access to information is (and will be ) universal – which means bad guys and good guys can both use it for what they want 

e) legal systems won’t ever catch-up with the pace of tech innovation 

And that means – ethics , privacy etc need to be core principles of tomorrow’s software . It cannot be “in pockets” as it happens today. And the education on this topic needs to be pushed down to even the earliest levels of schools. 

9 is really not a conventional number of bullets for a list – but given there won’t be anything conventional about the future of software development , I think now would be a good time for me to stop this list . Feel free to add , edit and challenge in the comments – I look forward to it .

Happy 2017 everyone!

Advertisements

Future of Project Management 


Next to programming , Project management is the role that gave me the most satisfaction in my career. So after Rethinking IOT and AI for future , and Future Of Technology Consulting – I spent some time organizing my thoughts on where project management is today and where it is headed .

This picture is an old one – where I was leading a consulting team as the PM at my client, and we were codeveloping a product with SAP. There was no way to distinguish who worked for which company in this team. It was a highly stressful time – but also the most fun and productive time of my life 

team

In general I think project management as a profession has lost its stature and for all the wrong reasons . I also think that it will regain its lost glory, and then some, starting almost immediately !

Utterly stupid is how I would describe the move to commoditize project management over the last few years . The PC version would be penny smart, pound foolish !

Several factors played a part – and I think the wrong use of PMP certification is one big reason.  I am personally not a big fan of certifications in general. I (and others) have successfully managed hundreds of millions of dollars worth of projects successfully without a PMP a . When I was a full time PM (also when i was a developer) , none of my clients ever asked me if I was certified . In my view PMP and tech certifications are a definite plus for the job – but should not be a mandatory requirement .

PMP gives a false sense of security and accelerates the path to “if everyone has a PMP , they must be roughly equal in skills – so let’s choose the cheapest one for the job” . When I convinced my old boss many years ago that I don’t need a PMP – my defense was that we commonly knew at least ten people in their early twenties – who have never even been a team lead – pass PMP exam with flying colors, and neither one of us were confident enough to let them run a team !

To be perfectly clear : PMP itself is not to blame . I have studied the “body of knowledge”  closely and it’s pretty good . I encourage all PMs and aspiring PMs to study it . I am just strongly opposed to treating it as a way to falsely equate everyone who has it to be of same project management ability .

Becoming a PM is best done in an apprenticeship model . Project plan , documentation , chasing down tasks etc are good things, and you can learn it from books – but successful projects are mostly about making people successful  , not tasks successfully completed ! There is a big difference and a full appreciation of that only comes from watching and learning from folks who do it consistently well . However smart you are – you can’t learn it by studying a book or taking a multiple choice exam .

Sadly – and probably due to the mandate to commoditize all parts of IT projects  , task management – which was a means to an end in the past – seems to have become all of project management today !

Consistency and repeatability and scalability are all good for efficiency . So dumbing down of some project management aspects have that aspect going for it . But what is missed out today is effectiveness – efficiency without effectiveness leads to failed projects . And effectiveness is all about people !

People have only so much intellectual and emotional capacity and not all of it is spent on work . Example – the best programmer in my team in Bangalore spent 4 hours every day on commute . Even then he was twice as good as the next programmer . I let him work Mondays and Fridays from home and he became three times as good at what he did . I knew that issue because I went to Bangalore and lived there for a month to see the team and work with them and become one of them . I couldn’t get the same result by asking him to document more or sit in more status calls . I also remember a situation where we had an unreasonable client who made constant demands of our time to meet time lines that were not realistic . After two weekends back to back at work – my team had no energy left . My solution was to stop working weekends and instead we all went out bowling for a whole day on Monday and followed by a potluck on Tuesday . Even the client could not believe we hit the deadline with room to spare !

Motivating and getting the best out of your team is one aspect – equally important is making your client successful . By that I don’t mean the client company – I mean the human beings from the client team who work with you and sponsor the project . This means you need to get to know them , what makes them tick and what success means to them. No certification teaches you empathy !

To make clients successful – you need to know their business and their industry cold , or know others whom you can tap into for that knowledge . You also need the ability to make short term vs long term trade offs .  I once had a finance director of a company as my client – and she was stressed out that there wasn’t enough time left to build 150 reports that were scoped for the project . I worked with her and told her similar projects in past only needed 50 or so reports for similar functionality and the two of us spent a day looking through the specs and quickly brought it down to 40 reports . My employer had a short term revenue loss because of reduction in scope – but this lady was publicly recognized by the CFO of the company for getting the project done on time and under budget . And she got a larger portfolio and I got a lot more business from her , which in turn helped my own career progression .

Project managers need the respect of their team to succeed. PMs who manage a project where they don’t know any aspect of what is being done generally find it harder to get the team’s respect. It can be done – but it is an uphill task and you need superior skills and patience. This is another reason why commoditizing PM skills is a terrible idea – people who grew into PM after being developers, consultants, team leads etc can empathize and add quality to their team’s work much better than someone who can only manage tasks.

Why do I think this will change quickly, and for the better ? Its because the complexity of projects and client expectations have both risen to a level where commodity skills and elementary automation cannot keep up. Fear of failure is very high today thanks to a lot of failed projects in past – and at the speed at which technology is progressing, there are very few “apples to apples” references to say “this will work”. Good solid project management is the need of the day to help realize the value of technology innovation happening around us. I think employers and clients are both ready – or very close to being ready – in treating PM again as a critical role in making projects successful .

Those of you who manage development teams as PMs might enjoy this post this post I wrote in 2010 🙂

PS : Might as well add a shameless plug – If you have experience as a PM in big data, analytics, IOT etc – I am hiring in North America. Ping me !

 

Free Does Not Work Most Of The Time


Ever since I had my first job, I have wondered about the concept of “Free” in corporate world. And many years later, I am pretty convinced free is usually a bad strategy for all parties. This is strictly a rant on my personal views – nothing official about it, and do not represent the views of my present and past employers .

Nothing is really free

That took me some time to realize – absolutely nothing is really free. Someone has to pick up the tab always. Since I have worked in the consulting business most of my life – let me use that as backdrop to explain . This is true for big and small consulting companies I have worked for.

Customers do pay a pretty penny for consulting – and rightfully think that since they are giving so much business for the consulting vendor, they should get some things for free. This is usually in the form of proof of concept work. And most of the time – Vendors do agree to throw in free POCs. Vendors do not do this out of kindness – since they typically do this only if there is upside down the line for them via more business. The POC starts – customer does not always go all-in for these projects , given they are not putting in direct money on the table. And eventually the project finishes with no one happy and no decisions made . The Vendor does not exactly lose here – they will make it up in next project, either at that customer or at another customer. Those vendors who do not have multiple projects and clients might actually lose serious money in these POCs, and hence they might not do it a second time either. Whichever way you look at it – no body is moving a good step forward in this picture.

Even when something is given away for free, it does not get used much

If you walk the show floor at any trade show, you will get a lot of free stuff – from coffee mugs to iPads. I have seen software companies give iPads to their prospects and clients preloaded with presentations etc, that usually end up in a teenager’s back pack in few days, usually without the presentation ever being looked at. And no prizes for guessing who is paying for those iPads 🙂

Then there is the case of vendor charging a maintenance fees for software, and using part or all of that fees for “free” new functionality added to already sold software. Customers might genuinely like to see all kinds of things to come out of this arrangement for free – new business processes, mobility, BI etc. The hard part is – it is next to impossible to draw a clean line on what should be free.

If you look at the adoption of the new functionality provided for free , very rarely do you see big adoption. But if you look in further – there might be other reasons for this , like cost of hardware, testing, change management etc. So on one hand, the vendor uses a lot of money for making stuff, and on other hand customer has no way of using it – due to lack of awareness, lack of resources or lack of interest. The take away here is again that Free did not work as expected.

It can of course be argued that in a multi-tenant SaaS model, customers might use free stuff more often. I seriously doubt it. Example : if you never had parallel ledgers till today, and now your cloud vendor gave it to you for free – will you implement it ? Many customers will not – either because they don’t care, or because it needs more change management , SI work etc (probably less than on premises world, but still usually enough to help inertia rule). And it is not as if cloud does not have lock in – if you have any doubts, look at the SEC filings of cloud companies on internet.

Should Vendors charge for a different UI or for mobile versions of existing applications ?

UIs will change over time – as technology changes (hardware and software advances, consumerization of IT and all that) . Should Vendors charge for that? Will making it free increase adoption and make customers happy? I don’t know the answer – but my (of course biased) answer is that it is fair to charge for this. Here is my rationale – when customers don’t like UIs, they will find work arounds. Vast number of screens delivered by vendors are replaced by loads from spreadsheets . I know a company where the financial analyst loads JV entries twice a day by putting it in an excel sheet and putting it in a sharepoint drive, System does the rest and the analyst is happy and productive. I have asked this guy personally many times if a different UI is a better solution, and he consistently likes to stick with his excel in sharepoint approach. I know a hundred other examples like this where people refuse to move to better UI for fear of change. Giving a free UI for these cases just would be a bad investment.

Mobile is a harder nut to crack. Vendors with a limited footprint – like just HR or just CRM as their offering, might throw it in for free and build it into their price case. I think that is the right thing to do for them since it makes business sense. However, for vendors who sell many different things – they might not have a good way to do this across their portfolio. For these cases – All I can say is “pick your battles” . They are probably better off selling packages mobile applications for best usecases. Or they might sell (or partner with) some development platform that helps customers build their own. Or maybe leave it to the partner ecosystem to bridge that gap. It might also make sense to throw in a few things for free on mobility front if it makes sense for competitive reasons.

But if you charge – what is fair and what is not? I have a simple POV on that – the price to charge is the maximum $$ that will not stand in the way of adoption. Price is driven by market – if you over charge, customers won’t buy and use it. So start with what you think is fair, adjust as you go – with a good tradeoff between adoption and your financial KPIs like revenue and margin.

What about consumer side of the house?

You would think that consumers are happy with all the free stuff they get – like facebook, free uploads in flickr and so on. Nothing is really free there either – you still pay in terms of sacrificing privacy, suffering through advertisements (or paying to avoid the advertisement). And of course you use the same facebook to complain of their free service 🙂

So in short (well, I guess this was not very short – sorry) – I doubt “free” really works anywhere. There is no free lunch – I have accepted that and made peace with it. What do you think ?

How do you measure success and failure of projects?


There are quantitative and qualitative ways to describe the success or failure of a project.

In general, a project is considered succesful when it meets its objectives while staying within an agreed up on budget and time line. This sounds simple enough, except that it is not !

First, who decides if a project is succesful? And if a project passes a series of tests and goes live, is it considered succesful? The common model is to celebrate success at go-live. But going live is just the beginning of the journey. It takes a fair amount of time to judge if the business got sufficient value out the project. Also, at what cost? Did they have to hire more people to support the implementation? Can they make enhancements to the functionality as requirements change, without costing an arm and a leg? Do we attribute this to the original project?

Since I am more familiar with SAP projects than non-SAP projects, let me quote examples from SAP area. In the nineties, SAP was not as rich in functionality as it is today. Many implementations had dirty modifications to source code to make it work according to requirements. These were called succesful projects at the time. 15 years later, these same modifications have caused the customers a lot of trouble in upgrading their systems. So do we still call them succesful?

Second, all stakeholders have to agree to a set of objectives that a project has to meet. Sounds simple enough – but then, it is kind of hard to accomplish too. First, there is the legal issue – if you cannot express it in a language that lawyers agree to, the objective will not be present in the contract. And if it is not in the contract – it is hard to ensure it will be met. Then there is the question of the definition of who a stakeholder is. In large projects, there are many people who are affected, and it is impossible to get them all enlisted to create a list of objectives. So a subset of people is assigned as leads or representatives, and they may or may not know how the new project is going to affect the work of the whole population. 

In one of my first succesful SAP projects, I remember going to the office the day after go-live and finding a large number of trucks backed up from the loading bay all the way into the street. Guess what? no one remembered to capture the requirements of the picking department – so they tried to do it the old way, like they have done all their lives – and then could not update the system with the results. Thankfully, we could solve it with some quick work around – but the point is – you cannot figure out every one’s requirements all the time in big projects.  Voice of the majority does not mean something is right – there are plenty of exceptions in every business process, and they all should be handled.  Consultants swear by “Mutually Exclusive and Collectively Exhaustive” – just that as scope increases, the chance for hitting this paradigm decreases.

Third – how do you agree up on project time line and budget? It is by estimation, which is an inexact science by definition. Based on prior experience, some one thinks that a certain task will take a certain amount of time to finish.  Just as the stock market has taught us that past performance is not a good measure of future (for the short-term),  there is no guarantee that these estimates will hold true across the life of the project. Techniques like PERT make us eliminate bias by using formulae that goes from pessimistic to optimistic. But as all experienced project managers will attest – estimates are best guesses, irrespective of techniques. 

Now – this means estimates have to be revised as projects progress and we have more data. But the question remains – if I estimated $2M today, and then re-estimated after 3 months to find that it actually costs $3M – is the project already a failure? The world outside the project typically looks at projects with this perspective. Once an estimate is made for time and money, then the world expects it to be set in stone, come what may. There is no room for re-estimation.  So most projects get into this situation that they are set up to fail from the beginning.

Finally, no project has unlimited resources – so scope and timeline usually is subservient to available money. This means some one has to prioritize scope. And in this process, some requirements will need to be dropped. Despite the due diligence in this exercise – it is hard to avoid a decrease in value to some part of the customer organization. But when the project finishes and these gaps get exposed – usually it is put as a failure of the project.

Now, I am not the only one to have this knowledge – most customers, consultants and software vendors know this. But why would not this situation change? Just a rhetorical question of course 🙂

South beach diet didn’t work for me, and neither did Agile development.


Normally, I would write this in my SCN blog – but this is not about just SAP projects, I am going to do it here. As always, these are strictly my personal opinions – not that of my employer.

Some of you have seen me  – at work, at some tech conference, dog show or at an airport. I doubt if “Agile” was the word that came to your mind when you met me. I could easily lose 20 pounds and have some one ask me “hey, why are you not doing something about your weight?”. You get the idea. I have tried many a weight loss / excercise plan – and have  come to the firm conclusion that the magic bullet for weight loss is to eat less of everything I like (Rice, red meat, fried food) and excercise more.

In parallel, I was going through a similar excercise at work – trying to find an optimum way of managing the projects I get to run. I have read and tried several different things over the years in a variety of projects. Over the last couple of years, I have been fascinated with Agile development and hence I have been reading, talking to others, and trying it out in teams I manage – and again, I have  come to the firm conclusion that, just like the various dieting schemes – Agile also does not work for me.

This is not to say Agile does not work for others . South beach diet must have worked for others, and I am sure Agile would too. For me – no sir, I will pass. That being said, let me explain why it does not work for me.

Would you pay an Agile contractor to build your deck?

I need a deck built, and I don’t know how to build one. So I hire some one else to do that. And this dude tells me he can do it one of two ways. 1. I can discuss with him on how a basic deck has to be built, and he can give me an estimate. Or 2. I can give him a rough idea, and he will start building the deck, and every day or two – he and I can get together to see how it is going and what changes I want, and I can pay him for work that he has done every day. Of course in the second option, he cannot tell me how much the deck will cost me or how long it will take him to build me one – but I can see progress every day. I don’t know about you – but I know what option I will go for.  Same thing with my clients – if the work and schedule are not predictable – it is hard for them to just pay as I go.

Global and Agile are like Oil and Water – They don’t mix.

I forgot the last time I had all the IT and Business guys and gals working in the same location.  Most often – we have people working on a project from all over the globe. It is seldom possible to get teams from Japan, India, Germany and US to be on a conference call. And even if you do – without a written document explaining the problem and/or solution – it is hard to get anything done. 

There are only so many super stars in this world

In any given team, the norm is to have a few super stars, several average performers and a few below average ones. This has a direct effect on pulling off delivery in an agile fashion. Not every one can go away with minimal instructions and come back with the right solution, and right questions to ask for next day. 

It can be argued that Agile needs less programmers, and hence you can keep just the super stars and let go of every one else. This argument works only if the scope is small.  The day still has only 24 hours – and even super stars  cannot work 24X7  all the time to get everything done.

Most projects do not have dedicated business users.

The whole idea of Agile is to get something back to business users faster than in a waterfall model, and keep them informed of progress frequently. This is very good – except it won’t happen in most projects. Most companies find it hard to dedicate full-time business people to a project. More often, business users have to do project work on the side. So – even if the tech  team wants constant face time , the chance of that working out is low.

A few product companies have tried out Agile successfully. However, in the couple of cases where I got a chance to talk to people from the team – it seems, the customer was almost never present in the scrum meetings. Instead, the product manager assumed the role of being the customer voice. If that is the case, I would have to wonder aloud “so what is new?”.  Product manager is not the one who has to live with the product after it is out – it is the customer.

What works for you –  people over process? or process over people?

This is what it boils down to – Agile manifest claims the superiority of people over process. And traditional waterfall puts process over people. For me – as a manager of big teams, with a deadline and budget that seldom cuts me any slack – I would trust a good process to compensate for the human errors most of the time. I think it is a high risk for me to trust that every one in the team will perform to the same high standards. Having a disciplined process helps me get the best out my team, despite not every one being a super star. 

Would you build a mission critical solution using Agile?

I don’t know – but I keep wondering if NASA would use Agile for designing systems for their next space mission? I somehow can’t see any application that is very important and deals with life and death, or dealing with large amounts of money – like air traffic control or stock market transactions –  to use Agile.

I hope every one knows about 3C. This was the big Chrysler project that was the poster child for XP (Extreme programming). Well, guess what – that didn’t quite work out. Check this out http://en.wikipedia.org/wiki/Chrysler_Comprehensive_Compensation_System  or just google and you will find several interesting takes on it.

Building a car and building a software solution have similarities in design. However, there are significant differences too. Just as it is impossible to build a car without knowing what exactly needs to be built – we cannot build a good software solution without knowing what the heck we are building. Otherwise, even if you follow Toyota manufacturing Process to build a Camry – you could end up with a Chevy Malibu.

Is Agile really good for long-term stability of a solution?

Most software is built by first putting a framework in place, and then building on top of it. It is the rough equivalent of putting a good foundation for your house. If you know that you will have a house with 2 floors – you will probably put a certain amount of concrete in your foundation. Now that you have finished the house – and for some reason, you now want one more floor – would you put one more floor without also doing some additional foundation work? And if you build by constantly messing with your foundation – I am definitely not going to buy that house or rent it for living.  Same thing with software – the way sprints happen from what I have seen, I doubt if it is possible to put a solid foundation in place.

It is fun, but is that good enough for customer to pay?

One thing I really like about Agile is that every one involved in it usually has more fun than in a waterfall project. This improves team morale and all the good stuff – temporarily. When you cannot get user involvement, or if a blame game starts – where there is no documentation to go back and clarify what every one agreed to, this fun does not always last. And fun for the development team, while important in a project, is not the sole reason why some one pays for a solution.

Waterfall is not such a terrible thing to do, as its opponents make it to be. And it is a big exaggeration to say the team does a very long design up front, and that user gets to see things only at the end. That is not how most projects run. Waterfall can also have users involved more frequently. Also, you can build in plenty of  feedback options and test driven development in a waterfall project. Also there are very strong visualization tools that can give the users a taste of the solution very early in the process.  For example – iRise is a great tool to use for that. Do try it out, and your whole perspective will change. Also, once you have a good change control process established, changing requirements can be handled very effectively.

So, when do I think Agile would work?

Despite all these points above, I am not an extremist when I think of methodologies.  In my opinion, Agile is going through a hype cycle exactly like how it happened with SOA.  Once the hype died down a little, we mostly figured out what is possible with SOA and what is not. SOA is great for several use cases, and terrible for others. Same is the deal with Agile.  Just because Agile possibly has the issues I called out above, it should not be inferred that Agile should not be used. If you have a bunch of highly skilled people, where the team has a clear vision of the end product, and/or where time to market is not such a big deal – all these challenges will vanish. Such a team can probably come out with a great solution using Agile. But in the type of projects I am dealing with – I don’t think Agile can succeed. 

Rather than take an extreme view of either/or – it is probably best that we let individual projects decide the methodology they want to follow.