So, you want to manage AI projects ?


Project management is hard as it is . There are three aspects of project management that I think need some extra attention when the project is about AI . Pro tip – if it’s your first time , consider buying a Costco membership for the BIG aspirin containers 🙂


The three aspects ( certainly not the only three) are

1. Estimation 

2. Change management 

3. Testing

It’s the very nature of AI based development that makes it harder than traditional project management . Let’s start by making a few assumptions 

Good engineers who can work well with AI are hard to find – but for this discussion I am going to assume skills are not an issue and somehow the PM formed a team with all the right skills. 

Since AI functionality is exposed as APIs in many cases , I will also make another “simplistic” assumption to make my point about PM that engineers don’t need to be big time mathematicians and statiticians to use the available APIs .  

Just so we are clear – you do need great skills , and you do need some experience with the math to do AI projects well . I am just making these assumptions to keep the focus on PM in this post . For good measure – I am going to use waterfall language here, but there is no difference if the project uses Agile 

Let’s start with estimation . The first step of the process is to define phases like say development and testing , and use some historical data to figure how much effort and time it takes . This is easy for many parts of development like web/mobile/DB etc since there are plenty of past projects to give realistic guidelines . Then we get to the AI part – let’s say just one model is in scope that uses Machine learning .

Now we run into the first issue – whether the model we chose is the right one for the app we are building . In many cases – we won’t know till we actually test it . And even if it works ok , it might not be valuable enough of an insight that it throws back to the app . So whatever we estimate – there is a good chance that while developing we will need to switch drastically to something else and that will invalidate prior estimates . This is on top of all the other things that can go wrong in the actual design of the app . 

Even if the model technically works – performance can become a dog . Not all models support parallel processing . In many cases , another statistical method might need to be considered after you realize performance is going to be a problem . Ergo – potential to rework is lurking throughout .

Then there is the second issue – which is that unlike traditional projects , source data can fundamentally change the fate of an AI project . Data is already a problem in traditional projects  – unclean data can increase ETL effort for example . But it’s kind of easy to estimate ETL effort after you profile source data . But even if data is technically clean – your model may not find it useful . You probably will miss data you need , or need data from additional sources and so on . And there is no good way to know this prior to putting the data through your model . It’s also not uncommon for your base application to change its paramarters significantly because your model needs different information eventually than what you originally designed . 

In other words – your dev and testing team can drive you nuts if you walk in as PM with “traditional” expectations. Now to be fair – it’s not because your engineers are willfully . They just have a harder time debugging when things go sideways – and a lot of things go sideways . The Costco membership helps with less expensive beer over weekends where you reflect your career choices  🙂

These two issues already make it hard to estimate and test an AI project . Now let’s look briefly at the change management aspect . 

Explaining what an ML model does in business terms is a non trivial challenge to begin with . Now to explain why the model doesn’t work as planned and why it needs to be reworked – without guarantees it will work the next time – makes it a much harder problem . Even when things work well – and if a business person asked you to explain why the model arrived at the result it did , its often hard to explain .

Good PMs educate their stake holders – AI projects can make them look like a tenured college professor 🙂 

So how does a PM mitigate these issues ? I will offer a few thoughts 

1. Start by educating yourself on fundamentals of AI . You can even create little chatbots etc with minimal to no coding . Get a feel for the actual work upfront before you take over the project . Make sure all your non technical people like Business Analysts do this too 

2. Set the expectation with the client on the nature of the AI projects before the project starts . You can do this with real examples using simple models that you can mock up in Excel and then help the client extrapolate what will happen if something goes wrong 

3. Insist on highest quality engineers . If you have to develop something from scratch today – developers need experience with multiple frameworks and know their trade offs . AI models add a layer of complexity on top and have their own trade offs . Having past experience won’t eliminate all the issues I called out – but it will help minimize the mistakes , and when things go wrong you will recover faster . 

4. Aim small , Miss small . Try hard to resist the temptation of trying to build a complex system in one go. Make small goals and course correct as you go . 

5. Double down on data quality . Encourage your ETL team and data scientists to keep looking at data , visualize them differently and identify problems as early as possible . 

6. Create a support system for you and your team . It’s a new field and everyone needs help from time to time . If you don’t plant your shade trees up front , you will face nothing but grief later 

I am very curious to hear your thoughts and comments !

Feedback – it is much needed, but it ain’t easy


The general idea of feedback in the context of management and leadership is like motherhood and apple pie. Feedback is good and without it, it is hard to improve. But that does not mean that we are good at giving and/or taking feedback .

Feedback becomes necessary mostly because, hard as we all try – it is really hard to be perfectly honest with ourselves to take corrective action in a time frame that matters. I am in my forties now – and I am perfectly honest about stuff that happened in my teens and twenties. I work hard on fixing some of those “issues” . I cannot say with similar conviction that I am honest about my strengths and weaknesses when I think about say the last 5 years. For that, I need to hear it from others.

People tend to underestimate or overestimate themselves. It is a lot easier to judge someone else ( and also say “who are you to judge me” )  🙂 On the work front – I have had very few managers who have consistently given me useful feedback . People ask me for professional guidance all the time too – and I am sure some of them think I suck at giving useful feedback as well.

Most feedback is pretty useless – generic questions get asked and generic answers are given. My two favorite examples of thoroughly useless feedback from managers to employees are

1. You need to improve your communication issues ( probably present in 90% of all performance appraisals for people in their first ten years at work)

2. You are doing great – keep doing what you are doing and good things will happen to you. ( Run, don’t walk – this comes from lazy managers who have no idea about your work or what they can do to help you get to the next level )

If you only seek or give feedback in long intervals of time, its generic nature is more or less guaranteed. A continuous process – can I make up a term “micro feedback” ? – is a lot more valuable. But for this to work – a few things need to fall in place

  1. For feedback to work – there needs to be mutual trust. And trust takes time. So don’t expect consistent overnight results.
  2. Aim small, miss small. Having a road map to your destination and being pragmatic about diversions along the way makes giving and receiving feedback a whole lot more valuable. Specific feedback is a whole lot easier if your goals are specific. I will go on to suggest – perhaps the most useful feedback you can get is to keep validating whether your roadmap itself is useful anymore, compared to just validating individual goals.
  3. Person getting feedback needs to know that ultimate responsibility of acting on feedback is with them, and not the giver. Its your life and career – no one will take care of it like you !
  4. Person giving the feedback should be comfortable in making the feedback specific or say clearly that they don’t have any good ideas this time. Ideally make introductions to someone else who can help if appropriate
  5. Person asking for feedback should resist the urge to defend themselves to the extent they can. By all means ask clarifying questions though to understand the perspective that is offered to you. As I heard in the Netflix series “The crown” where the Queen mother tells Queen Elizabeth “It takes every ounce of energy not to say or do anything” 🙂 . I know – I have bit my tongue a lot when I have heard candid feedback, and almost every single time it has helped me realize how I was fooling myself.
  6. Rather than defend, I prefer asking for feedback from multiple people and look at it as holistically as you can. If you ask only one person all the time – you are just inheriting their bias. One way that has worked for me in the past is to ask someone close to what I do for feedback ( lets say my immediate manager) , and another one who I know has experience in what I do ( say a senior executive in another company or another part of my company).
  7. If you decide to not follow the advice you got – try to explain it to the person whom you got feedback from. This is best done if you both set up expectations upfront. This is harder than it sounds – but if you don’t do this, there is no sense in asking the same person again. They will tune out at some point. Over time, my own way of asking for feedback is to create a set of options and ask for advice on which one I should choose and why. I am always open to a new option that I might get from this exercise.
  8. While persevering at something is laudable – the pragmatic way to look at it is that not everyone is good at everything. If after receiving feedback and acting on it for a while does not seem to make you good at it – you absolutely should question both the source of feedback, as well as your own ability to improve.
  9. Always have a realistic plan B. Not all ideas pan out even if you do all the right things (including acting on feedback). If you don’t have a plan B, you are bound to be driven by fear of failure and that is not always a good motivator. Best time to develop a plan B is when plan A is working well.
  10. Never be afraid to have a strong point of view. Your ability and necessity to take feedback should not be confused with the need to have your own strong opinions. Just be willing to stand corrected

 

So you want to join a large company , eh ?


In my couple of decades in corporate world – I have done stints at both large and small companies . I have also hired a lot of people over the years and watched their careers in those companies . These days when I hire – a lot of applicants tell me “it’s a really large company and that worries me” . So here is an attempt to provide some color commentary on this large company thing to help you think through .


Why do you want to work for a large company at all ? 

The truth is that while the company is large – YOU are probably going to be working in a team that is not that large . The better questions to ask is about the team you will be a part of . If you don’t like the team’s mission or the people in it – walk away and don’t look back .

Large companies mostly do things at larger scale . What they occasionally lack in speed , they make up in scale . Scale comes in many flavors and not everyone can deal with scale very well . For example – in last 5 years , I had to run portfolios that were an order of magnitude larger than previous ones . I had to unlearn and relearn a lot to make it work and it was not easy .  I have seen this work both ways – some people feel stifled at smaller companies because they want to change the world and they can’t find an opportunity to do so where they are . Some others at large companies beat their heads against a wall because they can’t move their ideas at the speed they want despite having access to vast resources . Choose wisely ! 

I often get asked “wouldn’t larger companies be really political?” . My answer “absolutely – but not any worse than smaller companies”. Politics is everywhere and you need to learn to live with it and navigate it . Also what is politics for you will be routine for someone else . Don’t sweat too much on that front . My own experience with small companies as an employee – which obviously is not a valid sample – is that favoritism and other political shenanigans are alive and well there , and more magnified because of smaller number of people . 

Another common question is “wouldn’t I be lost in this big ocean?” . And my answer is “yes – unless you show real results”. Large organizations are unwieldy to manage and hence get matrix management structures  . It is very easy to get lost in the system and it’s no fun to work that way . BUT – if you are good at what you do , and can show real results , the system favors you by design and you will get noticed quite quickly . If you are average – you will be the tree that fell in the forest that no one ever heard . So if you are not sure of your abilities to consistently deliver above average results , and if it’s important for you to get recognized – you should rethink the idea of joining a large company . 

Here is another one “I have heard the only way to succeed is to have a godfather in the higher ranks”. Well – having a god father certainly doesn’t hurt . But your real question is how do you get one ? Bosses like team members who make them successful . When you see a top executive giving special attention to someone – don’t just assume sinister things are at play or that the employee is sucking up . While those things all happen from time to time – the majority of cases , that employee had gone above and beyond in making their boss successful and is just getting noticed for good work . Also – sucking up to the boss is rarely a sustainable strategy . A VP of sales cannot “hide” a poor performing director of sales for very long. Unfortunately in very large teams where metrics are not clear – you may run into these bad scenarios . I have witnessed it a few times in large engineering and marketing teams . 

Large companies are often blamed as slow and bureaucratic . There is absolutely merit in that allegation . However , it has a good side too . Large companies are predictable in the sense that they rely on policies and procedures a lot . The policies themselves might be terrible and outdated – but you know what they are upfront . Also – if you run into problems like say a bad performance appraisal , or a commission dispute, you can be rest assured that there is a well defined process to rectify that and at least in my personal knowledge – it mostly favors employees . There are exceptions and those usually get the most publicity . 

One last point before this flight lands – the reason the large companies want to hire you usually is because they think there is something special about you that they value . They are not really looking for one more of what they already have usually . So find that out while you assess your future employer – if you think you have to morph into something you are not , this might not work out well for you or the company despite all the money and titles . I learned this lesson the hard way and hopefully you don’t have to 🙂

Making workflows sexy again with machine learning 


Since I grew up in ERP space , workflows are near and dear to my heart . I have set up a lot of workflows myself and I have been subject to the tyranny of bad workflows a lot too . Over time “collaboration” became a thing but classic workflows still largely rule our work life . The first time I directly set up a workflow was for a purchase order scenario in the late 90s – and I remember the client VP took me and a colleague to a fancy dinner to thank us . It solved the biggest pain for him in routine business and for two young consultants – that was like winning backstage tickets to a rock concert 🙂


So why do people use workflows ? The “useful” reason is that some decisions are usually complex and can’t be taken by one person – because of skills , legal and other reasons . There are many “useless” realms too . What is not talked about often in polite company is that lack of trust in fellow human beings is a big reason for the zillion workflows we all live with . 

I have several friends who specialize in workflow and collaboration systems – and they take great care of their clients in setting up the most efficient and effective workflows . What doesn’t always happen is that life changes often, but workflows don’t mostly change with it . And this can lead to comic and tragic and tragi-comic situations !

For example – Lets say there is an executive who runs a business, which has annual revenue that has a lot of zeros on the right side . But if she damages her phone and needs a new phone , she will need approval from her manager to get one . 

If a company trusts her to handle millions of dollars worth of business , shouldn’t she have an automatic approval for a phone ? Sure she does – and one call to the CIO can probably get this workflow fixed right away for her and everyone like her in the company . But it’s not just her – what if this is a non executive employee who has a critical job function like door to door delivery where the ability to reach a customer by phone  is paramount ? Sure he needs it too – and another call to IT (but this time from an upline manager in escalation mode ) can fix that problem quickly as well . But how many variations can happen in a workflow before it reaches the “this crap cannot be sustained” mode ? It takes very little time and I have lived through that nightmare a few times when I was a young consultant . And however carefully we craft the design of workflows – we won’t be able to predetermine all options that become necessary across enterprise as market evolves and business adapt to keep up .

It’s probably never going to get fixed completely – but machine learning can help solve a lot of these painful problems . Even if an automatic fix is probably hard, given legal and financial policies don’t move at the speed of innovation in STEM, we can make a tangible impact with meaningful insights .

The data about existing workflows is easy to get . That is enough information to get patterns for an algorithm to start on . Then it’s a matter of introducing other data sets and see what we can learn – like say weather , sales data , budgets etc . In our example of the executive – an algorithm that learns that there is a huge business impact if she loses her phone , it can trigger an order automatically . This is a much more sustainable way than a deterministic “if exec , then auto approve” rule . Why ? Say the same exec moves to a non P&L job and has a desktop where she has access all day while a new phone gets ordered . 

Humans cannot keep track of all the workflows that are set up over time  . No one needs an extra notification or email if they can help it . So machine learning can also be used to keep track of how the workflow landscape evolves over a period of time and suggest meaningful ideas to the workflow admin on options to optimize . 

If an hour a week gets saved for a given employee by eliminating useless workflows and making existing ones smarter , that is more than a week’s vacation that you can give that person at no extra cost in productivity . How cool will that be ? 

There are really no data scientists in the wild ! 


There are statisticians , there are mathematicians , there are engineers , there are machine learning programmers , and  there are many other types of experts out there – but there are really no data scientists out there in the wild ! What exists are data science teams and many are generally awesome . That is my conclusion after trying really hard to become a data scientist myself over the last few months . I am not giving up quite yet – but I am at a stage where I need to express my opinion on the matter for what its worth 🙂

2_221863

I thought I had good odds to be a decent data scientist, at least on paper. I think I am a good programmer and while I still think in C when I am coding , I can work on R and Python without sweating it too much . I am an engineer and have a degree in business – and I was convinced that I have more than an elementary capability in math and stats . I can do most data engineering work to get bad data into a shape that a machine can crunch. I spent a lot of time in BI – which made me believe I can visualize data really well. And so on . Yet , I didn’t become a data scientist despite my honest efforts to become one, and I think I now know why .

Between my engineering/business background a couple of decades in consulting – three things come naturally to me when I am faced with solving problems

1. The classic MECE approach

2. Thinking about it from the client view and working back to what I can do

3. Trying to get to a solution from first principles so that I trust the output

On the flip side, when I cannot do a good job on any of these three things, I get extremely frustrated. And in this effort to become a data scientist, I stumbled on all three. I also am close to questioning the idea of calling this domain as data science . It has more of an art feel to it – its like a half way point of an architect and an engineer, a bit weird. This could be an emotional response, so I am not going to make a fuss about it in this post.

As I played with it for a while – I understood that a few things need to come together for data science to work effectively for my clients, not necessarily in the linear fashion I call them out below.

  1. Define a problem in a way that it can be solved – some kind of designer/consultant type skill which I am generally good at, I thought. Turns out you just keep redefining the problem as you learn more.
  2. Create an abstraction – what programmers call “logic” or “algorithm” , and what math geeks call “model” . This needs a lot of “rinse and repeat” as I figured. I could have saved a lot of trouble if I started plotting data in simple dimensions first – a lesson I won’t ever forget.
  3. Find, clean and assemble data to feed into the model – the data engineering skill, and it becomes a challenge when data is really big. Analyzing data makes you wonder about your sampling strategy throughout. There are always gaps and it will make you say “maybe” or “it depends” as the answer to most questions.
  4. Figure out your model is crap, and explore alternatives endlessly. I realized I had forgotten how common substitutions worked in integral calculus and it took a lot of googling to get me comfortable on a first principles basis that what I am doing was sensible math. On the bright side my linear algebra skills are still sharp – but clearly that is not enough.
  5. Figure out what is worse – false negatives or false positives, and have a strategy to not have too many and how to explain the few you will always get. This needs extremely good domain/industry knowledge and the kind of assumptions you make can be comical when you run it by a real expert
  6. Finally – you figure out a half decent solution, fully knowing you can’t be really sure. At this point – you need to figure out a way to tell the story, usually with visualization. Voila – your entire perspective on how to tell a story with data will change quickly. I always loved D3, but now we are soul mates.

It is nearly impossible for one human being to be great at all these things – the best case is that you get to be really good at one or two, and have a solid appreciation of the rest. In other words – a bunch of such experts in these areas together can be brought together to form a great data science team. But it is just impossible to have one person have all these skills and hence be called a data scientist.

I also feel I should express my “amusement” about machine learning on two aspects before I end this rant.

  1. Depending on whose book you read, or who you talk to – you will think machine learning has two distinct flavors. One is a math flavor, and the other is a programming flavor. I have more developer friends than math geek friends – so I mostly got a math flavored “black box” answer every time I had that conversation. But the books I studied were mostly written by stats majors.
  2. The fact that a model is the right one does not mean that it performs well in production. You can sample ( I am staying away from my endless fights with bias, even for “simple” cases) and take smaller data sets to make your model work . But then you get the idea of running your logic against big hairy data – and suddenly you realize that your “black box” algorithms don’t all scale to work in parallel mode. I am now stuck in a debate with myself on whether a code rewrite , or a different math approach is better to crunch all the data I want.

Its clear that stats majors and CS majors should really talk more and not let me be the one worrying about these kinds of problems . I am happy to buy the pizza and beer for you folks 🙂

PS : my dear friend Sameer who is the chief of Kahuna , showed this blog to his data science leader Andrew – and here is his  counterpoint . You should absolutely read this too – debates and strong opinions are good things !

When machines think on our behalf !


I don’t really think machines will displace humans in significant numbers for a long time – but I do think we have an interesting time ahead of us where we let machines think on our behalf quite a bit . 

Every company out there has rebranded themselves to an AI company . The first generation of this is broadly of two categories 

1. Telling an AI system what we want to do – order a coffee , close the curtain – pretty much call an available API to do something 

2. Use AI to learn and do something better – like switching carpet bombing marketing campaigns and target better 

But this is just a temporary phase. Why do you want to ask your AI wizards to order coffee for your home – isn’t it better to let the machine reorder coffee when it gets to some level ? Should it even ask you to confirm these kinds of routine activities or just do it without asking ? About half the things I need routinely -I am totally cool with having a machine do it without asking me anything , especially about coffee . I get mad at myself when I forget to pick up coffee and I don’t have much left in the kitchen when I need my coffee . I am sure I am not the only one who is ready to offload some routine activity to machines .

So this poses some interesting challenges  like- if my AI system is the one ordering groceries for me without my input , how do other coffee vendors ever get my business ? 

My wife already thinks I spend way too much on coffee . So she maybe able to tell the AI system to limit my purchases to say $50 a month . So now my AI thingy needs to be coupon shopping and stuff to stay within budget – but that is easy, machines can do this math stuff  better than us anyway . 

This makes me wonder about what is the future of marketing itself ?

Simple – brands stop marketing to me and instead they ( as in their AI systems ) will market to AI systems ! And Brands will do whatever they can to convince my AI system to feed me their coffee first to increase their chance of my business being a reorder situation !

Well , guess what – this means we are in the “my AI is smarter than your AI” world at that point . The bright side – email spam reduces significantly for me as a human , and I have some more time on my hands . 

But this is not without its share of dilemmas too – for example , what if the AI provider for me and the coffee company are one and the same , or if they are two companies that share my data ? Am I going to be put in a situation where I am negotiating against myself ? So we do need some clear guidelines established on ethics , legality, security  and even morality before we get there to dealing with this problem . 

We have a good grip on what happens when AI does smart stuff when humans deal with it – like customer service , sales etc . But the thing that excites me the most is when both sides of a transaction are AI systems . I am betting it won’t take even 5 years for us to see this mainstream . Are you ready ?

Future of Software Development 


There are so many angles to this topic – and this is my third attempt in three days to organize my thoughts around it . The trouble is not that I don’t have enough ideas – it is that most ideas seem to contradict each other when I read them again .  Let’s see if the third time truly is the charm 🙂


1. Everyone will NOT  be a (meaningful) programmer in future 

I know that is not a popular position to take today – but that is where my mind is at now . We will need to water down the definition of coding significantly to make “everyone is a coder” be a true statement . If I can change the tires and oil of a car  , and program the infotainment system – should I be called an automotive engineer ? That is “roughly” how “everyone will be a coder” sounds to me now . 

Don’t get me wrong – I do think that understanding how code works is important for everyone in future . That doesn’t translate to everyone coding , at least in traditional sense . Very complex applications exist today in MS Excel – created by business analysts who are not traditional programmers . If we change the definition of coding to include that kind of development – I can buy into “everyone will be a coder”. The closer statement – though less sexy – would be “everyone will be a designer or modeler” !

2.   More code will be destructed than created 

World around us is changing extremely fast and that means we need lots of newer kind of applications . But the pace of change is such that no application can satisfy a customer for any length of time . Building better components and better orchestration mechanisms are the only way to deal with it . Neither concept is new – but the execution will kick into a higher gear . API designs will need a lot more flexibility than we are used to 

3. Performance will trump simplicity 

By simplicity – I mean what “humans” think of as “simple”, not machines . Code for tomorrow will be used more for machine to machine communication than for machine to human – by orders of magnitude . Creation of code itself might not need a lot of human help for that matter . And while maintainability and human readability are important today , it might get trumped by the need for extreme performance tomorrow  . For example – if both ends of an interface are machines , why would they need to communicate in text and pay for the overhead of things like XML/JSON tags that need to be converted to binary and back again to text ? 

4. You won’t need as much code in future 

A lot of code is written today because a human does all thinking and tells computers what to do in very prescriptive ways with conditions and loops and all that. When computers get to “general AI” – they will learn to think and adapt like humans – and won’t need step by step instructions to do what they do today . Less code will do a lot more than a lot of code does for us today . We may be decades away at most – we are not centuries away from that stage . Software will eat the world now , AI will eat software tomorrow 🙂

5. Software offshoring/outsourcing  will not be for development or maintenance – it will be for training 

It’s already possible for machines to learn from vast amounts of .  Some time in far future , machines will self train too . Till then – and that’s at least a decade or more – humans will need to train machines on data . And that will need to make use of local knowledge , labor arbitrage etc and hence will be an ideal candidate for offshoring and outsourcing ! 

6. Community of developers will be the only thing that matters  

Well – that is already true, isn’t it . I have forgotten the last time I have checked official documentation or real books to learn anything . I google or search on stack overflow to get most of what I need . I am not a full time developer – but looking at the communities that help me , I am sure full time developers do what I do , a lot more than I do 🙂 . A better way of mining this treasure trove of information is the need of the hour to get significantly more engineering productivity. 

7. More and more use of biological sensors 

Human bodies and brains are the ultimate computers and we are some ways away from mimicking human thought . In near future I expect simple sensors for all kinds of chemical and biological stuff ( how cool would smell be as an input , like touch is today ) that provide input to and also dictate how code should work . Text is fast becoming the most boring part of data any way 🙂

8. We haven’t even scratched the surface of parallelism 

What we call as massively parallel today in all probability will be amusing and funny to tomorrow’s programmers . The over heads of making parallelization work today is pretty high – and that will go away soon. A part of the problem is also that majority of developers don’t think of parallelism when they design code . I guess the near term solution will be for more primitives in popular programming languages (like for data access) to have built in parallelism . Note to self : get better at functional programming in 2017

9. Ethics and Privacy become core to all development 

A few things are happening together now

a) data is exploding and we are leaving our digital finger prints everywhere 

b) applications won’t stay around long enough to have ethics and privacy as a “future release” issue to be fixed

c) more and more software affects humans , but is controlled by machines with little human input 

d) access to information is (and will be ) universal – which means bad guys and good guys can both use it for what they want 

e) legal systems won’t ever catch-up with the pace of tech innovation 

And that means – ethics , privacy etc need to be core principles of tomorrow’s software . It cannot be “in pockets” as it happens today. And the education on this topic needs to be pushed down to even the earliest levels of schools. 

9 is really not a conventional number of bullets for a list – but given there won’t be anything conventional about the future of software development , I think now would be a good time for me to stop this list . Feel free to add , edit and challenge in the comments – I look forward to it .

Happy 2017 everyone!