Making Business Travel Bearable 

I had a huge fascination for air travel as a kid . My dad traveled frequently ( at least once a month) on work , and I had some rich relatives who flew regularly to America and Europe for family vacations . They would tell me great stories of their travels and bring me back little goodies – like a can of Coke ( which was not available in India at that time ) My resolve became stronger over teenage years that I need a job that let me fly frequently all over the world .

My wish was granted . Fast forward 20 years since I graduated – and millions of miles behind me – all I can say is WHAT THE **** WAS I THINKING ???

By conventional terms for me  – 2016 so far has been a “light travel” year. And today I saw that I had already qualified (yet again) for the highest tier of my Airline frequent flier program for following year , as well as for the hotel chain I use . I am sure more than half my colleagues had those same emails 6 months ago. The last time I felt good about getting such emails was the first time I got it and I can’t even remember which year that was . 

Those who know me can vouch that repetition bores the heck out of me . That is why despite the extreme dislike for travel – I am still in the consulting business . I get the variety of challenges that keep me motivated every day I wake up . If clients for any reason choose not to challenge me – I am sure my employer will pick up the slack and throw a few challenges my way …. you know , just to keep me sharp🙂

Effective and efficient travel is a life skill for anyone in this business . I keep picking up new skills and make little tweaks as I conquer the sky miles . Here are a few that I think are my basics – with no claims that it will work for you too🙂

1. Be a minimalist about everything you pack

If you need 3 shirts for the trip and want 5 , stick to 3 . If you run into an emergency – buy a new one or hit the laundry. In 20 years I have had to do that maybe three times . 

A big part of traveling comfortably is to pick a great bag to be your constant companion . Although not fashionable ( and un-executive like according to my mom)  – I use a backpack for my laptop and books , and one stroller for my clothes be it a one day trip or a 5 day trip. Many friends choose multiple bags to suit length of travel. 

Many consultants go the same city every week for several months . When I had that kind of travel,  I used to leave some dailybuse stuff at my regular hotel (or under my desk in a bag) to avoid carrying it . 

I dress for comfort . Unless the client needs me to – I won’t wear a suit and tie . Comfortable shoes that also look decent is probably the best investment I make on shopping front.

2. Try as hard as you can to not checkin luggage 

You cannot buy time . Even when you have nothing better to do – it’s better spent reading a book , or (and?) drinking a beer at the airport bar than standing in line to check in your bag and then waiting to pick it up at the destination . 

3. Ignore the pain and earn top tier loyalty levels at airlines and hotels ( optionally car rentals too)

Pick an airline that works for 80% of your travel and stick to them till your breaking point . I have come dangerously close to getting out of it a few times but I haven’t taken the final step yet . As you travel more – upgrades become your best friend . And the bonus points help a lot . I forgot the last time I paid for a hotel or airline when I took a family vacation . I have stopped renting cars almost fully – mostly because of the nature of my current job . I stick to cabs and uber now and it works splendidly . But in many cities – rental cars still make sense . 

4. Choose a credit card wisely for travel 

Those points help with vacation . Some will also make airline club memberships cheaper . Balance it against annual fees and pick one up and use it regularly . Always keep a backup card too – Murphy is always watching you ! 

5. Enroll in TSA-Pre and Global Entry

Although those lines are getting longer compared to when it got introduced – for the most part it’s easier to get through them than the regular frequent flier lines. I have a few friends who don’t enrol due to privacy concerns – and it makes a great beer conversation after their tired selves join me at the bar after a two hour journey through the regular line . For me this is the best $100 a consultant can spend every 4 years 

6. Minimize the need to travel 

It’s really hard to not travel at all for business – except in a few cases ( say where you have extremely good skills and are in a hot market without a lot of competition ). But all of us can minimize travel by good use of phone , email , social media etc . I often choose to travel even if I can get work done through electronic media – mostly because human-to-human interaction has greater quality . 

7. Build a time saving routine 

Routine keeps us sharp and reduces variance and hence reduces risk. I am on autopilot for several things when I travel . Be it packing , booking , driving , eating or exercising – build a routine and it will help tremendously over time . For example – I know it takes 12 minutes for me from gate to exit at PHX airport . So I book my uber ride to perfectly match when I am stepping out and have zero wait 

8. Strike conversations every chance you get 

I can check email later – but if I can strike a conversation with a stranger , I will do it . These are not long boring talks and if the other person is not interested I move on quickly. But I have learned a lot from these conversations – especially from cab drivers across the world . When I have hypothesis to test on social and political issues – nothing beats airport and hotel bars . Over the years – I have even built the foundation of a few business relationships this way . Funny enough – I have met more fellow IBMers in airports than at any other place🙂

9. Never eat alone 

I try really hard to not eat alone when I am on the road . After work conversations over food and beverages are the best way to know your customers and colleagues . It has the Magic effect of building solid relationships over time . And it keeps the boredom away 

10. Call home 

I usually call in evenings to catch-up with my wife and daughter . And I call my parents and in-laws from cabs on my way from airports . And I post updates during the day on Facebook so that they know what I am up to . What I don’t do well is to restrict the audience to just my wife , my sister , my mom etc – but that is just my laziness . 

11. Music , reading , writing , exercise ..

I use my phone for listening to music , to read and also to write emails and blogs . I always prefer being agile over being elegant and formal and it mostly works for me . I am not big into working out – so I use airports and offices to walk fast , climb stairs and so on to make up for it . I don’t always succeed . 

What are your tips ?

The confusing terminology around AI 

Few weeks ago , I attended our college reunion in India . Hard to believe 20 years have passed since I got a piece of paper mailed to me that I am a bonafide mechanical engineer . In these twenty years – the “work” conversation I have had the most consistently , multiple times every year , is the difference between terms like data, reporting , BI , analytics etc . After about ten years of fighting the good fight – I gave up and reconciled in my mind that it doesn’t matter what you call it as long as it solves problems for my clients . 

Over these past two decades , there was always a movement in IT to show that careful analysis of data will help the business take better decisions . As a result – a lot of improvements happened in both data management as well as analytics . As the technology got more and more sophisticated – the terms we use to describe it got more and more confusing too . At this point – people use AI , Cognitive , machine learning , neural networks and deep learning etc interchangeably . 

The amount of confusion this generates is not trivial . So now – not only do I get to explain the old ” analytics vs reporting vs BI ” , I also get to spend countless hours explaining nuances between ” cognitive vs AI vs…..” . 

If we need one umbrella term – I would stick to ” Artificial Intelligence” as that term . AI was a term coined by the late Prof McCarthy over 60 years ago . Over the past few years – led by IBM, several people have started using Cognitive computing as an umbrella term too . 

I have asked around for and read a lot of definitions for AI – and it’s hard to find any consensus . The way I look at it is AI is the discipline that is today about doing things only humans could do in past , and one that is aiming for a tomorrow where computers also think like humans do . 

A friend of mine and I usually joke around AI just being a series of nested if-else statements , just that it is written in Python🙂

That joke is not fully off base . The traditional approach has been to model the thing we want to analyze and then ask questions of it . Intelligence comes from the brilliance of the designers – not really “artificial” . The challenge is of course , things change over time . A better approach probably is to model how humans think – so that even if things change , answers can still be found . Just that it is contrary to how we ( or most of us ) have learned all this while to design and code . This is the concept ( or more precisely just my understanding of ) behind “deep learning” . 

Is “Supervised” learning really much different from the maintenance and enhancement aspect of traditional programming and hence is that really AI ? I am conflicted on this – mostly because human learning also needs supervision in many cases . 

Some of the confusion can be avoided by thinking of today’s world of AI as “narrow” intelligence and the vision for tomorrow’s world as “general” intelligence . Machine learning – perhaps the most visible part of AI today – is mostly used today (at least from my limited point of view) on the “narrow” use cases . The easiest way to think of it for me is that rather than make continuous code changes , the algorithm keeps up with changes by detecting patterns as it gets access to more and more data 

The challenge for me with the term AI is the definition of “artificial”. I think the expectation for “artificial” is a lot higher than say “augmented”. And that is perhaps why “cognitive” doesn’t get as much push back as it should . 

Another challenge to move from old world to the AI world is our fascination for precision . Most decisions only need directionally correct information and options – they don’t need precision . But that is not something that a lot of people will agree without significant pushback . AI type projects need a lot of expectation setting and some education on the basics of probability . I had to dust off a few of my statistics books before I could talk semi intelligently to my clients . 

As machines get smarter and the primary communication becomes mostly machine to machine – perhaps machine learning doesn’t need to try to think like humans anymore. Whether it’s going to be more complex or less complex is anybody’s guess . All I am sure of is that we won’t be spared of some more jargon🙂

Before I sign off – here is a shout out to my friends in the world of hardware . Without the extreme speed of innovation in the hardware world, AI ( and old world computing too) would never have had a chance to get on the fast track . Look at how much the world of AI has changed since GPUs became mainstream as an example . The last two or three years have seen more progress than the decades before it. It’s gonna be a wild ride  

Trusted Advisor 

Since the day I walked into business school couple of decades ago , I have been told that the definition of success would be for me to become a trusted advisor to my clients . It sounded logical , and I accepted it as an absolute truth and never really thought about it too much . Over time, I dutifully convinced others who asked me for career advice that they too should become trusted advisors for their clients . Thankfully – no one asked me exactly what I meant and accepted it as an absolute truth . 

That changed recently . A new hire didn’t take me on my word and pushed me to explain more on what I meant . I gave some half baked explanation and a few examples and ended that conversation – but I immediately knew I had failed . I don’t like to fail ( fast or slow) – and hence started thinking about why I couldn’t answer in a crisp manner . 

To begin to comprehend what being a trusted advisor to a client is – we perhaps should understand what “client” means . Someone who buys from you is your customer . So all clients are customers , but the reverse is not true . A customer who continues to buy from you for a long time , because they recognize the value you bring beyond any given transaction, and who in turn proactively adds value to you even if they are not buying anything from you ,  is a client  .  

Another way to think about this is – when your customer becomes a client , you become their trusted advisor as opposed to just a vendor !

My career at IBM  started as a BI consultant at a large semiconductor company . After two years of working there – I was bored out of my wits and asked the account partner to reassign me . He declined , and instead took me out to dinner . During dinner, I learned that he himself came into that account as a fresh hire out of college and stayed there till he made partner . He had his fair share of battle scars – certainly it wasn’t a bed of roses . On the flip side – he was one of the youngest partners in the firm too. 

I quickly learned that nurturing a relation I already had for two years is infinitely better than building a new one from scratch every few months . To cut the story short – I bought a house next to the client offices to be close to them , doubled down on finding and solving problems for them and at one point – I realized I know their business better than most of their own employees . I was still personally involved with that account when I made partner in the firm too, just like my boss had ( though I had to take on additional customers and turn them into clients to prove that I am not a one trick pony) .

There is an important aspect here that we often overlook – being a trusted advisor is less about the client company than about people in that company . Business is always done between people – not between companies ! All relationships – irrespective of levels, roles and titles – matter . People (both you and client employees ) move around roles and employers , and the trust you built with them moves with them . 

By no means am I suggesting that you should put all your eggs in one basket . There are times when a client has no reason to buy from you directly and you might start to doubt whether it’s worth spending any more time when you don’t have a near term pipeline . But if you have built a strong relationship – they could be a strong reference for you in another part of the company , or for another company . There is nothing more valuable in winning business than a strong reference from a happy client ! 

While I can’t possibly pinpoint when a customer has turned into a client for me in past – there are some characteristics that I find in common across all of them . They have asked me (and I have asked them) for career advice . We have known each other’s families well and have been invited for meals and drinks to each other’s homes . Calls and emails to each other are promptly returned . And finally – neither side expects the other to BS . 

A parting thought – While it’s really hard to become a trusted advisor , it’s real easy to lose that status . Trust is based on respect and transparency . When either side takes undue advantage – trust breaks and there is usually no superglue that will fix it back seamlessly . 

Reverie ’96 : Old met new, and got impressed !

I was part of the class of 96 of TKM College of Engineering – one of 90+ newly minted Mechanical Engineers to walk out of the iconic red building . And in the next 20 years , I had not been back in college . That changed on Friday , August 22nd – when my parents dropped me back at the college , which looked a lot lighter in color but much improved . To say I was overpowered with Nostalgia would be an understatement . 

The first stop naturally was my most favorite spot there – the college canteen . It did not look anything like what we had in our days , but what the heck – I walked towards it . Next thing I know , a familiar figure started waving frantically at me from one of the tables to speed up . There he was – Prof Nasar – my favorite teacher . He taught us automobile engineering (One of the very few subjects I was passionate about) , and was my guide for the final year project (which was to design a machine part for Hindustan Latex LTD for rolling up condoms in their manufacturing line without causing tears – I still remember the embarrassment of explaining it to the panel of teachers , and prof Nasar coming to the rescue of Ajith , Anup Nair and me) . 

I asked him if he remembered me – of course he did , including my name , who my parents where , where I lived , my love for dogs and many other little things . He was very proud of me and introduced me to several of his younger colleagues . 

Next on the agenda – after a quick lunch at the Beach hotel with several friends – was a “meet the students” session . The idea was for 5 of us to be in a panel in front of a couple of hundred final year students and impart our wisdom(?) and answer a few questions . Ganesh , Sindhu, Boby and Rejin formed the panel , which I moderated . We had a dozen or more from our former class join in the audience to provide some moral support.

I was convinced – remembering the extreme reluctance I had in attending these kinds of events on a Friday afternoon when I was a student – that no one will show up. But boy was I wrong ! The room was standing room only by the time we started . 

We kept the tone informal , and switched between Malayalam and English . It’s probably the first time in my life that I had a chance to speak in Malayalam in front of a big audience and I loved it . The students gave us their full attention too – and asked some excellent questions on our perspectives on how to prepare for their careers. 

A word on the panelists – what an amazing bunch ! 

Ganesh did his Grad school in US, worked in US and Europe and then became CEO of his own company . Boby worked at BPL straight out of college (one of the very few who bagged that opportunity I might add) , went to US on a work assignment , did her masters there and is now an engineering leader at Motorola . Sindhu taught engineering , and now is a top researcher in Siemens . Rejin – not surprisingly – is an entrepreneur, and has a cool robotics company. I am clearly the under achiever in this group🙂

A few questions from the students made us dog deep and search our souls for answers . Boby explained how she went through her own career progression , often as the “only woman in the room” . Sindhu explained how to focus on first principles to build a career in research . Ganesh explained the need to build a network  of contacts to get ahead in life . And Rejin brought forward the need to apply the theory we learn in class to make things – which fits his own fame as a master prototype builder in college ( ahem … Not always amusing some of  his teachers who wanted him to stay focused on winning the first rank ). 

When I studied in TKM , I had no such insights . I didn’t know who the alum were or how to find them . I saw first hand how much the students appreciated the candid answers from the panel . It also showed me first hand how important it is to give back to the next generation in a “pay it forward” fashion . 

Just as the day started , we finished off af the canteen . Every single one of us was full of enthusiasm and appreciation for our beloved college . Some even got invited to present again to the students during the week.

The man behind the new found focus on sustained alumni relations is Prof Sudhir, who was a year senior to me in college . He explained how it is now a well organized effort to establish a two way constant communication between the old students and us . 

Prof Ayub , who is now the principal of the college is a forward thinking academician . He is also a big fan of strong alumni relationships and frequently travels the world meeting alumni . It was a lot of fun being in the principal’s office after the panel to have a cup of tea and listening to his vision and plans . To say I was impressed will be an understatement .

Saturday , 23rd July, was the formal reunion day. Our chief guest was Shibu Baby John , who till recently was a state government minister . He is a fellow alum (10 years my senior) , and that too from Mechanical Engineering ! I never had a chance to be on the big stage while at college – and thoroughly enjoyed the opportunity to talk about the importance of paying it forward . And it was fantastic to hear from the minister , Prof Imtiaz , Prof Sudhir and our very own Ajith Varghese who was the college Union chairman in 1996 . 

Pretty quickly we figured out that there were 6 of us from the class of 96 in IBM – Nitin, Hemant, Jayaraj, Sunil , Teji and me . How cool is that ? We missed two – Jayaraj was in the middle of a project in Abudhabi and Teji in US . 

What an event this was – we had participation from every corner of the world . We picked up conversations from where we left off 20 years ago . It was fantastic to re-establish the old friendships .

None of this would have happened without the leadership of our friends Mahesh Nair , Krishnakumar , Boby Iyer and many others who herded the proverbial cats all over the globe  . Thank you – you are rock stars !

I can’t wait for the next reunion ! 

One year ago , this day 

I joined IBM …again , after nearly a 3 years gap, for my second innings !

Today morning , as I woke up in the morning in my hotel room in Asheville, NC – I reached for my iPhone to check mail . Then , as usual I quickly scanned Twitter and then Facebook . And there it was – FB alerted me that this was the day last year that I joined IBM , with a photo that I had posted from my orientation class in Herndon,VA . Talk about “digital transformation” being real🙂

I can’t believe a year has passed ! Time really flew by me . In this short time, I have already worked in three roles across sales , delivery and service line leadership . That is one reason I haven’t noticed the time – each role needed me to learn something new and that has kept me on my toes . That in a way surprises me . In my last two jobs – after a few months, I generally had a feeling that I have a grip on how to run the business. Once I came back to big blue – most days I get up thinking “there is so much more to learn”. My current passion is machine learning – and how I wish I spent more time listening to my statistics professor .  All of this comes with its own fair share of pressure , but for the most part – it’s pure fun!

Travel has not reduced significantly – but not as many international trips given my patch is North America now . I have travelled about 70K miles already this year, and probably will do another 80K in second half of the year . But I do get to spend some more quality time with family and have slowly gotten better about not being on email throughout weekend etc . Not sure if my family and fur kids think of it that way😉

One thing that helped me get settled quickly in my second innings was that I tapped into my network of mentors early and often. Even got a few new mentors along the way . I have also consciously decided to spend more time coaching our new hires and helping them get settled – and nothing is more gratifying seeing them succeed . It doesn’t hurt that I am blessed to have a fantastic team that makes me look good😉

Our team is very diverse now – across multiple dimensions . I wouldn’t have it any other way now – the business results prove how valuable it is to have a team that is diverse . I have some great partners, associate partners and senior managers in my team – but what makes my gang special is the sheer talent of the New college and grad school hires. These are our future leaders and I already think I will work for some of them before I retire . Absolute rockstars !

I have also had the chance to try a few new things in my job . I now interview candidates at all levels over coffee at the local Starbucks on Friday afternoons . It helps my recruiting team schedule better , and I get to observe them in a neutral setting which helps them be at ease mostly. We even do some joint coding or design to see if we can work together . I am also slowly getting comfortable with interviewing on video via hireview. 

Another experiment that is working out well is changing how my staff meeting works . We meet every Friday morning for an hour . For the first half hour we discuss regular business – pipeline , talent etc. For the second half hour – I invite an eminent external guest to address us. These are customer CXOs, analysts , executives from competitors , etc. These are informal sessions – they talk candidly on a topic and we get to ask questions . No PowerPoint , no recording . Just straight talk ! And it helps keep us grounded and minimize the echo chamber effect . 

There are plenty of areas to improve too – like delegation . At every order of magnitude of your portfolio, you have to relook at how you delegate . I have some ways to go there . The ability to better balance short term with long term is something I am learning from my boss now – and I am sure I will figure it out soon . 

A few things have suffered along the way . Reading list has grown , reading has not . Same thing with blogging . And I generally avoided conferences for most of last year – except two. Those are all in the bucket list to get better at .

Before I end this rather long blog – if I could only thank one person for keeping me productive in IBM, I will choose my executive assistant Thelma Reed (and Andrea Edwards who backs her up, and who is literally a clone ) . I don’t know how she does it all so gracefully and efficiently , but if she wasn’t there – I know I will be paralyzed just dealing with my calendar . Thanks for everything you do Thelma !

Can’t wait to see what is in store for future 

Digital Transformation – we are so far away, it’s scary !

A couple of months ago, my wife and I found a house that we really liked – nice neighborhood, close to our daughter’s school and so on. So we decided to put an offer and see if we could get that house.

Our realtor uses docusign – so the entire offer process was done online. And I started to think that this home buying exercise would be such a frictionless experience. Boy , was I wrong !

Lets start with the loan application.

We had enough savings to put a downpayment .  I have an extremely good credit score and fairly low Debt to income ratio. I have also had my checking, savings and credit card at the same bank for many years. I even had my past mortgage and line of credit with them . The bank would know everything that happens in my life – after all I have been in their top tier of loyalty program . I generally expected to walk into the bank’s office , sign some docs, share my tax info and W2 etc,  and expected a loan to get funded in a few weeks . 

I also wanted to take a home equity line of credit from the same bank and filed an online application for that .

Guess what – turns out , my bank doesn’t know anything about me . My line of credit application went into a black hole . Few weeks later , I called them and forced them to work on it . 90% of information they collected from me are things they readily have with them but didn’t seem to have access to ! 

The end of this process was even more weird . They sent me a letter saying I asked for $X and it is denied because my house appraised at a value that is slightly (by few thousand dollars ) lower than what’s needed for the line of credit . So rather than offer me a slightly lower credit line , they just denied it . So much for all the top loyalty tier and time spent on it . I am still shaking my head on that . Absolutely bizarre !

The actual loan application didn’t fare much better . And finally they came back with a loan that cost me more than what internet lenders offered . Plus they needed several more weeks to close than others . 

Cut to the chase – I went with another lender who offered a better rate and faster time to close . My bank just lost my business – they tried really hard to not win it . 

In hindsight , I should not have been surprised at all . A trained monkey looking at my checking account statements would know when my car lease would end . My bank was never once able to use that information to offer me a car loan . This doesn’t need complicated analysis – it’s just stupid . Clearly, you can’t fix stupid . 

My lender wanted to get a HELOC note for a line of credit I had paid off and closed in past. I  went to the bank only to be given a toll free number . After speaking to multiple people , I found a helpful lady who found the note for me . However she only had two options to send it to me – regular mail or fax . Email apparently is still alien technology for them. I had to call my wife to hook up a fax machine and get the document . My neck hurt from all the shaking .

So yesterday I went to the bank to get a cashiers check for the down payment . I used my debit card – which has my photo – to take the money out of my account . The teller then exclaims she cannot verify my signature because I had opened that particular account online without a paper form showing my signature . My debit card PIN and my photo on it , and the signature on my drivers license didn’t count . She had to get her manager , make me sign more paper before they could get me a cashiers check . Easily an hour lost for me , and three employees at the bank . I now have a bald patch on my head thanks to scratching my head since then till now . 

So that was the bank . Let’s move on to the actual paper work after the loan is secured. Barely 20% of the documents can be electronically signed . Rest needs trees killed and ink spread on paper . Emperically – redundant information to be filled by the title company is probably around 50% across all the forms I had to sign . Chances of making errors are everywhere , let alone the sheer time needed to fill all those forms . Even if “digital” can be ignored , why not at least streamline the paper forms ? 

A company like DocuSign can single handedly disrupt this terrible manual process and bring it to 2016 digital standards . They will have the gratitude of home buyers and sellers across the country – and their money !

These are not insurmountable problems even by technology capabilities of a few years ago – which is why I am all the more annoyed with this . I am not calling out the bank by name – but I am going to find the CEO’s email id and send this information . And then I will wait by my fax machine to hear if he can do something about it . 

The fascinating world of advanced analytics 

I had a lengthy conversation with an old client yesterday on his plans to start some cool new projects . That is what triggered this blog 

Most of you already know that I am not too big on categorizing this topic into predictive , prescriptive , cognitive and so on . I never cared about the old “is it analytics , is it BI , is it just reporting” debate either . All I care is whether data can be used to solve problems somehow . So when I say advanced analytics – all I mean is using data slightly more sophisticated than to report past performance . 

On the software side – volume is no longer as big a challenge as it once was . Storage ( and RAM and CPU and …) is getting cheaper and compression is getting better . It’s still not trivial ( for example – when a semiconductor chip is produced , one optical scan can produce 10 to 30 TB of data per wafer . Even at current prices , that is expensive to store all of it) . 

The hard parts are still velocity and variety . Everyone can eventually get to the same result – but competitive advantage is only for the first few who can see the result . Even within that set – only a small number can actually act of information quickly . Now if the raw data hits you really fast – there are real challenges . 

Software ( whether it is app or database)  is hardly optimized for read and write at the same time when the incoming data is variable . If you need to put a lot of data into your system at a high speed like say in some IOT scenarios – there are databases that are optimized for it . But those databases need extra fittings to make that data available to be analyzed in real time . There are others that can do sophisticated analysis , but they don’t always allow data to be put into it at the speed it arrives . Essentially a lot of compromises and data duplication are still daily struggles for many of us . Granted it is getting better – but it’s not there yet . 

In many of my customers – even after all the software puzzles are solved , we hit a wall on network bandwidth . Cloud is sexy and all – but every hop takes a toll . 

After we figure out the right way to put all the data into the places we need – then starts the analysis pieces . Between COTS and opensource , there is no shortage of software that can do the job . But the idea of democratized advanced analytics is still a distant dream .

There are many aspects to this problem 

1. There just isn’t enough talent who understand statistics ( err data science I mean)to begin with 

2. Generic data scientists won’t cut it . Sales data  from an automobile company cannot be analyzed the same way as sales data for an aerospace company . Industry knowledge is key . That makes it even harder to get the right people on the job – and hence you need teams that have data scientists and industry experts for foreseeable future . 

3. There is hardly any consistency in legal matters across the world . Now we also need lawyers in such teams to make this work in a way that no one goes to jail😉

4. Legal does not mean ethical always (there is a surprise) . So now we need an ethicist (such people do exist) to help answer some questions on what data and what analysis is ethical . Then you might need an MBA to figure out the solution with all the constraints applied😉

5. Even if you have all these people and all the right software , you still need to convince the customer that it’s a big production and it comes at a price , although for all the right reasons . Then you need to explain why it is not a good idea to get two data scientists from the body shop to create a model 

6. Even if the customer sees the value , and spends the money – after the team shows the model , it could look really simple and customer will again ask “why didn’t I just hire two junior data scientists to get it done ?” . (The sausage making ( fitting the models) is not fun – it usually is darn tiring grunt work- to watch unless you are a data scientist yourself )

7. Neither Models nor data stay static . Unforeseen things can come up and there might not be a way to predict meaningfully using past data and analysis . For example – lookup why Nate Silver could not predict Donald Trump becoming GOP nominee 

8. Every prediction comes with caveats – some trivial and some complex . Trivial ones can usually be ignored and an automatic action triggered ( like for example ABS kicking in a car when some conditions are met) . The complex ones need significant explanation – and that is not easy unless the recipient of the information understands some basic statistics . There are software vendors who claim their wares can make predictive analytics available to lay users . What they don’t do is explain what caveats apply when those users see results of their analysis . 

9. Like with everything else, things go wrong all the time in analysis too . Complex analysis is really difficult to debug today 

10. Even if all these challenges are over come , and you tell the customer there is a 90% chance of door 1 being the one to open to find the pot of gold , it could still be that door 2 was the right answer that time . So now you have to explain why that happens 

I can go on , but I am sure you get the rough idea already . If not – buy me a beer and I will give you some examples of real life situations I have dealt with🙂