Big data and systems that need to go around in circles

I had a very interesting conversation with my seat mate in the flight to SFO early today . He works as a manager at an airline, and grew up the ranks . Currently he is exploring ways of “learning faster” given his team was having a hard time with baggage handling and they want to fix it like yesterday .

He had a big data book in his hand – which is why I started the conversation with him , and I am glad I did .

Here is his situation in a nut shell. He can get pretty much any report he wants on any data that affects his work . It is not real time , and that is fine by him . He needs printed reports that his team can carry around in their pockets and check periodically . Essentially real time is not an issue and neither is mobility .

However , when a problem occurs – he has to trust his memory to remember when it happened the last time and what he did to solve it . None of his reports give that information to him .

In his words – this makes him go in circles , while his information systems go in a straight line . He wants a solution that will go in circles with him in perfect sync . Why circles ? Because he says almost every problem he encounters is a repeated problem that has been solved in good and bad ways before . He doesn’t have the time to sit down and document solutions in a word document given he has a very busy job . His question is “why can’t the system figure it out that it is a repeated problem and tell me everything that has been done before and what worked and what did not ? “.

I asked him how he ended up with the big data book . He said his wife is a college librarian , and she told him that “big data” is the fancy technology that solves all kinds of complex problems and found him this book . He also said it is the first time he is reading a book other than an airline manual in more than a decade .

Eye opening conversation for me – to say the least . I wonder how many such employees and managers exist in the world today , where the IT systems cannot help them the way they want to solve problems in real life . This is not just a big data problem – the whole IT industry needs a reality check .

At the very least – it is a good proof point that yesterday’s solutions won’t solve today’s and tomorrow’s problems . Time for us to stop thinking in abstract ways with lot of jargon thrown in – and just go help solve real problems .


Published by Vijay Vijayasankar

Son/Husband/Dad/Dog Lover/Engineer. Follow me on twitter @vijayasankarv. These blogs are all my personal views - and not in way related to my employer or past employers

9 thoughts on “Big data and systems that need to go around in circles

  1. Vijay,
    Given the brief summary, with absolutely no details (and the devil is ALWAYS in the detail) , I would suggest that Airline guy needs some kind of COTS Enterprise Help Desk tool; so he can store all his incidents AND the responses to them, with indicators of success, then have a smart (oh OK, call it “Big Data” since we want to charge a premium for the product) search engine over the top.

    PS is this comic sans I see before me ? FTW


  2. You are right yeserday’s issues will need tomorrow tools to be solved and learned from. Watson? May be?

    I love your blog. We miss you at Big Blue.


  3. time to think about the “solution” in a very different way and not to think about the “technical solution” first. Had a very interesting conversation here in South Africa with a retailer and very interesting approach on how they solved their problem (did include “big data”).


  4. Hi Vijay – nice read. Pre-defined / straight-line / SVOT BI will likely give us 50% adoption today, and maybe not even 25% in 5 years. As you know self-service tools are developing, but perhaps the biggest development need is with analysts themselves. Some analysts need to provide [knowing where to go dig], making the right decision based on data. SVOT probably a dying breed…..maybe SVOT is becoming SPOA [smartest plan of action :)]..


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: