And Vijay Says…

IOT has always fascinated me. As a young developer in the 90s, I made a living doing mostly integration work – making two computers talk somehow . And then I moved into the world of data management and analytics. IOT is one of those things that just combine all the good and bad of what I know from my data and integration background. It is of particular interest off late to me given its part of the portfolio I lead at IBM – so I have a practical need to understand it better and think about where it is headed.
When I think of IOT – like many others, the first thing I think of is mobile phones to get the idea clarified in my head. 

I started with “How does an app really work ?” . You do something with the app as a user -leading to some minimal processing that happens in the phone using its RAM and CPU and then some information gets sent to a server somewhere ( as in cloud) where it does the heavy lifting and then it sends  back simple instructions back to the phone. Rinse and repeat for every action . 

The computers in the data center and the processing power of the phone are all improving exponentially . What is not keeping pace is network bandwidth. Network is the biggest bottleneck today and with the explosion of data that we live with today – its going to choke even more tomorrow . This is already true for non IOT world – for example I know SAP Hana projects that could not go forward because of insufficient network capacity . 

That is just phones. What about all the other “smart” devices that have processing power ? Everything from toothbrushes to cars already have, or will have, significant processing power soon . And that is only going to increase over time. 

I don’t know for sure – but between our two cars, I am guessing there are couple of hundred processors sitting in my garage as I am typing this. And my cars are not Teslas,  which probably have many more. As “things” stop needing humans to do their “thing” (see what I did there cleverly ? )  – they will have more and more processing power and local memory. They will generate even more data that needs to be analyzed quickly . And “network” as we know it will be toast dealing with all the traffic .

Already machine generated data is many times bigger than human generated data. These “things” won’t rely on “small data” to work without humans – this will be the “real” big data challenges we need to solve. It won’t be simple text files that count as data – it will be things like sound and video. 

The volume, variety and velocity of data will become quite unmanageable if we wait for these things to ping the cloud , transfer data and get a result back. You don’t want a drone to crash into something because it could not reach its cloud source to exchange data and find out how to land. You get the general idea – things that need quick request/response generally cannot work like how the average phone app works today. Even the next gen of phone apps probably can’t work like how they work today !

So what is the big deal ? Well – the current thinking on systems architecture needs some refresh. Maybe a significant refresh !

I don’t think cloud will go away per se. Just that it will become more “hybrid” in nature. My purist cloud friends who hate the term hybrid cloud will probably yell and scream at me – but for now, I am going to take the heat on that 🙂

What I mean is – a lot of processing where latency is a critical factor – the “things” will become mini clouds themselves. I am sure there is (or will be) a better term for it – but till then “mini cloud” is the official technical term for me. 

These mini clouds might be very powerful – many CPUs/cores/storage/RAM etc – and should be able to do pretty much all real time processing, and also be capable of some point to point interfacing. 

For example if I ordered Sushi and my wife ordered a burger – and two drones are trying to both land in our front yard to deliver our orders – they should be able to talk to each other on who lands first and not wait for both to talk back and forth with their respective clouds. If there are only two drones – perhaps we can tweak existing infrastructure and design patterns to make this work. But what if there are two million drones over my city doing different things ? I don’t want to take cloud to a drone fight !

My other passion is AI and analytics . How do they play in the IOT world ? Well – they need to play in two places at least . A micro version needs to play in the “thing” (the mini cloud) and a “macro” version needs to play in the traditional cloud . And there might even be a third version that brokers information flow in the middle . 

The reason is straightforward  – things need to make autonomous decisions like a drone figuring out where to land given what it can sense . There might not be enough time to ping the cloud and get an answer and hence the mini cloud needs algorithms that help with the task at hand . But this way the drone never gets much better beyond what is already coded . For the drone to get better at its job – it needs to learn from every other drone , as well as some external data like weather etc . In some smaller scenarios I guess point to point interfaces can solve this problem . But at scale – this would mean the big cloud runs advanced algorithms ( probably AI types ) and what it figures out gets shared with all the drones that it serves . 

This is a non trivial problem . Just to reliably sync regular text databases in a distributed system makes life complicated for IT shops around the world today despite such techniques being available for a long time . Just getting one set of machine learning models going is a pain for many teams . IOT just makes it much harder problem to solve – especially considering different vendors might own what goes into the brains of a drone and what goes into the centralized cloud . 

Not to make it more complicated – there are all kinds of “quality of service” questions to be considered too like performance , security , disaster recovery, standards and so on . And of course there are the unknowns – like what happens if a brand new communication protocol comes up that eliminates the network bandwidth problem ? 

Not all scenarios need all the bells and whistles of the drone example I used to make my case . A smart toothbrush probably doesn’t need as much sophistication to reach its peak potential compared to a drone . What that means is that whatever is the future architecture we come up with – it needs to be “sliding scale” friendly . Otherwise at a minimum the economics might preclude scale and viability of the solution . 

Interesting world eh ? If this kind of work fascinates you , let me know – I am always hiring 🙂

 

 

 

 

Last year , this week , I was in Austin TX -meeting my new boss and forming a team and planning for what we wanted to accomplish in 2016 . Turns out , this year I am going through the same thing this week – just a different boss/city/team . How cool is that ! On top of that – this might be the last flight of the year too . After the first 100K miles, I don’t keep track closely – but I am sure I didn’t cross 200K miles this year . There is a blessing !

Last year in fall, I lost my father in law and my uncle . This year , I lost my fur kid Boss .

He left me sleeping in my arms , just as he slept in my arms as a puppy the day he came home . I will miss you for ever kiddo !

Sadly , my sister lost her fur baby too – two year old Charlie whom I loved just as dearly . Such a tragedy 

There were just two international trips this year – the lowest in two decades . I could get used to it !

The trip to U.K. was extra special this time – I got to catch-up with friends and relatives I haven’t seen in ages . The last time I saw my young cousin , he was in high school – and now he is a plastic surgeon who bought me a lovely dinner ! 

Next time I should find some time to visit my old Liverpool neighborhood .

Then there was the trip to India for my college reunion . I never had a chance ( and had a fear of public speaking ) in college to get on a stage . So it was amusing and exciting to get an opportunity to address my friends as well as the final year students this time 

Also managed to have breakfast with school mates , hit a movie with my bff from childhood, and ran into a buddy from school after 25 years and he works in IBM too, and managed to have meals with friends in Bangalore that I am in touch only on social media these days   – absolutely priceless !

Even sneaked in a conference on sentiment marketing at SCMS business school 

The best part of that trip was the couple of hours my father and I went out for dinner at the new restaurant next to his house. Achan used to take me to dinner frequently as a child and it was very nostalgic to do it again just by ourselves 

It was also pretty cool that we managed to celebrate Onam with my sister and family in Dallas – something that has not happened in a long time . 

Talking about Dallas – we started the year there celebrating my brother in law’s 40th birthday . And to add to the fun – I also managed to recruit a young rock star developer at the party ! Always be recruiting 🙂

Things were pretty good on work front too . Between (my) luck and (their) smarts – we had an amazing year with fantastic results . The sheer variety of transformative projects – redefining guest experience for a theme park , cognitive analytics to get fine grained insights on viewership at a cable company , advanced smart metering at utilities ….the quality of work we did for clients was outstanding . I can’t wait for more in 2017 .

The real highlight for me this year was that I spent equal amounts of time with my team that I spent with clients . I think I have somewhat succeeded in letting go of my urges to be a front line player and become more of a full time coach for the team – of course with a few(?) exceptions 🙂 . The best part of the time I spent with the team was with my younger colleagues who are startup off in their careers . I am convinced that I want to work for some of them before I retire !

I am also blessed with a bunch of senior partners that I learn a lot from every time we talk . The guy on the left of this picture below – Dave Lubowe – was the one who taught me many years ago that we don’t have customers ,   Only clients !

A highlight for me this year was college recruiting . The passion and curiosity that I see in the next generation of leaders that I met at universities – its the most energizing experience . We are in the talent business and I will do more of this next year without a doubt 

I took a conscious decision last year to minimize the number of conferences I attend . I only did 4 – and the big ones were SAP’s Sapphirenow and our own World of Watson . Both were amazing events and it was great to to catch-up with old friends and make new ones . But the one thing I will never forget is meeting Grady Booch ! 

I also have mixed feelings when I look at the photo below that we took at sapphire . My two dear friends Gagan and Dale have both moved on from IBM . I look forward to cheering them on as they start their new adventures

There was a lot more work life balance this year . I got to drive Shreya to school , attend her orchestra concerts and swim team meets and go for weekly family dinners to our favorite restaurants . Dhanya and I also managed to go to an Ilayaraja concert in Dallas . 

All it took was to reasrange my work life a little by taking less critical calls while driving , delegating a bit more and generally stopping a bunch of low value work that I routinely did in the past . The only part that didn’t work out was a Canadian vacation which we had to abort after one day to rush home to take care of Boss . The one casualty was that I had to get out of a few whatsapp groups to make this work – need to find a way to make this work as I absolutely love the conversations there .

Hobo is now the senior guy since Boss left . He is about as mature as a 6 month old puppy , although he will be 8 on New Year’s Eve ! I am still not used to seeing only two fur kids in my photos instead of three

The photo above reminded me that we did move to a new house earlier in the year . It was quite dramatic with how the process rolled out – but it’s home now and we love it . The only regret is that I have not even hit the driving range once , let alone played a round here . On the bright side – the sheer number of golf balls that land in my back yard gives me comfort that there are plenty of bad golfers out there in the wild . I am in good company 🙂

India has had an amazing year of success in cricket and I managed to catch most of the matches live by sacrificing sleep . If they continue to have this form – I think 2017 will see me sleeping less too . That said , I hardly attended any dog shows . Need to change that next year . 

While I did not manage to read all the books I wanted to – it was an above average year in terms of quantity and quality of reading . The three that I loved a lot outside business books are here 

On blogging front , I was just plain lazy . I read a lot of blogs but generally didn’t write as much as I usually do . Laziness combined with lack of inspiration is a bad combo . I will need to try harder to write more . For the first time since I started blogging , I have plenty of drafts in my wordpress app – just need to decide if any are worth publishing . My gut feel is that I will delete all or most of the drafts over Xmas . 

Merry Xmas and Happy new year everyone !

Its December , and its that time of the year when I am torn between planning for next year and closing out the current year on a strong note. As I sat down with a bunch of paper drawing up crazy ideas of what we will do next year, I realized I am falling prey to my biggest strength and weakness – the desire to measure everything at the most granular level.

I grew up in BI where if you had granular data, you can generate very sophisticated insights. And I have spent countless hours modeling such data, and writing ETL scripts to get that data in a shape I like. Over time, this has become second nature to me even though I don’t get to do the fun stuff with data any more. Most of my decisions are made on aggregate data today, and the only time I need granularity is when things go wrong and I have to “debug” to find out what the heck happened.

Granular data comes at a high operational overhead – in terms of management itself, and a lot of data wrangling. You need codes to tag every bit of data – and I am not kidding when I say that some times I come across more attributes about the data, and the number of data records itself 🙂

So as I sat here staring at the stack of paper on my desk and the array of spreadsheets on my macbook, I came to the sad realization that my penchant for granularity is simply a representation of my fear of failure. Over time, I have taken over more and more responsibility at work – and the measurements have also become more and more complex and time consuming. This is true for pretty much every employer I have worked for and every client for whom I have designed solutions.

Which brings me to the “scale of failure” issue. As your responsibilities increase, the number of ways in which you think you can fail also increase. And to compensate, we try to measure across multiple dimensions, and matrix the organization some more. At some point, you will absolutely realize that operational over head is not worth the trouble (do you need more checkers and double checkers than people in the field?) but by then you are also a creature of habit that cannot get away from this mess you created for yourself. And finally you also make everyone else in your team miserable – because you force them to tag more and more attributes and unlike you, they might not even know why they are tagging it. I am not kidding – there are many things I was told to do 10 years ago that I had no clue why I had to do. It would have lessened my grief significantly if my bosses at least explained why they made me do it then 🙂

Tagging data and multi dimensional operational reports have another consequence that was perhaps never intentionally designed – the sinister idea of taking credit for someone else’s work . At some scale of business – especially in sales type work – it is hard to pinpoint what all led to a given sale. You will have a direct sales team, and assorted over lay teams that all think they were the ones who drove the business. So they will enthusiastically start tagging more attributes to show their value add and we end up with “everyone is a winner” type scenarios. Even then we won’t typically stop this madness.

So my current plan is – I am going to sacrifice some granularity of measurement in our measurements. I want me and my team to design our work around the idea that we are aiming for success instead of lack of any kind of failure. I would rather fail responsibly quickly and stop doing things that don’t pan out, rather than worry about it every step of the way for all activities for ever. Lets see how that goes 🙂