It’s about time we set a higher bar for analytics


First off – I have no interest in being nuanced here about data vs information , big data vs regular data , BI vs Analytics vs Reporting and so on . Use the terms you like in the following rant . If all we missed today was the nuance between these terms , we would have been in a better place already .

There might not be a more tired part of technology out there today than Analytics . It’s been a top CIO concern for as long as I can remember . It is a top concern for me personally as an executive running a business at my employer . There are very few people who are truly happy with their ability to make sense of data . And no wonder BI companies young and old are all thriving and marketing their hearts out that they are already in “next gen”.

Big data – and the Gartner 3V model – brought good focus on information management side . But it did not exactly democratize “data based decision making “.

We don’t need to get into high volume or velocity or variety of data to see the failings of today’s BI . The thought leadership in analytics is along the lines of “ask good questions to get good answers”. This is a much needed part of analytics for sure – we should have the sophistication in our systems to answer the best and most complex questions . However – that should be more of a table stakes thing in the world of data .

For analytics to take a leap into “next gen” – pretty visualization is not enough either . It is a little more along the way into future than the ability to answer complex questions . An answer is not good if the user does not understand what the system is trying to say . So yes – let’s find more and more ways to visualize data . It’s a good thing that some companies got started with it and are making progress .

Before I leave the topic of visualization – I have to say this . For a given question and its answer , computers must be able to provide a default representation that is the “theoretical best” , without a user being asked to create a representation from scratch every time . By all means – allow the user to change things around , but there is very little value in making a user guess what is the best representation visually . The type of chart , positioning in screen , scale , default filters are all things that software should be able to figure out by itself . Personalization is important – but again it should be treated as table stakes by now .

But there is a gap that still remains at the core . If the best we can do is “ask good questions to get good answers”, we are still stuck in the world of art , and refusing to move a bit closer to science .

My hope is that we start building systems that can
1. look at available data and start by
2. offering clues to what kind of questions can be asked of it ,
3. what kind of meaningful patterns the software can be seen already ,
4. and what kind of data would be great to add to currently available data set to make even better inferences .

Sure , the person who can ask better questions will still retain their edge in making better use of the data – but if the system can prompt decent questions to users , I think this whole promised land of “data driven decisions” would be a lot closer to where we are now than if we inched along the current trajectory .

This might be really hard to do as a generic horizontal platform capability in near future . But if vendors focused on taking this approach for targeted vertical apps , a lot of these challenges can be mitigated . Such learning can then be used to build horizontal general purpose solutions over time if it makes sense .

It all starts with setting a higher bar of expectations of analytics – incremental innovation won’t cut it .

Before we pelt social media stones on Satya Nadella …


Satya Nadella is a very visible technology industry leader who made a totally wrong and awful comment about women asking for wage increases .

Over the last couple of days there has been a storm in social media crying for his head . Shortly after the gaffe – he admitted publicly that he was wrong , and that women should just ask for a raise of they need it .

In my mind – that was the right first step in righting a wrong . But clearly I am in the minority in thinking so . Many people I admire and respect – like industry analysts and CXOs think saying “sorry” does not count . Some believe he should be fired . Some others don’t exactly say what they actually want him to do – except that something beyond saying “sorry” is needed .

I don’t think he should be fired at all . All leaders – all humans – make mistakes . The honorable thing to do is to admit openly when they are wrong , and do something to fix the problem and trying hard not to make the same mistake again .

In this case – he did say sorry and he agreed that women who feel they need a raise should just ask . Now it is up to Microsoft women employees to ask him for raises when they feel they deserve it and for him to give it in deserving cases . Not just women – any minority , or even any employee – should enjoy that at their place of work . Bosses should ALWAYS encourage employees to speak up .

Microsoft diversity numbers look like pretty much every other big company – gender/ethnic and all other kinds of equality still remain elusive . So it is not just Satya -every leader should be held accountable to making this right .

Also I wonder if this is a problem with male leaders alone . There are a minority of F100 companies like IBM, HP, Pepsi etc where CEOs are female – are their diversity situation much better ? If not these women leaders should be held to the same high standards too I would think .

A big problem with the fight against inequality is the lack of unity amongst Minorities . Women are a minority – physically challenged folks are a minority , Hispanics and blacks are Minorities and so on . They all fight for their right to be treated as equals – but only for themselves . Why not join forces and make the collective voices heard ?

( Example – It always has amused me when some Indians who have gained US citizenship make fun of the folks coming after them in the immigration queue on an H1B . )

It was interesting that no peer CEO of Satya defended him nor criticized him in public . And as far as I could see on social media – they were not asked for their take by the journalists and bloggers either .

The current rhetoric is mostly against Satya, not the underlying problem of inequality . Are we outraged because he sounded honest but not politically correct ?

What exactly is the priority here ? This is a country that had a president who falsely went to war claiming weapons of mass destruction , and another who lied under oath after marital indiscretion. Both seem to be popular till today , though their actions failed entire populations . Neither one even remotely made a meaningful apology but were politically correct . Even they were forgiven . What Satya did was wrong – but not as terrible as what two presidents did . If we can forgive the presidents of USA , I would think we can give a second chance for a CEO too.

I can’t speak for Satya at all – but I have a hypothesis on what made him say the awful thing he did . He is a young CEO who had a fast track career . He probably never had to ask for raise in his career . He might have assumed – wrongly – that his own experience ( or Karma ) is applicable to women employees too . Hopefully we will learn more with time .

What I would like to see happen positively out of this episode is two things

1. the world holding all leaders – at all levels – accountable to principles of equality , rather than unleashing fury on just one CEO .

2. Women – and anyone else who feels like they deserve a raise – being encouraged to speak up , and their leaders acting on it constructively

PS : Just saw the first hand commentary of the lady who interviewed Mr Nadella . Even she is surprised at the twitter fire storm

http://www.bloomberg.com/video/harvey-mudd-college-president-reacts-to-nadella-s-comments-m3CzZR0dTDOVXZCkeV34Rw.html

Do you need a college education to have an IT career ?


One of the most common questions I get from my younger colleagues and mentees is about the value of college education in pursuing an IT career .

In my early 20s, I had one big regret – I did not go to a big name US college to get a degree . My engineering and MBA degrees were from University of Kerala in India . A lot of my friends did take their degrees – some times a second degree even – from reputed US schools . I had some kind of an inferiority complex about that when I started out – but I got over it at some point soon.

Education in India is not as expensive as it is in the US . Four years of engineering college and two years of MBA together cost about $5K including food, travel, hostel and so on . My parents picked up that tab and within my first year of getting a job, I repaid them (They didn’t ask for it ). I got my first job from my MBA campus – and became a consultant in Tata Consultancy Services .

I have always been startled to see how much of a burden the students who take degrees from US carried on their backs as student loans . Most of them have at least $30K loans after under grad and many have $100K+ loans after MBA from a good school . There are very few who get full scholarships and even fewer cases where parents pay for education like back in India in my college days . It takes a very heavy toll on these kids to repay those loans .

To begin with – I never quite understood the rationale behind taking a big loan for a liberal arts degree , especially out of state , given the low demand in employment market . But then – people are free to take their decisions and live with the consequences . I have occasionally seen liberal arts graduates transition into fields like IT later in life , and some are great entrepreneurs . So probably there is a way to make it work .

Which leads me to think – is there a real value in a college degree anymore ?

Sure there is the fun aspect of hanging out with other students and the drinking games and so on . But that fun is negated by the first few years after college with a loan to repay . It can be argued that such fun could be had slightly later in life when you earn a salary or make money from a business .

Looking back – college did not teach me much . My first job was as a programmer – my mechanical engineering classes and financial management classes did not help me directly in writing C and ABAP code . Sure there were occasional indirect benefits like understanding finance theory helping write better code for financial users etc . But there were programmers without that background who also were able to learn it on the job .

Did my MBA help much ? Maybe it did later in my career – after I had a few promotions etc . MBA certainly did not directly help me get any of my promotions either . It is possible that if I took an MBA half way through my career , instead of getting it at the beginning – it would have helped more . But I also think that my career would not have progressed as fast as it did if I took a two year break in between at middle management levels to go back to school . So if I could go back in time – I would have not done my MBA and instead would have started working full time two years sooner .

Without a doubt – everything that helps me in my career today are things that real projects and real customers have taught me . Neither my engineering school nor B school gave me skills to even write a good status report at work .

I probably would still have done my engineering college if I could time travel back to 90s – but that is not because the course work helped me , but only because a 4 year engineering degree was minimal requirement to get a good entry level IT job back in 90s in India . That is not the case today – I have seen plenty of people who can code well get very good jobs without any formal education .

Talking about India in particular – thee is something rather stupid about hiring policies . If IT demand is so high and every branch of engineering sends its grads to IT jobs for the most part, why do they keep churning out as many mechanical and civil and electrical engineers instead of computer science engineers ? Why doesn’t demand and supply match ? It’s a lot less expensive to set up a computer lab than the type of labs traditional engineering needs . And why is it that there is demand mostly for engineers ? Why can’t commerce grads and economics grads learn to code like mechanical engineers do in the first year of their jobs ? I hope the scene has changed in India . Oh before I leave the specific topic of IT in India – why on earth do employees of some companies have to wear a tie to work every day in the hot and humid weather ? Their counterparts in other countries rarely wear a tie and most customers don’t wear a tie . What exactly is the point that their employers are trying to make ?

With broadband access not being an issue unlike when I was in college – there are a lot more options to get a degree at a lower cost today , if getting a degree is a big deal for you .

So is there any reason at all to go to college that I consider a positive ?

There are two –
1. The network you build . The relationships you establish at college will open doors for you for the rest of your life . That alone might make it worthwhile to bear the burden of a hefty college loan . But remember – that network needs work to build and sustain . Just being in a college doesn’t always lead to a good network .

2. Getting the first corporate job . There are plenty of companies who think a college campus is a great place to find entry level talent . So if you choose to go to college – it would help to put the energy into doing everything that would attract the employers who hire from there . This also means that if no one is hiring from there – spend the loan money at another college .

I am not basing this on my own case either – I am also thinking about my experience hiring people over the years . While I admit that a big name college on a CV catches my attention when I do my preliminary reviews – I have never given it a lot of weightage in my final hiring decisions . I put more trust in what they have done in their last jobs and how well their references check out . As far as I can remember , no employer has asked me about my degree or college after my first job interview .

There are always exceptions to all this stuff – but for the most part , I think your chances of getting a break in IT is higher if you do roughly the following

1. Learn to code the soonest you can ( I am partial to C and C++ – but more modern languages might work well too )

2. If coding is not your thing – learn something else around computers like networking . Or get a non IT job and come back to IT as a domain expert .

3. If you don’t have someone like your parents picking up the tab and still want to do college really bad – go In-state and try to do everything to maximize your employability in your first job ( ideally STEM, good grades , side projects etc)

4. Once you code well, get your presence felt in open source communities . Contribute code – and choose a license that works in your interests .

5. The moment you get your first job – do all you can to increase skill levels and start aiming for next level . Get a mentor or two the soonest you can – without that it is hard to know what “next level” even means . Every chance you get , do non-IT things like sales and presales and so on . Breadth matters as much if not more than depth.

6. Learn about how companies are formed and decide if you want to start your own at some point . If you have money to spare for an MBA – consider investing it first in your own venture , or in someone else’s before giving it to a business school . If you make it to the top few B schools – by all means go for it and it will probably help you . But for everything else – think really hard before spending that money .

PS : When I chose mechanical engineering to graduate in , the ONLY reason was that my dad was a mechanical engineer . If you go to college, and need to take a loan to do so – I really hope you have a better reason .

PPS : Unlike my dad who is a terrific engineer , I did not pick up any useful skills in my engineering college . I did pass in the top % of my class though . However , I am proud that one of my class mates from school is a lead engineer for India’s successful Mars Mission . Moral – it is a great university that I went to , I just did not choose my major wisely . I hope you folks choose more wisely .