And Vijay Says…

As always – these are strictly my personal thoughts and not that of my employer.

Over the last few months, the most frequent question I get from my friends and family is about the future of tech jobs – given I have been in this industry for a while. This past weekend- I spoke with a friend who lost his engineering VP job at a big tech company and then with his son whose job offer has now been delayed twice as an entry level engineer.

The first question I get asked is whether people are let go from tech companies because AI can do their jobs better.

I think that’s part of the answer – when it comes to HR, Operations etc. AI and other tech are mature enough today to take over a lot of functions that have historically needed humans to execute. The fact that AI can do something doesn’t mean that you can switch it on and fire people that afternoon – it still needs a lot of change management and all change is hard. But in the short to medium term – I do expect a lot of functional tasks to be done by machines. I also think that many companies will rush into this without proper planning and get it wrong and pay a price.

For engineering jobs – where someone has to actually code, AI today at best is a job aid and not a replacement. But it is an excellent job aid – which means engineering managers might not need as many developers to get the job done, or they maybe able to get more done with current capacity. The reality is that a lot of engineering teams have excess capacity hidden in plain sight – people doing manual testing and code review instead of having better CI , people hand coding deployment pipes and so on. It doesn’t even need AI to find productivity – but AI will just make a strong case to look at developer productivity at all levels.

The next at risk role is managers who are largely serving an aggregation function with no hands on skills – engineering managers who cannot code at all, sales managers who only look at CRM and don’t make client calls , Ops managers who have no skills in optimization and so on. AI is excellent at summarization – and we already have other tech that can aggregate numbers and make comparisons and so on. All organizations have inertia when it comes to attacking structure – so it might take time, but there will be no place to hide soon for people who don’t have higher order skills to either make more money for their employer or save costs.

Why is this happening now?

That’s just the nature of competition and business cycles in a capitalist system. Resource allocation happens where companies expect the most return. This doesn’t only happen in down turns – look at the massive hiring and wage inflation that happened during the COVID years where the belief was that exponential growth will continue. When that growth stopped real fast – cost simply didn’t keep up with revenue. At high interest rates – revenue going up was always going to be difficult and hence the logical option was to cut costs to improve efficiency.

It also just happened that AI just got some new wind on its sails at the same time – so the risk perception probably is lower now about letting go of employees.

Just as companies hired way more people than they needed in the boom cycle – there is a good chance they will over correct in the bust cycle. I expect this to reverse as soon as fed cuts interest rates but probably we won’t see the levels/speed of hiring we saw in the past given AI is also progressing rather fast along the way.

Sales jobs might be the one exception to this. Example : When Elon Musk fired a lot of people at twitter/X – engineering seems to have found an equilibrium, but they still have revenue issues.

AI and its compute problem

One reason for some tech companies for finding more investment is the initial cost of AI compute. Training AI is extremely expensive and time consuming. The current approach with deep learning ( transformers included ) is to teach a system with the collective knowledge of all humans – and when that runs out, to augment with synthetic data. That’s not how any one human being learns – no child reads all of Wikipedia for example. The GPU, the data centers, the lack of top talent – all of this adds to the cost very fast.

This causes two things

1. Companies will need to find money quickly to fund these expensive R&D initiatives to remain competitive

2. Smaller companies will have to stay a couple of steps behind the big ones in some cases given the high cost.

Open source AI solves some of the problems but what we need is for AI to learn and think in less expensive ways. I am hoping that the research community comes up with viable alternate approaches quickly

So what about the people whose jobs get affected?

1. The skill we need the most to stay ahead of the tech onslaught is to learn and unlearn really fast. There is no saying what is the skill we will need in 5 years – which means we need to be willing to always be learning new things.

2. The “real” top talent will always be in hot demand in all markets. For the rest – we need to have an alternate strategy on top of the constant learning. If you are an above average skilled person in a top tech company and you lost your job – you could still be in high demand for the tech companies in the next tier.

3. Disrupt instead of being disrupted. If you gain experience in using AI to improve HR functions in your current company – you probably have high odds of being considered for a higher order role in your company, or be in demand to do more of that in another company that wants to do AI in HR. I just used HR as an example – this is true for engineers too.

4. Objectively assess your value add – especially if you are a manager. This is a lot harder than it sounds – most of us are not that objective about ourselves. Get to be more hands on and add value. Status quo will get challenged sooner than any of us will like

As always – these are just my personal thoughts.

One of the questions that most of my clients are debating about is whether Open source LLM will ever catchup and over take the closed source models. It is a question worthy of some debate given there are serious investment decisions to be made and long term implications of all sorts to be considered.

GPT 4 is considered state of the art on code generation, language understanding and other important things where there are decent benchmarks – and it’s closed source. That’s probably the big reason why there is even a debate on whether companies should bet on Opensource at all.

My current thinking is that open source will win the enterprise in the fairly near future.

Consumer and enterprise requirements are very different. A good example is my own preferences. For my day job – I have been using a MacBook for a long time. It does everything I need – and it is easy to switch to watching a YouTube video or play a game. I also code – though no longer for a living. I have a marked preference for Linux in enterprise applications (which is what I focus on). I see no conflict between using MacOS and Linux for two different needs that I have. I am not sure how well the analogy extends – but I think enterprise world will not make it a requirement to standardize on whatever becomes the standard in the consumer world.

One big limitation today is cost – which is largely about the GPU hunger of training the model. Not a lot of companies will spend $100M on training a model. But I have to imagine that two things will happen

1. The cost of compute will come down rapidly making it affordable for open source models to catchup

2. The AI research will figure out a less expensive way to train models, which in turn makes it viable for open source models to compete fair and square

In any case – there are companies like Meta who are on the open source camp who have the ability to spend $100M on their models.

Also – assuming the near future is largely about sticking to the transformer architecture but making much bigger models, I think LLM will hit a plateau anyway. The kind of historical jumps in capability that we say till GPT4 came out – it’s hard to imagine that continuing for auto regressive LLMs perpetually into the future. That’s another reason why I think open source will catchup just fine.

Enterprise world is one full of narrow use cases. What’s important there are things like accuracy, compliance , security and so on. While enterprises will absolutely benefit from multi modal AI – there is plenty of value that can be extracted with just text based models. Given that case – I would think that an open source model that is fine tuned ( and most probably with RAG ) is the more viable approach for most companies. Also given GDPR like use cases – they will need work arounds on how to implement things like “right to forget” , “data localization” etc.

So what is the future of closed source models in enterprise ?

I can imagine enterprise software vendors creating their own models from scratch at some point mostly for regulatory reasons and then for their business model reasons – they might choose to keep it closed source. That’s the same pattern as perhaps a large bank creating a purpose built model from scratch. When the compute becomes cheap – a lot of more experiments will happen and it will be fascinating to see what models emerge.

I think the debate about open vs proprietary models will be settled within the next year or two – and as of now I am betting on open source being the winner. but that might not be the biggest issue for enterprises in the context of enterprises.

I think the biggest problem will be about scaling LLM based solutions for an enterprise use case. A POC is easy enough to do – including fine tuning and RAG. But a 4 week POC usually cannot be scaled to an enterprise use case in a straight forward way. I need to organize my thoughts on that – but I plan to write about it hopefully soon.

I spoke with about 50 colleagues yesterday – quick calls to thank them for all they do and to wish them happy holidays. In many ways, 2023 was a stressful year for many of us and it came up in various forms in our conversations. I was asked by some of them on how I handle work pressure. After a couple of coffees in the morning, I thought I should offer some perspective on this topic.

1. Have safety valves – do not stew in pressure

You need some proven ways – note the plural; one might not be enough – to destress. For me those are – long walks, calling one of my trusted friends, and Carnatic music. There is a 6 mile circuit that I reserve for such walks and it invariably helps calm me down when I need it.

2. Put boundary conditions if you are constrained on time on high risk decisions

I have a mental model that tags the consequences of my decisions two ways – what’s the chance of the risk happening and what’s the impact it happens. Stress gets in the way only on decisions where the risks have a high chance of materializing and the impact is high if it does play out.

Pressure is inevitable when you have to make such decisions with imperfect information within limited time. If I have to take such a decision – I insist on reasonable boundary conditions like “ok let’s do this now – but here is how we will check if it works and we will stop in a month if it doesn’t trend like our hypothesis”

3. Form follows function

I prefer a rigorous debate than a well structured document as the basis of a decision. I do like documenting decisions once we make it though.

Similarly – some metrics have a way of hurting decisions as much as they help. People lose sight of the principle behind the metric and sometimes become a slave of ratios etc to make decisions. So I prefer asking a lot of first principles based questions before I make decisions – and it helps minimize stress because it makes everyone involved think logically.

4. WHY is the critical question, WHAT will follow

Leaders create stress unintentionally when they ask the team something without explaining why they are asking. The more senior you are – the more the risk of you creating unnecessary stress in the organization.

If you have taken the time to hire, train and manage performance of your team – you should have the confidence that if you explain a problem, your team can find solutions. Your job is to find the right question to ask and to explain why solving it is important. If you don’t have the confidence in your team – shift left and solve for the quality of your team as quickly as you can.

5. Eliminate and simplify

Not all problems need a solution with high quality and they don’t all warrant similar effort. Even a given problem can usually be decomposed to find what’s the critical part and what can wait. Eliminating the noise and simplifying the problem statement goes a long way in eliminating stress.

Don’t assume that the person asking has thought through all aspects before asking you. So you should feel validating whether that you think is the crux of the issue is what they also think. I am always grateful when I am challenged by my team when I ask a question. I am very comfortable asking clarifying questions to my boss as well.

6. Know the audience when you communicate

Even the best decisions can lead to additional stress if you don’t think through the answer from the recipient’s point of view. If I have to convert the same information to both the executives in my team and to new hires – I often will need to say it differently. Similarly the response to a client might look differently when it is addressed to the buyer vs someone who is the actual user.

I am convinced that more stress is caused by poor communication than poor decisions.

7. Keep shifting left and make as many things a routine as possible

I used to train dogs for high level competition. Before we start training – my dog and I will go through a certain routine which gets us both into the right frame of mind. Routine helps minimize stress and keeps us focused. High level sports people all have routines they follow.

I have routines in personal life – I wake up at the same time most days, make an espresso, do a two minute training session with my dog, and solve the daily Wordle and then go for my walk. It helps me get into the right frame of mind for rest of the day.

At work – processes don’t die a natural death. That’s a real curse. They don’t even evolve very much and people start hiding behind processes as a their safety thing. Even when they criticise these “bureaucratic processes” – they take it for granted that it’s sacrosanct. A massive amount of stress happens because people don’t kill irrelevant processes. Your job as a leader is to make sure that every process gets critically evaluated and either eliminate or work around the ones that don’t add value.

Similarly – the moment you have a stable solution, make it a routine so that people don’t need to waste their time thinking about it deeply every time. All I would caution is “premature optimization” kicking in. Especially in large firms – there is a tendency to institute process upfront in a top down way. It almost never works – the better idea is to experiment, evolve and then standardize the process.

The shift left often ends up with hard questions about your skills as a leader and whether you have the right team around you. Don’t sit and stew on those – make changes when you are convinced that there is a problem. Bad news doesn’t turn for the better with time without interventions.

8. Learn from everyone around you

There is no monopoly for good ideas. If the marketing team has a good process for recruiting – shamelessly steal it and adapt it for engineering teams. It is much better for one person to solve it and others to adapt than all functional heads stressing over solving common problems from scratch.

I am sure there are more things I should call out – but it’s time to drive to the gym, so I am not going to stress over the rest for now 🙂