Every Lesson Creates Another Judgment


A service starts timing out intermittently after a routine deployment. Nothing looks particularly alarming. The deployment itself appears clean, the underlying systems are healthy, and latency has increased but not enough to trigger the usual alarms.

An engineer eventually notices that the slower requests are interacting with a retry policy. The policy is perfectly sensible on its own. If a request takes too long, try again. But under the conditions created by the deployment, some requests are taking just long enough to trigger retries. Those retries create additional traffic, which increases latency, which causes more requests to retry. Neither the slower requests nor the retry policy is enough to explain the problem by itself. The problem is in the interaction.

Once the team understands what happened, it can fix it. Maybe the retry policy changes. Maybe the timeout does. Maybe the system gets better monitoring for this particular pattern. The exact engineering answer isn’t important here.

What interests me is what the company learns.

The most immediate lesson is very specific: change this retry policy so this particular sequence can’t happen again. That’s useful, and it should be done.

But somebody looking at the incident might extract a broader lesson. When latency starts rising, watch for retries that can amplify the problem. That lesson could help somewhere else in the system, even where the exact policy is different.

Go one level higher and the lesson becomes broader again. When individual components appear healthy but the system isn’t, look at the interactions between them.

All three lessons came from the same incident. All three are useful. But something changes as you move from one to the next. The first tells you almost exactly what to do, but only in a narrow situation. The second travels further, but somebody has to decide whether the conditions are similar enough for it to apply. The third could apply to almost anything, which is precisely why it tells you much less about where to look.

The more portable the lesson becomes, the more judgment it takes to use it.

I’ve been thinking about that because it feels like the problem sitting underneath the series of essays I’ve been writing about AI and work.

In The Average Is Lying to You, I argued that when AI takes routine work first, the average can stop describing the work people actually do. In The Side Door Is the Whole Building Now, I looked at what happens when exception processes built for occasional use start carrying much more of the human workload. The Dollar Threshold Is Static. Risk Isn’t. was about what happens to authority when the remaining work needs more judgment, while The Work Was Doing Two Jobs was about what happens when some of the routine work disappearing was also how people became experienced.

Then in The Normal Case Was the Company, I started wondering whether these were really separate problems. We didn’t simply build processes around normal work. Much of the organization itself was designed around it.

There is one more piece of that argument I hadn’t considered.

Organizations have always had to figure out how to make what one person learns useful to somebody else. One of the best ways we’ve found is to turn experience into something that travels. A rule. A process. A checklist. A standard. A precedent. A test.

When the lesson generalizes cleanly, this works extraordinarily well. The next person doesn’t need to repeat the original experience because the organization has already extracted what matters from it. We shouldn’t romanticize judgment and pretend everything needs to remain inside somebody’s head. Much of the progress of management has come from doing exactly the opposite.

The interesting cases are the ones where the lesson travels but doesn’t quite tell you what to do.

Take the retry incident. The company can preserve the exact technical fix. It can also preserve the broader heuristic about retry amplification. It can record the engineer’s reasoning, the alternatives she considered and the evidence that changed her mind. AI should make all of that much easier. Instead of reducing an incident to a few paragraphs in a postmortem that nobody reads six months later, an AI system could retain far more of what happened and retrieve it when a new problem looks similar.

That could be a significant improvement in how organizations learn.

But suppose six months later another service starts behaving strangely and the system retrieves this incident. Now there is a different question. Is the new problem actually similar to the old one?

Perhaps both involve rising latency, but for completely different reasons. Perhaps both involve retries, but the retry behavior that mattered last time is irrelevant this time. Perhaps the new incident looks nothing like the old one at the component level but has the same deeper structure: several things behaving normally and producing an abnormal result when combined.

Which similarity matters?

That’s judgment again.

And there is no reason to assume it always has to be human judgment. AI may become very good at this too. It may compare thousands of incidents, recognize structural similarities that people miss and tell an engineer which previous cases are genuinely relevant. It may eventually become better than most people at deciding which old lesson belongs with which new problem.

But then another distinction appears. Two situations may be similar in nine respects and different in one. Does the difference matter? A pattern may have predicted the right action fifty times. Is this the case where following it for the fifty-first time creates the mistake?

The problem isn’t that judgment can’t travel. Organizations have been making judgment travel for a very long time, and AI may allow much more of it to travel than ever before.

The problem is that every time judgment travels from one situation to another, somebody or something still has to decide where it applies.

That has made me think differently about the Context Loop I described in Migrating Scarcity. I was interested in getting relevant context back to the point where a decision is being made. I still think that’s important. But having more context available doesn’t eliminate the need to decide which context matters. In some situations, it may make that question more important.

This is the same scarcity movement at another level. Solve the narrow technical problem and value moves toward recognizing the pattern. Get better at recognizing patterns and value moves toward knowing which pattern applies. Automate that discrimination and the valuable question becomes which difference should override the pattern.

There may not be a permanent line somewhere in this chain marked “human judgment.” I don’t think the argument needs one. AI may keep moving further along it.

What matters is what happens to the work each time it does.

If AI increasingly takes the situations where we already know what matters, the remaining situations will contain a larger share where figuring out what matters is the work. If AI gets good at those too, attention moves to the next distinction it cannot yet make reliably.

That is why I’m increasingly skeptical of discussions about AI that end when a task has been automated.

The task was never necessarily the final constraint. It was simply where the constraint happened to be sitting at the time.

Organizations know how to make lessons travel. Rules do it. Standards do it. Precedent does it. Experienced people do it. AI may become the most powerful mechanism we’ve ever had for doing it.

But every time a lesson arrives somewhere new, there is another decision hiding inside the transfer: does it apply here?

Perhaps AI gets very good at that too.

And then the scarcity moves again.

The Normal Case Was the Company


Klarna is an interesting AI story partly because the AI worked.

When the company launched its AI customer-service assistant in 2024, the early numbers were extraordinary. After its first month, Klarna said the assistant was handling two-thirds of customer-service chats and doing work equivalent to 700 full-time agents. Average resolution time had fallen from 11 minutes to under two, repeat inquiries were down 25%, customer satisfaction was on par with human agents, and Klarna estimated the assistant would improve profits by $40 million in 2024.

A year later, something seemingly contradictory happened. Klarna began emphasizing humans again. CEO Sebastian Siemiatkowski acknowledged that the company’s aggressive focus on AI and cost reduction had affected service quality. The company began hiring again and put renewed emphasis on service quality and the role of people alongside AI.

It would be easy to tell this as another story about AI being overhyped, except Klarna didn’t abandon AI.

The more interesting question isn’t whether Klarna’s AI worked. It’s what happens to the human job when it does.

If machines handle a large share of straightforward customer interactions, the people who remain aren’t simply doing less customer service. They’re increasingly handling the things the automated system didn’t resolve, the things customers still want a person for, and the situations where something about the normal path didn’t work.

Klarna shows the first part of that pattern. The rest is what I’ve been trying to understand through this series. When machines disproportionately remove routine work, the averages can start lying to us about the work people actually do. As more human work consists of things that don’t fit the standard process, the side door can start becoming the whole building. When those decisions require more judgment, the dollar threshold starts looking increasingly crude because risk isn’t static. And when routine work was also how inexperienced people became experienced, the work was doing two jobs.

I’ve been treating those as different problems. I’m no longer sure they are.

The normal case wasn’t just most of the work. It was the organizing assumption of the company.

Think about how most organizations are built. Processes exist because enough situations repeat that we can decide in advance what should happen next. Jobs exist because enough tasks can sensibly be bundled together. Averages are useful because they tell us something about what people actually do. Approval limits work because fairly simple rules are good enough for a large number of reasonably predictable decisions. People get better partly because they do enough ordinary work before they’re asked to handle the difficult stuff. Even a cumbersome exception process makes sense when exceptions are actually exceptions.

The normal case made all of that possible, which is why I keep coming back to the argument behind Migrating Scarcity. When technology makes something abundant, scarcity doesn’t disappear. It moves to whatever the newly abundant capability still depends on.

There is something I didn’t take far enough in the book. What if the thing becoming abundant isn’t just a resource the organization uses? What if it’s the kind of work the organization was built around?

Much of the work companies are automating first has something important in common. It is structured enough, frequent enough or predictable enough to hand to a machine with confidence. Those are also many of the characteristics that made the work possible to standardize in the first place.

As more of that work moves to machines, what reaches people starts to look different. More of it involves ambiguity or disagreement. Facts don’t line up neatly. Objectives conflict. The policy covers most of the situation but not quite all of it.

And those aren’t just differences in difficulty. They put pressure on how work is measured, where decisions get made, when something gets escalated and how people learn to exercise judgment.

That raises a question I hadn’t really considered before writing these essays. If you were designing an organization mainly to handle that work, would you design the organization we have today?

I doubt it.

Take a different, hypothetical company. It automates 70% of a process. Unit costs fall, response times improve, customers get answers faster, and the business case delivers exactly what management promised. The automation was successful.

But the remaining 30% may now need different skills, different measures of productivity and more room for judgment. Escalation paths designed for occasional exceptions may start carrying much more of the human workload. Authority may still be calibrated to the old mix of work. And the company may eventually discover that some of the routine work it removed was also helping develop the people it now needs most.

The company solved the problem it set out to solve, but in doing so it exposed the next one. That’s the part that connects back to Migrating Scarcity. Removing a constraint and capturing the value released by removing it are not the same thing. If the constraint moves, eventually the organization has to move with it.

I think this may explain some of the frustration we’ll see as AI deployments mature. Automation rates will rise and unit costs will fall. Individual AI projects will show perfectly respectable returns. Yet executives may still wonder why the organization isn’t becoming as fast, adaptive or productive as the technology seemed to promise. The natural reaction will be to ask what else can be automated, and sometimes that will be exactly the right question.

But sometimes the AI will already have done its job. It will have removed enough of the normal work to expose something we hadn’t thought much about: much of the organization was built on the assumption that people would still be doing that work.

At that point, another model isn’t necessarily the answer.

The organization has to move with the scarcity.

For more than a century, one of management’s great achievements was learning how to make ordinary work repeatable enough to scale. We broke jobs into pieces, standardized processes and built organizations capable of handling enormous amounts of recurring activity efficiently.

AI may turn out to be extraordinarily good at precisely the work we became extraordinarily good at organizing. That doesn’t make the organization unnecessary. It changes what we need the organization to be good at.

We designed the organization around the normal case and built special machinery at its edges for everything that didn’t fit. AI may be taking the normal case first.

What happens when the edges are increasingly where the organization lives?

The Work Was Doing Two Jobs


A junior claims adjuster spends her first year handling fairly ordinary claims. Some of that probably teaches her something. Some of it probably doesn’t. She sees recurring patterns, gets decisions corrected, watches unusual cases get escalated and gradually takes on harder work. She may become excellent. She may plateau. She may leave before anyone finds out. Nobody designed this as an optimal curriculum. The claims needed to be processed, and she needed experience, and the same activity happened to serve both purposes.

That arrangement exists across professional work. Junior lawyers review documents and contracts. Analysts build models and check numbers. Developers fix bugs and write ordinary code. Accountants reconcile transactions. We tend to describe all of this as junior work, but it has always had two possible outputs: the work itself, and whatever capability the person develops while doing it.

AI changes the economics of the first output. If an AI system can process the ordinary claim faster and cheaper, the company has less reason to give that claim to a junior just to get the work done. There may still be regulatory, liability or audit reasons to keep a person involved, and those will vary by industry. AI may also make the junior dramatically more productive rather than remove the role at all. But wherever the human contribution to routine production genuinely declines, something changes: the relationship between the work a junior produces and the experience a junior accumulates is no longer automatic. The learning value of the work starts to become visible as a separate reason for doing it.

In the first three essays in this series, I’ve been following what happens as AI removes more routine work. The Average Is Lying to You was about how the mix of human work gets harder. The Side Door Is the Whole Building Now was about what happens when exception processes become a much larger part of the human process. And in The Dollar Threshold Is Static. Risk Isn’t., I looked at what happens to authority when the remaining decisions require more judgment. Put those together and you get a problem that isn’t really about whether junior jobs survive. It’s about how the people doing the harder work become capable of doing it.

The obvious response is to preserve some of the old work so people can learn, and I’m not convinced that’s necessary. Doing a thousand things isn’t the same as learning from a thousand things. Repetition without good feedback can just make someone very experienced at repeating the same mistakes. What matters is what happens inside those repetitions: variation, feedback, increasing difficulty, correction, consequences and the chance to compare your judgment with someone better than you. Once you think about it that way, AI could actually improve professional development rather than destroy it.

A junior adjuster doesn’t necessarily need a thousand randomly arriving claims. She could work on cases selected because they expose an important distinction: two claims that look almost identical but have different outcomes, a routine-looking claim containing one subtle warning sign, a case where experienced adjusters disagreed, a decision she makes herself before seeing what the AI or an expert concluded. Aviation has been doing a version of this for decades. We don’t insist that pilots encounter every dangerous situation for the first time with passengers behind them. Simulation is one part of a much broader training system, but it demonstrates something useful: experience can sometimes be designed rather than simply accumulated by chance. Professional work may move in that direction too.

But better training doesn’t solve the whole problem. In some ways, it makes the real problem easier to see: who pays? When a junior was producing economically useful work while developing, production and development were bundled together. Whatever training happened wasn’t free, but some of its cost was supported by the value of the work being produced. If AI takes away enough of that production value, development becomes easier to see for what it is: an investment. And part of the return on that investment can walk out the door.

A company can spend years developing a claims adjuster, lawyer, engineer or analyst only to watch that person take the resulting expertise somewhere else. Companies have always had this problem. AI doesn’t create it. What AI may remove is some of the economic activity that helped offset its cost.

When junior production had value, companies could employ people at the bottom of the pyramid knowing some would leave. If the productive value of those roles falls while the cost of developing expertise remains, the calculation changes. Maybe companies train fewer people. Maybe they train fewer people much better. Maybe AI makes juniors productive enough on harder work that the economics remain attractive. And maybe the market corrects. If enough insurers decide they would rather hire experienced claims professionals than develop them, experienced people become harder to find and more expensive, and eventually developing your own starts to look attractive again.

Except expertise has an awkward property: it takes time. You can raise the salary for an experienced adjuster tomorrow. You can’t create one tomorrow.

That is where this starts to look like a scarcity problem.

In Migrating Scarcity, I argued that when technology makes something abundant, scarcity doesn’t disappear, it relocates to whatever the newly abundant thing still depends on. That’s the mechanism underneath the book: relieving one constraint exposes another. I was mostly looking at technology, organizations and economic value when I wrote it, but the same mechanism may be operating inside the labor market.

If AI makes routine production abundant, one of the next scarcities may be accumulated human judgment. And accumulated judgment sits on a very different improvement curve from software. Models can improve quickly. Compute can be added. Software can be copied. Human judgment still has to be developed through some combination of exposure, decisions, feedback, mistakes, reflection and increasing responsibility.

We may get much better at that process. Simulation can compress some experience. AI can provide faster feedback. Training can become more deliberate than the accidental apprenticeship many professions rely on today. But compression isn’t elimination, and there is still a stretch of time between deciding that you need experienced people and having them.

That matters because each company can make a perfectly sensible decision on its own and still contribute to a problem for the whole industry. An insurer can prefer hiring experienced adjusters to developing juniors. A law firm can want associates who arrive ready to exercise judgment. A technology company can decide it needs fewer entry-level developers and more senior ones. If enough companies make those decisions at roughly the same time, they can all end up trying to buy something fewer of them are producing.

Eventually the price signal should matter. Salaries rise, experienced people become harder to recruit, and developing talent becomes economically attractive again. But markets can adjust prices much faster than they can create experience.

That delay is what makes this more than another argument about whether AI destroys entry-level jobs. Companies can invent new entry points. Training can improve. Simulations can become remarkably good. People may learn alongside AI faster than they learned without it. The harder question is whether enough organizations have an economic reason to pay for that development before the shortage becomes obvious.

Perhaps employees will finance more of their own professional development. Perhaps companies with exceptional training systems will treat talent development as a competitive advantage. Perhaps universities and professional bodies will move closer to the work itself. Perhaps new apprenticeship models appear precisely because the old bargain between junior production and learning no longer works. I don’t know which model wins.

But I think AI is exposing something the old professional pyramid allowed us not to price very carefully. Junior work wasn’t just cheap labor. It was also one of the mechanisms through which an economy produced experienced judgment.

AI may make much of the labor faster and cheaper without making judgment equally fast to create.

If that happens, the scarcity hasn’t disappeared. It has moved from doing the work to producing the people capable of handling what remains.

And that may be a much slower constraint to solve.