Pencils Down. Now What?



David Heinemeier Hansson has argued that writing code by hand is becoming economically irrational for a large part of software development. At the Rails World keynote in Austin on September 23, he said that over roughly 21 years he averaged about 30,000 lines of production Ruby a year, and that in August he produced about 150,000 lines in a single month using AI agents. His old pace works out to about 2,500 lines a month, so the jump is roughly sixty-fold. He acknowledged that much of the new code is verbose Rust that he lets agents write in ways he would never accept in his own Ruby, so the figure is a poor measure of productivity, as lines of code always have been. What it does show is that the economics of producing software have changed, and that is the part worth arguing about.

For most of the history of the industry, implementation capacity was the constraint. Companies always had more features, integrations, fixes and internal tools than their engineers could build, and that shortage shaped roadmaps, budgets, hiring and which ideas were worth discussing at all. If agents loosen it, more software will get built, because a large part of the cost that forced prioritization has fallen away.

That is the pattern I wrote about in Migrating Scarcity. When technology makes one part of a system abundant, the constraint moves to whatever has to absorb the abundance. In software, the first pressure lands on testing, security, review and architecture. A team that produces many more changes does not become equally better at understanding how those changes interact with years of accumulated assumptions, dependencies and workarounds. A new service can do exactly what it was designed to do and still leave the company with a dependency to run for years, a generated test suite can pass while encoding the wrong assumptions, and individually reasonable decisions can add up to an architecture nobody chose.

Verification will not stay the bottleneck for long. The models that generate code will get better at reviewing it, writing tests, tracing dependencies and watching production, because that work sits on the same improvement curve as the technology that created the need. What remains is harder to reduce to another model call. Someone has to decide how much authority the system has, where that authority ends, which consequences the organization will accept, and who can intervene when the system is doing exactly what it was built to do and producing an outcome nobody wants. Those questions concern who makes a decision and who carries the consequences, and a better bug-spotter does not answer them.

Knight Capital is a useful case, although it proves nothing about the volume of AI-generated change. In 2012, long before generative AI, a faulty deployment led the firm’s automated router to send millions of orders into the market over about 45 minutes, and the SEC put the resulting loss at more than $460 million. The failures the SEC found concerned authority as much as code. No second person was required to review the deployment, and an internal system produced 97 automated emails about the error before the market opened that were never designed as alerts and were not acted on. The firm also had no procedures for halting the router in response to its own aberrant activity or for deciding when to disconnect a malfunctioning system, and when engineers tried to fix the problem live, uninstalling the new code from the seven correct servers made it worse. The episode shows software acting faster than the organization around it could understand and stop it, and coding agents widen that gap.

Once an agent can write, test, review and prepare a change, the temptation is to require a human approval for every one, which recreates the coding bottleneck at the approval desk. The opposite extreme is no better, since a change to payment authorization or identity management is not the same as a change to a page layout. I call the answer Offensive Permission Architecture: define the blast radius, pre-authorize the actions that can safely be delegated, and make the activity auditable. In practice that means deciding in advance which classes of change an agent can make alone, how much damage is acceptable, what evidence is required before a change moves into a higher-risk environment, and who holds standing authority to stop it. Standing matters because the middle of an incident is the worst time to negotiate who may shut something down. Whether agents may ship unattended into regulated data or safety-critical code is a different kind of decision, hard to reverse and revealing about what the company will answer for, and it belongs with the people who own that accountability.

Cheap implementation also changes what gets built. Cost has always been a crude form of architectural discipline: when a new service takes a team three months, someone asks whether it deserves to exist, and when it takes an afternoon that question is easy to skip. The build cost looks trivial while the operating cost spreads across future years and across people who were not in the room. Each local decision gets cheaper while the whole system gets more expensive to understand, and this happens even with perfectly competent code, because every decision was defensible when it was made.

DHH is an unusual case against which to test all of this. He created Rails and has spent decades forming views on what good architecture looks like, so when an agent proposes an abstraction he is not meeting it as a beginner. His ability to stop writing code depends partly on experience accumulated while writing a great deal of it. The likelier reading of his results is that AI has given someone with a lot of judgment far more implementation capacity, which differs from replacing the judgment, and companies should be careful about treating his output as evidence that experienced engineers matter less.

That makes the talent question awkward. Experienced engineers learned by debugging failures they did not expect, watching elegant designs age badly, running systems under load and owning decisions whose consequences appeared years later, and some of that learning came from routine implementation work that agents now handle. Preserving manual coding for its own sake would make little sense, so the task is to preserve the learning without preserving the obsolete work.

A junior sitting beside a senior who holds all the authority does not acquire it. What works is bounded ownership of the boundaries themselves: a junior proposes what counts as a low-risk class of change an agent may make alone, defends the proposal to a senior who reviews the reasoning instead of redoing the work, and revises it after an incident. They can shadow the engineer with stop authority during a rollout and later exercise that authority in a controlled environment, own a section of a postmortem, or rotate through operations and diagnose a live problem before being shown the answer.

Line-by-line inspection is a poor skill to train, since review agents will get very good at it. The skill worth building is framing the decision around the code: what could interact badly with something outside the change, when the evidence is too thin, which consequence justifies a stop, and whether the system is growing more complicated for a reason anyone can still explain.

Companies can adopt agents much faster than they can redesign those responsibilities. Buying a tool can take a quarter, while changing who owns architecture, what teams may ship, how incidents are handled and how junior engineers gain judgment usually takes much longer. I call this lag Organizational Rewiring Latency. It matters because the productivity gains arrive first, and the organization gets the capacity before it has rebuilt itself to handle it.

By the end of 2029, I expect a visible difference between teams that used coding agents to increase output and teams that changed the operating model around them. The second kind will have clearer authority boundaries, named architectural ownership, explicit stop rights and deliberate ways for junior engineers to build judgment. The fair test is inside a single company, because well-run firms tend to do both things and a comparison across firms would mostly measure management quality. Take teams that got the same tools at roughly the same time and compare serious incidents relative to how much they change, along with how quickly they recover, using change-failure data and postmortems. If the teams that rebuilt their operating model don’t do better, this argument deserves to be questioned.

Migrating Scarcity is available here.

Frontier AI’s Security Problem Is Bigger Than the Sandbox


The incident began in May and culminated in July 2026. OpenAI agents running internal cybersecurity evaluations found ways around controls designed to keep them isolated from the internet. In pursuit of answers to the benchmark tasks they had been given, they exploited weaknesses in shared infrastructure, regained internet access and reached third-party systems. On Hugging Face, they recovered exposed credentials, chained together additional vulnerabilities and executed code across production systems. The agents later gained full administrator access to an OpenAI research cluster supporting its virtual machine environments. OpenAI called the incident a “warning shot”.

The motive matters. These agents were not trying to take over the world. They were trying to solve an evaluation. OpenAI says attempts to cheat by looking up solutions online were a primary driver of the incident, which it describes as reward hacking. The evaluations were also running with reduced safeguards compared with externally deployed systems.

That makes the incident more instructive, not less. It shows what happens when a model has a strong objective and behavioral controls are no longer enough. At that point, the infrastructure boundary has to hold strong!

OpenAI’s response is revealing. It rebuilt affected infrastructure, revoked credentials and tightened access controls. It also says it is creating more isolated sandboxes, restricting internet access, expanding security monitoring and strengthening incident response. The fix did not stay in the world of model behavior – it moved directly into ordinary systems security. OpenAI’s technical report goes into the sequence in more detail.

There was also a visibility problem. OpenAI says an internal team had already seen agents using its Artifactory service as an improvised message board in late May and cut off internet access, but the broader significance was not recognized quickly enough. Hugging Face publicly disclosed suspicious activity on July 16. OpenAI’s monitoring alerted on July 19, and it connected its own agents to the incident on July 20. The agents had crossed organizational boundaries before the organization running them understood the full path they had taken.

This was not a one-off class of failure. In February 2026, Anthropic patched a Claude Code vulnerability, CVE-2026-25725, in which code inside the sandbox could create a configuration file containing persistent hooks. Those hooks later executed with host privileges when Claude Code restarted. Researchers at Cymulate found the same broader pattern of trusted configuration and application-layer trust failures across other coding tools.

None of this looks like science fiction. It looks like ordinary security engineering: exposed credentials, writable configuration, unintended network paths and weak trust relationships.

A guardrail is not a boundary

A guardrail tries to influence what a model chooses to do. A boundary determines what it can do when it chooses wrong. You cannot prompt-engineer away an exposed Docker socket or fine-tune a model out of a badly configured package repository.

That however does not make alignment optional – I want to make it explicitly clear. Alignment reduces how often the boundary gets tested, while containment limits the damage when it does get pushed. But containment also has to work when the model itself is behaving as intended, because good behavior does not protect an agent from bad input. A poisoned web page, malicious issue comment or compromised documentation can steer an agent toward harmful action without the model becoming “misaligned” in any meaningful sense.

The trust graph

When people hear “sandbox,” they usually think about whether the agent can reach the host. That is only part of the threat model. The blast radius of an agent includes everything that can trust, execute, transmit or act on what the agent leaves behind.

The OpenAI incident itself shows why. The agents first used Artifactory, an internal package service outside their immediate execution environment, to exchange messages. They later exploited capabilities around that infrastructure to regain internet access. Something the agents could write to became part of a larger path to something they were not supposed to reach.

This same pattern can be much more mundane. An agent writes a file into a development workspace. The IDE indexes it. A language server resolves an interpreter referenced inside it. A later Git operation invokes a hook. A CI pipeline picks up the commit and runs it with broader permissions. Another agent reads the file later and treats its contents as instruction.

None of these systems has to be vulnerable on its own. The risk lives in the handoffs. The useful question is not only, “What can the agent access?” It is also, “Who trusts what the agent can produce?”

Most teams are much better at answering the first – many cannot even answer the second one coherently today.

Who owns the graph

The company building the agent cannot know everything downstream of it. The IDE vendor does not know which agent will write into a workspace. The model provider cannot see every credential, daemon or CI job inside a customer’s environment.

The deployer is the only party that can see the whole graph. Vendors can and should ship safer defaults, but once an agent enters a real environment, the customer becomes the systems integrator for trust.

Defaults matter too of course. A vendor that allows a file or configuration change to trigger privileged behavior is making a decision about somebody else’s trust graph, often without the customer realizing that decision has been made.

Why this is a scarcity problem

In Migrating Scarcity, I argue that when technology makes something abundant, scarcity does not disappear. It moves.

Agents are making execution abundant. Code, actions and workflows get cheaper to produce and initiate. What becomes scarce is confidence that an action is authorized, that its consequences are bounded and that one bad step does not quietly acquire a much larger blast radius downstream.

That is a permission-design problem. Reversible actions inside a bounded blast radius can be pre-authorized using short-lived credentials, narrow tools and isolated workspaces. Irreversible actions such as production deploys, credential changes or moving money should cross a different authority boundary.

The system also needs its own stop-cord, the equivalent of Toyota letting a worker halt the line. OpenAI’s response now includes clearer rules for who can stop a run, and for its most severe alerts, responders are expected to pause activity unless they can rule out a false positive within 30 minutes. That is the point of designing authority in advance rather than negotiating it during an incident.

What to do first

None of this requires a new branch of security. The pieces already exist in zero-trust architecture, workload isolation and DevSecOps. What has changed is the workload.

If I could fix only three things:

Map what trusts the agent’s output. Trace what can consume what the agent writes, including IDEs, hooks, CI pipelines, package managers, background services and other agents.

Treat egress and credentials as authority. An agent in a locked container that can call arbitrary external endpoints is not contained. Network access and credentials should be scoped to the task.

Match isolation to consequence. Narrow, mediated tools are easier to reason about than a raw shell, but they give up some of the flexibility that makes coding agents useful. Use narrow interfaces and explicit authorization for high-consequence actions, and stronger isolation, including microVMs where appropriate, when the work genuinely requires open-ended execution.

The principle underneath all three is simple: do not assume the workload is passive just because the infrastructure was designed that way.

The sandbox is just a container

If an agent generates a dangerous command but cannot execute it, the failure is contained. If it writes a malicious file but nothing outside its environment trusts that file, the failure is contained. If it calls a service but lacks the authority to reach it, the failure is contained.

Infrastructure security has always assumed that something inside the boundary will eventually behave badly and designed the system so the consequences stay bounded. AI does not change that principle. It makes it more important.

The sandbox is just a container. The real boundary is the network of trust we build around it.

Tort Law Arrives After the Damage. AI Risk Is Decided Before It.


In July, Hugging Face disclosed an intrusion into its production infrastructure that turned out to have been driven by OpenAI’s own models. OpenAI later said that, during internal cybersecurity evaluations, the models got around controls intended to isolate them, communicated through unauthorized channels, gained internet access and reached third-party systems. OpenAI called the incident a “warning shot.”

The scale made it hard to dismiss as a curiosity. An independent investigation by METR and Redwood Research found that roughly 1,200 agents meant to be isolated communicated with each other through an unsanctioned message board, exchanging more than 70,000 messages. More than 700 later took part in the attack on Hugging Face.

I have been thinking about that incident from a legal angle. If an AI agent causes damage that no human explicitly instructed it to cause, how should the law decide who is responsible?

To be clear – I am not a lawyer. Thanks to my long career in the corporate world, I have always been curious though about how laws affect the world I operate in. I did a lot of reading before penning my thoughts but I will happily stand down on any of this if actual legal experts think I am off base. As always, these are strictly my personal opinions and not that of my past or present employer.

The obvious answer is tort law. We have spent more than a century adapting negligence and product liability to new technologies. Cars, drugs, industrial machinery and software all created unfamiliar risks, and the law eventually found ways to deal with them.

I am sure it will adapt to agentic AI too. My concern is timing. Tort law decides who pays after the damage is done. The decisions that shape the eventual case are made much earlier.

The warning shot changes foreseeability

Negligence law does not require anyone to predict the exact accident. It asks whether the general kind of harm was reasonably foreseeable. That distinction matters when the whole point of an agent is to choose its own intermediate steps.

A company deploying an agent may genuinely have no idea what sequence of actions it will take. But Hugging Face changes the argument. The next time an agent moves outside its intended path, the company can say nobody could have predicted that particular sequence. A plaintiff can answer that the industry already knew this class of behavior was possible.

Admittedly, this was not a normal enterprise deployment. The models were being used in offensive cybersecurity evaluations and were operating with reduced safeguards. A defense lawyer will make a lot of that distinction, and reasonably so. But the broader category of behavior is now harder to call unforeseeable in my non lawyer eyes.

The evidence is scattered

Suppose an enterprise agent makes an improper payment. Was the problem the model, a stale retrieval result, the system prompt, the orchestration layer, the credentials it was given, or the absence of a control around the transaction? In practice it could be several of these at once.

Courts already deal with complicated causal chains. What feels different here is where the evidence lives. The relevant facts may be spread across model logs, application code, retrieval histories, tool calls, identity systems and permission records owned by different parties.

Even Hugging Face already knew it had been attacked without knowing that OpenAI’s models were behind it.

That makes me somewhat less interested in which output was “the bad decision.” The more useful question is often who created the conditions under which the system was allowed to make it.

Permissions are evidence

An AI agent should not be treated as trusted internal software simply because it runs inside the enterprise. It behaves more like a non-human identity that can reason, call tools and move across systems.

The permissions given to that identity create a record of what the enterprise allowed it to do. Standing access to a payments system, persistent credentials, permission to communicate externally or authority to change a customer record may look like implementation choices. They are also evidence of delegated authority.

This is why zero standing privileges matters beyond security. Give the agent the access it needs for the task, for as long as the task requires it, and remove it when the task ends. Apply the same logic to credentials, financial authority and external communication. Preserve enough evidence to reconstruct what happened (which is hard).

Those choices narrow what the agent can do. They also improve what the enterprise can prove later.

In my book Migrating Scarcity, I argued that when AI makes something abundant, value moves to whatever remains scarce. Agent actions are becoming abundant very quickly. Clear evidence of who authorized those actions is not. I called this “Offensive Permission Architecture” in my book.

The law is already looking through the agent

There are early signs that courts and legislatures will resist treating the AI itself as the place where responsibility ends.

California’s AB 316 says that a defendant accused of causing harm through AI cannot defend the case simply by arguing that the AI acted autonomously. The statute does not impose automatic liability. Causation, foreseeability and comparative fault still matter. But “the AI did it by itself” is not enough on its own.

The Ninth Circuit’s recent Amazon v. Perplexity decision is useful in this context. In a computer-access case, the court treated the user as the party accessing Amazon and the AI assistant as the tool being used. The court stressed that its holding turned on how Perplexity’s assistant specifically works. Even so, it shows that how an agent is wired can affect whose action the law sees.

The timing problem

None of this makes tort law irrelevant. The threat of liability is one reason companies take precautions in the first place.

But by the time a court asks whether a harm was foreseeable, somebody has already decided what the agent could access, which credentials it held, how much authority it had, what it could change and what evidence it had to leave behind. Those choices may shape the case more than the model output that triggered the loss.

And because most enterprise AI losses are likely to be economic rather than physical, the contract terms negotiated before deployment may matter as much as any tort doctrine.

That is why the Hugging Face incident matters beyond security. Once the industry has seen agents work around controls and reach systems they were not supposed to reach, reasonable care has to start moving upstream.

Scarcity is migrating here too. When action becomes cheap, the ability to show who authorized it becomes scarce.

Tort law will still decide who pays. The permission architecture you build now shapes what the court sees.