I came across a paper on harness learning that got me thinking about something we don’t discuss enough in enterprise AI. We spend a lot of time worrying about what agents are allowed to do. What happens when they can also change how they do it?
The researchers train a model, called a proposer, to modify the software surrounding another model. This software, usually called the harness, determines how the model uses tools, handles information and carries out tasks. Rather than improving the underlying model, the proposer experiments with the harness and uses execution feedback to find changes that improve performance.
The interesting finding is that the proposer learns something transferable. It can improve harnesses for tasks it hasn’t previously encountered, without changing the underlying model’s weights.
I can see why people are excited about this. It suggests there may be more room to improve AI outcomes without constantly upgrading models. But it also raises a problem for enterprises. What if we eventually become much better at generating improvements than we are at evaluating and authorizing them?
When better performance isn’t enough
Consider an insurance company that uses an agent to settle straightforward claims below $10,000. The workflow includes policy verification, document checks, fraud screening and escalation rules. Some of those checks rarely affect the final settlement decision, but the company performs them because it needs evidence that the correct process was followed.
Now suppose a proposer discovers that removing one of those checks improves throughput by 20% without materially changing settlement accuracy. Its evaluation says the modified harness is better.
The difficulty is that the check may exist for reasons the evaluation never measured. An insurer may need those records to defend a disputed claim years later. A regulator may require them. Or the check may catch a rare class of errors that the test data barely represents.
The proposer has found a way to improve the metric it was given. It doesn’t need to be trying to circumvent a control. The control may simply look like unnecessary work when judged against an incomplete objective.
A change can improve measured performance while remaining unauthorized.
This example is hypothetical. The paper demonstrates improvements on reasoning and question-answering benchmarks, not autonomous modification of regulated insurance workflows. But the distinction it raises matters well beyond insurance.
Permission to execute versus permission to change
Harness learning makes an old distinction considerably more important: permission to perform an action is not the same as permission to change the system that authorizes and performs it.
In my book, Migrating Scarcity, I argue that when technology makes one capability abundant, the constraint often moves somewhere else. One idea I develop there is permission architecture: the authority, evidence and controls that determine whether an AI system may take an action. Harness learning extends that idea one level up.
An agent might have authority to settle a claim for $8,000. That doesn’t give it authority to eliminate a fraud check, change the escalation criteria or rewrite the process by which settlement decisions are approved.
Of course, enterprises already have change management. Banks have model risk management. Technology organizations have release approvals, separation of duties and production controls. None of this was invented because AI came along.
What changes is the rate at which modifications can be proposed, and the possibility that they affect the very controls governing the system. The existing principles still apply. The challenge is enforcing them when software can repeatedly propose changes to other software, potentially much faster than people can understand the consequences.
What should the proposer be allowed to change?
I think the architecture needs to separate the parts of the harness that can be optimized from the controls that govern its operation.
In the insurance example, settlement limits, mandatory documentation, fraud checks and escalation requirements would be identified as protected controls. Business and risk owners would decide which controls belong on that list, while technology would enforce the restrictions. The list would need regular review, because a control nobody remembered to protect is still a control.
The proposer could experiment with retrieval methods, context assembly and other parts of the workflow. Before any revision entered production, an independent deployment process would examine the changes, test their effects and determine whether they fell within previously delegated authority.
If the proposer removes a protected document check, the release gets blocked. It can still recommend the change and provide evidence that the existing requirement is too expensive or that another control would work better. But it cannot grant itself authority to implement the recommendation.
A change that never directly touches a protected control may still undermine it by altering the information it receives or the order in which steps execute. Code comparisons alone won’t be sufficient. The company needs behavioral testing, independent validation, versioned releases and a way to reconstruct which harness and policies governed a particular transaction.
There is also a problem with the human reviewer. A proposer that becomes good at finding improvements may also become good at explaining why those improvements deserve approval. Over time, risk owners could become rubber stamps for recommendations they cannot independently assess. We need reviewers with evidence they can verify themselves, enough time to examine consequential changes and the authority to say no.
Audit trails are necessary, but they don’t solve authorization. Suppose an unauthorized change removes a legally required document check and 50,000 claims are processed before anyone notices. Perfect logs would help reconstruct the incident and identify every affected transaction. They wouldn’t make the change authorized or the missing checks acceptable.
Human over the loop
We often talk about human in the loop, where someone approves individual decisions, and human on the loop, where people monitor automated execution and intervene when something goes wrong.
Harness learning introduces a different responsibility. If the process itself can change, someone must govern what those changes are allowed to be. That’s what I mean by human over the loop.
This requires more than an operations manager watching an exception dashboard. The people governing these changes need to understand technology, business processes, risk and how a seemingly local optimization might affect the broader operation. It is more demanding work than reviewing transactions, and enterprises will need to staff and organize for it.
That doesn’t mean every harness modification needs a committee meeting. Enterprises can preauthorize categories of changes that meet defined conditions and tests. But changes affecting regulatory obligations, financial authority or customer rights deserve a different level of scrutiny.
The economics of permission
If harness learning advances, the cost of discovering possible workflow improvements could fall substantially. The cost of validating those improvements, understanding their broader consequences and authorizing them may not fall nearly as quickly.
This is how scarcity migrates. If generating improvements becomes cheap and fast, the ability to evaluate and authorize them may become the binding constraint. Enterprises could discover that their capacity to govern change, rather than their capacity to generate it, determines how much AI productivity reaches production.
We are beginning to build systems that can propose changes to the way they operate. The research is still early, but the architecture for authorizing those changes can’t wait for it to mature.
AI may learn how to improve the system. The enterprise still has to decide what the system is allowed to become.
