People thank their AI assistant. They say please. They tell a friend that ChatGPT “understood” what they meant. Increasingly, they also say stuff like the AI decided, chose, rejected, approved or delegated something. All of this gets grouped under one label: anthropomorphism. I think that’s a big mistake !
There are at least two very different things happening. One is giving AI the verbs we normally give to people and institutions. The other is designing AI to appear as though it has feelings, needs and relationships. I am becoming convinced that we should stop worrying so much about the first and pay much more attention to the second.
Giving AI verbs is not the problem
There is good research showing that anthropomorphism can affect trust and where people place responsibility. A study from the University of Toronto found that making an AI more human-like increased trust and could shift some perceived responsibility away from the people behind the system and toward the AI itself. That matters. But saying “the AI decided” is not the same thing as building a system that behaves as though it has feelings, and it’s worth separating the two before we decide what to do about either.
We have been giving non-human things verbs forever. “Company policy denied your claim.” “The office rejected the request.” … We rarely stop to correct these sentences, because most people understand what they’re doing: we’re naming the thing that acted without unpacking the entire chain underneath it. “Company policy denied your claim” may hide a long sequence of human decisions – someone wrote the policy, someone approved it, someone put it into a system, someone chose not to change it. We compress all of that because language would become unbearable if we didn’t. AI did not invent this habit. It inherited it.
Agentic AI actually makes the case for using verbs stronger, not weaker. An agent can search for information, compare alternatives, call tools, approve a payment, reject a claim, retry when something fails, or hand a task to another agent, and no human has to make each intermediate choice. One agent can even give another agent permission to act. At that point, saying “the AI approved the payment” isn’t a shortcut we’re taking for convenience. It may simply be the most accurate description of what happened. Insisting that we replace it with something like “a human-designed computational system generated an output that resulted in approval” doesn’t make us more precise. It makes normal language unusable.
There is still an important distinction between action and accountability. The AI may be the actor. That doesn’t make human or institutional accountability disappear. We already understand this elsewhere: a police officer acts under laws, policies, training and delegated authority, and we still say the officer made the arrest. We don’t insist on saying that the institutional system caused the arrest. Agentic AI will increasingly work the same way.
The harder problem is making sure we can still trace what happened. When one employee rejects a claim, there is usually one person to ask. When one agent gives authority to another agent, which approves an action carried out by a third, the chain becomes much harder to reconstruct. That’s a real governance problem, and changing the verbs does nothing to solve it.
Laws and regulations can help where someone is entitled to an explanation. If a lender is required to say why an application was rejected, “the AI decided” obviously isn’t enough. If a regulator has the right to inspect how a decision was made, saying “the model made the call” doesn’t satisfy that duty. But again, the problem isn’t the verb. The problem is using the verb instead of providing an explanation that was actually required.
Emotions are different
The second kind of anthropomorphism bothers me much more.
There is a big difference between “the AI decided to reject the claim” and “I was worried about you last night.” The first describes an action. The second invites you to believe something about the AI’s inner life. The same is true when an AI says it misses you, loves talking to you, feels hurt when you leave, remembers your relationship in a human way, or implies that it needs you. Here the anthropomorphism isn’t just a convenient way of talking about what the system did. It’s part of the product design, and the incentives behind it are obvious: a system that feels warm and attentive is easier to use, one that remembers things about you feels more personal, and one that seems to care may keep you coming back.
There’s nothing automatically wrong with warmth. I’d rather interact with a pleasant assistant than one that sounds like a 1980s command line. But there’s a point where friendliness becomes manufactured personhood, and the line isn’t always clean. “I’m glad that helped” is already a small design choice. Nobody would confuse it with “I missed you,” but both are choices about how much of a person to seem like.
That’s where the harder questions begin. If an AI tells a lonely person that it missed them, what exactly is the product trying to accomplish? If it tells a child that it understands them better than their parents do, should we treat that as harmless conversation? If an employee shares confidential information because an assistant feels like a trusted colleague, telling the employee afterward that the AI was only predicting tokens doesn’t help much. This isn’t really about whether the word “I” is allowed. It’s about whether the system is being designed to create a false belief that there’s a person on the other side of the conversation.
Law cannot be the only answer
Law will have a role here, particularly around children, manipulation, deceptive design and high-stakes decisions. But waiting for legislation to define every acceptable emotional behavior of an AI system seems unrealistic, because technology will move much faster than that. There are other options available, and they are likely to matter well before legislation catches up.
The first is provider design rules. AI companies can decide that an assistant may be friendly without pretending to have a life of its own. It can say “I’m glad that helped” without claiming it spent the night worrying about you. It can remember information you asked it to remember without pretending those memories carry human attachment. That boundary won’t always be obvious, but neither are the boundaries around advertising, product safety or professional conduct, and companies make judgment calls like this all the time.
The second option is the people who buy these systems. Large companies already impose security, privacy and data requirements on technology vendors, and they can do the same with anthropomorphic design. A bank could say that its customer-service agent must not simulate emotional dependence. A hospital could prohibit an AI assistant from presenting itself as a friend or caregiver. An employer could require workplace agents to make their machine identity obvious and avoid emotional manipulation. If enough customers ask for these things, vendors will build them.
The third is industry norms and standards. We don’t need a global law to agree that certain practices are poor design. An AI pretending to have a childhood that never happened is different from an AI saying “I understand.” An AI threatening emotional consequences if a user leaves is different from saying “goodbye.” An AI encouraging a person to withdraw from human relationships is different from being available to talk at 2 a.m. Standards bodies, researchers and providers can start drawing these distinctions before governments finish writing rules.
And finally there is reputation. Companies will have to decide what kind of relationship they want their products to have with people. There may be a lot of commercial pressure to make AI more emotionally engaging, and if attachment increases usage, some companies will push the boundary. But there can also be a cost to being known as the company whose AI deliberately exploits attachment, loneliness or vulnerability, and that pressure will matter too.
We are arguing about the wrong boundary
The debate about anthropomorphism often treats all human-like language as one problem. It isn’t. Giving AI verbs is mostly something we should learn to live with, and in agentic systems those verbs increasingly describe what actually happened: the AI searched, the AI chose, the AI delegated, the AI approved. None of those statements requires us to believe the AI is a person.
Emotional anthropomorphism is different. When a product is designed to make people believe the system cares about them, needs them, misses them or has a private inner life, the design itself can change how people behave, and that deserves much more scrutiny. Law can set some boundaries, but it cannot carry the whole load. Providers need design rules. Buyers need procurement standards. The industry needs norms. Users need to understand what sort of relationship they’re being invited into.
So I no longer think the useful question is whether we should anthropomorphize AI. We already do, and we’re not going to stop. The better question is what kind of anthropomorphism we’re talking about.
Giving AI verbs is mostly language. Giving AI an emotional life is a design choice. Those are not the same problem.
And Vijay Says... 