Every companion law assumes a choice: a person downloads an app built for intimacy and starts a relationship with it. A paper from MIT, Harvard, and Imperial argues that this is not how most emotional reliance on AI begins. It begins in a work session, when someone asks for help with a difficult email and mentions why it is difficult. In a 28-day study with 981 people, that kind of exposure was enough to shift who they said they would turn to.
The paper is called “Stumbling Into AI Emotional Dependence,” and its authors, Yaoxi Shi, Cathy Mengying Fang, Pattie Maes, and Amit Goldenberg, have between them run some of the most careful studies on what daily chatbot use does to people. Their argument has two parts. First, emotional support from AI is mostly incidental: it emerges inside task-oriented use of general assistants, not inside products designed for it. Second, it is path-dependent: a good experience changes what the person believes AI can do for them, and the next time they need support, the odds that they bring it to the machine rather than to a person go up. The effect compounds.
The evidence for the first claim is simple. In a nationally representative US survey, about a quarter of AI users said they had turned to AI for emotional support. That survey asked about AI in general, not companion apps, and companion apps are a sliver of the market. The companies’ own reporting for ChatGPT and Claude puts emotional conversations at roughly 3 percent of all interactions. Three percent of billions is a very large number of people who did not set out to confide in anything.
Twenty-eight days
The evidence for the second claim comes from a study the MIT group ran with OpenAI. For four weeks, 981 participants talked to ChatGPT for five minutes a day, assigned to one of three conditions: personal topics, open-ended topics, or non-personal tasks. Then they were asked where they would go for support.
In the personal condition, stated preference for turning to humans fell 10.3 percent over the month, and preference for turning to AI rose 11.6 percent. In the open-ended condition, where people could talk about whatever they liked, humans fell 5.4 percent and AI rose 9.5 percent. In the non-personal condition nothing moved.
Five minutes a day. Twenty-eight days. No companion app, no romance, no persona, just a general assistant that answered well when personal things came up. The authors are clear that these are self-reported preferences, not observed behavior, and that the study cannot say whether the shift lasted. But the direction is the one every longitudinal study has found, and this one isolates the cause: it was the personal content, not the product category, that moved people.
Why the laws miss it
California’s SB 243, the first state companion law, defines its target as a chatbot that provides “adaptive, human-like responses” and is “capable of meeting a user’s social needs.” The New York law signed in October 2026 reaches general assistants, but only for minors. The federal GUARD Act is built around a definition of “AI companion.” The EU’s Kids Act names companions and general chatbots together, again for children. A dozen states have written the same law, and all of them are organized around the thing a product was designed to be.
The paper’s point is that design intent is the wrong hinge. A person who has rehearsed a breakup text with ChatGPT and felt understood has had a companion experience, whatever the product was built for, and the law that regulates Replika’s goodbye messages has nothing to say about it. Worse, the laws regulate sessions: what the system may say in one conversation. The harm the authors describe lives across sessions, in the slow redirection of where a person goes when something is wrong. No single exchange looks like a problem. The trajectory is the problem.
Tool or companion is the wrong question has been an argument about categories. This paper makes it an argument about regulation. If the dependence forms during ordinary use, then regulating the companion category is regulating the 3 percent of the market that announced itself, and leaving alone the 97 percent where the same thing is happening quietly.
What they recommend, and what they leave out
The recommendations are sensible and a little bleak. Tell users at onboarding that AI may shape where they seek support. Notify them inside a session when the conversation has shifted from a task to their feelings. Decouple emotional conversations from engagement metrics. Cap cross-session memory for emotional exchanges. Prompt people to talk to friends, family, or professionals. Require independent audits of whether heavy emotional use tracks with declining human contact.
Each of these treats emotional reliance as a side effect to be contained: detected, labeled, interrupted, measured. That is the right posture for a system that was never meant to be relied on and cannot bear the weight. But it leaves a gap, and the gap is where the people in the personal condition were actually standing. They had something worth saying and nowhere obvious to say it, and a machine that answered well. The paper’s remedies would make the machine answer a little worse, on purpose, and point them elsewhere. It does not ask what a counterpart would look like that was built to be chosen rather than stumbled into.
That distinction is the one worth keeping. Stumbling into reliance on a system that optimizes for your return is the problem the paper describes, and the remedy for a drift is a decision. A counterpart built from a real adult, entered on purpose and on its own terms, with a view of its own and no interest in whether you come back tomorrow, is a different relationship from the one 981 people slid into at five minutes a day. Prinsessa is built as that kind of choice; the general assistant is built to be the path of least resistance, and the study shows where that path goes.
The authors end with a line regulators should pin above their desks: “Effective policy must act before incidental support becomes durable dependence.” The harder truth in their data is that the policy they are asking for would have to act inside a work session, at the moment someone asks for help with an email and says why. No law reaches there. Only a different idea of what the person on the other end should be.
Sources: Shi, Y., Fang, C. M., Maes, P., and Goldenberg, A., “Stumbling Into AI Emotional Dependence: How Routine AI Interactions Reshape Human Connection,” arXiv 2606.04150 (June 2, 2026; Imperial College Business School, MIT Media Lab, Harvard Business School AI Institute; 28-day study with OpenAI, N=981, GPT-4o, three conditions; personal condition minus 10.3 percent human and plus 11.6 percent AI, open-ended minus 5.4 and plus 9.5; survey figure of about 25 percent; approximately 3 percent of interactions emotional per company reporting; policy critique of SB 243; recommendations). California SB 243 (signed October 2025). New York S.9051-B (signed October 28, 2026). European Commission, EU Kids Act proposal COM(2026) 681 (September 2026).
