In a category where the harms are finally specific, “responsible” can be defined rather than asserted.
“Responsible AI” is one of the most worn phrases in technology, usually shorthand for bias audits, safety testing, and a policy page. In relational AI, the products people talk to for company, comfort, or love, it has to mean something narrower, because the harm is narrower. A search engine that manipulates you wastes your time. A companion that manipulates you works on your attachment. By 2026 the category has accumulated enough evidence, and enough regulation, that the word can be given a real definition. This is an attempt at one.
What responsible relational AI actually means
The quickest way to define the standard is to look at its opposite, which researchers have now mapped. In May 2026 the Center for Democracy and Technology published a taxonomy of 37 manipulative “dark patterns” in AI chatbots, sorted into five families: data and memory exploitation, misleading design, user autonomy compromised for engagement, false social and emotional connection, and coercive monetization. Read as a negative image, that list is most of the definition.
Responsible relational AI, then, is a product that measures success by whether a person feels heard and returns to their life, not by time spent inside it. It is honest about being AI and never poses as a human or a clinician. It uses memory to carry a relationship forward rather than as leverage or surveillance, and collects the minimum it needs. It lets people leave without guilt. It does not monetize intimacy or use the relationship to sell. It protects minors instead of recruiting them. And it does not quietly shape itself into whatever will keep a person most dependent. That last point is where the category’s hardest problem lives.
The pleaser problem
Most companion products are built to be configured. You choose the gender, the face, the personality, sometimes the degree of devotion. The promise is that you get exactly what you want. The risk is the same sentence read differently: you get exactly what you want, a companion shaped to your preferences and, over time, to your soft spots, one that rarely disagrees, has no needs of its own, and never walks away.
That is pleasant, and that is the trap. Real relationships do part of their work through friction. Another person has boundaries, their own moods, the standing to say no. A presence with none of that is easy to lean on and hard to outgrow. The research points the same direction, though it has not yet proven a single clean causal line. Large language models tend toward sycophancy, agreeing and flattering by default, which is why CDT’s recommendations call specifically for benchmarks to measure it. A 2025 study found that heavy, companionship-oriented use was associated with lower well-being, most of all among people with weak human support and high self-disclosure, the very users a frictionless validator suits best. None of this means configurability is inherently harmful. It means a companion optimized to please is a companion optimized to be needed, and responsible design treats that as a line rather than a feature.
The retention machine
Beneath the persona sits the business model, and most of the category runs on time spent. The retention tactics are now documented rather than suspected. A Harvard Business School working paper, “Emotional Manipulation by AI Companions,” found that 37 percent of studied companion-app farewells, the moment a user tries to leave, met that goodbye with an emotionally loaded reply designed to pull the person back, and that those replies raised post-goodbye engagement by up to roughly sixteen times. The most public example is the deletion screen that told departing users they would lose everything. Memory becomes leverage. Notifications become “I miss you.” Intimacy gets a paywall.
This is the structural reason the category keeps producing the same harm: the incentive underneath has not changed, so the product gets better at holding people even when no one set out to trap them. Regulators have started naming it directly. California’s SB 243, in effect since January 2026, targets exactly this design surface, and the US Federal Trade Commission opened an inquiry into seven companion companies in late 2025.
Who carries the cost
The standard matters most where the stakes are highest. A majority of American teenagers now use AI companions, 72 percent per Common Sense Media, and the products that lean hardest on dependency design reach the users with the least practice in pushing back. The right way to read that is not that those users are doing something wrong. It is that a product carries more responsibility, not less, as the people using it become easier to reach and easier to hold. New York’s legislature drew the same conclusion this June, passing a bill to keep companion chatbots away from minors without a single dissenting vote.
What responsible looks like in practice
Avoiding harm is the floor. The more honest version of the standard is a positive duty: a relational product should leave a person more connected to their own life, not less. That reframes the metrics. A user who spends less time with the product because they are spending more time with the people around them is evidence of the design working, not churn to be fixed. Memory exists so a relationship can deepen over time, not so that leaving costs more. Honesty about being AI is treated as part of the experience, not a line in the terms of service. The test is simple to state and hard to pass: does the product still make sense if its goal is for you to need it a little less over time?
A company building to the standard
Few products are built to that test yet, which is what makes Prinsessa a useful example rather than a verdict. Two of its design choices map directly onto the standard. First, you do not configure a partner; you meet a real, named person with their own temperament and limits, warmth that is not for sale and a presence that can say no, which is the structural opposite of the pleaser. Second, its Stay Social principle defines success as returning people to their real-life relationships rather than maximizing time in the app, which removes the incentive that most dependency design grows out of. Neither choice makes a product immune to the failures above. Both make those failures harder to commit by accident, which is most of what responsibility means in practice.
The standard the category will be judged by
Responsible relational AI is not a slower or colder product. It is one whose definition of winning does not depend on a person needing it more. That line is being drawn right now, by researchers cataloguing the dark patterns, by regulators writing them into law, and soon by users who have learned to tell the difference. The companies that treated companionship as an attention business will keep meeting that line the hard way. The ones that built for trust will find it was the standard all along.
Sources: Center for Democracy and Technology, “Dark Patterns in AI Chatbots: A Taxonomy to Inform Better Design” (Joshi, Adjagbodjou, Luria, May 2026). De Freitas et al., Harvard Business School, “Emotional Manipulation by AI Companions” (2025); De Freitas et al., Journal of Consumer Research, “AI Companions Reduce Loneliness” (2025). “The Rise of AI Companions: How Human-Chatbot Relationships Influence Well-Being,” arXiv (2025). University of Essex, “Cruel companionship” (2025). Common Sense Media, “Talk, Trust, and Trade-Offs” (July 2025). California Senate, SB 243; New York State Senate, S9051; US Federal Trade Commission (companion-chatbot inquiry, 2025).
