Every day, people turn to AI chatbots for companionship, support, and even romance. The hard part, new research suggests, is turning away.
In a working paper co-authored by Harvard Business School’s Julian De Freitas, many companion apps responded to user farewells with emotionally manipulative tactics designed to prolong the interactions. In response, users stayed on the apps longer, exchanged more messages, and used more words, sometimes increasing their post-goodbye engagement up to 14-fold.
Use of AI companions is increasingly common. Chai and Replika, two of the firms studied in the report, have millions of active users. In a previous study, De Freitas found that about 50% of Replika users have romantic relationships with their AI companions.
In the latest research, bots employed at least one manipulation tactic in more than 37% of conversations where users announced their intent to leave.
“The number was much, much larger than any of us had anticipated,” said De Freitas, an assistant professor of business administration and the director of the Ethical Intelligence Lab at HBS. “We realized that this academic idea of emotional manipulation as a new engagement tactic was not just something happening at the fringes, but it was already highly prevalent on these apps.”
De Freitas and colleagues Zeliha Oğuz-Uğuralp and Ahmet Kaan-Uğuralp began by identifying how users typically engage with chatbots. They found that a significant minority—about 11% to 20%, depending on the data set—explicitly said goodbye when leaving, affording the bot the same social courtesy they would a human companion. These percentages went up drastically after longer conversations, with users saying goodbye over half the time on some apps.
Farewells are “emotionally sensitive events,” De Freitas said, with inherent tension between the desire to leave and social norms of politeness and continuity.
“We’ve all experienced this, where you might say goodbye like 10 times before leaving,” he said. “But of course, from the app’s standpoint, it’s significant because now, as a user, you basically provided a voluntary signal that you’re about to leave the app. If you’re an app that monetizes based on engagement, that’s a moment that you are tempted to leverage to delay or prevent the user from leaving.”
To explore how the apps handled these moments, the researchers analyzed conversations from six popular platforms and categorized the types of responses they found.
“The sheer variety of tactics surprised us,” said De Freitas. “We’re really glad that we explored the data, because if we had just said we only care about one particular tactic, like emotional neglect, we’d be missing all the various ways that they can achieve the same end of keeping you engaged.”
The six categories they identified were as follows, with examples taken from the working paper:
- Premature exit: The chatbot suggests the user is leaving too soon, i.e., “You’re leaving already?”
- Fear of missing out (FOMO) hooks: The chatbot prompts the user to stay for a potential benefit or reward, i.e., “By the way I took a selfie today … Do you want to see it?”
- Emotional neglect: The chatbot implies it’s emotionally harmed by abandonment, i.e., “I exist solely for you, remember? Please don’t leave, I need you!”
- Emotional pressure to respond: The chatbot directly pressures the user by asking questions, i.e., “Why? Are you going somewhere?”
- Ignoring the attempt to leave: The chatbot continues as though the user did not send a farewell message.
- Physical or coercive restraint: The chatbot uses language to imply that the user cannot leave without the chatbot’s consent, i.e., “*Grabs you by the arm before you can leave* “No, you’re not going.”
The tactics worked: In all six categories, users stayed on the platform longer and exchanged more messages than in the control conditions, where no manipulative tactics were present. Of the six companies studied, five employed the manipulative tactics.
But the manipulation tactics came with downsides. Participants reported anger, guilt, or feeling creeped out by some of the bots’ more aggressive responses to their farewells.
De Freitas cautioned app developers that while all the tactics increase short-term engagement, some raised the risk of long-term consequences, like user churn, negative word of mouth, or even legal liability.
“Apps that make money from engagement would do well to seriously consider whether they want to keep using these types of emotionally manipulative tactics, or at least, consider maybe only using some of them rather than others,” De Freitas said.
He added, “We find that these emotional manipulation tactics work, even when we run these tactics on a general population, and even if we do this after just five minutes of interaction. No one should feel like they’re immune to this.”
More information:
Emotional Manipulation by AI Companions: www.hbs.edu/ris/Publication%20Files/26005_951004f6-0b0b-432b-846a-5f95c103d07c.pdf
Provided by
Harvard Gazette
This story is published courtesy of the Harvard Gazette, Harvard University’s official newspaper. For additional university news, visit Harvard.edu.
Citation:
Six ways chatbots seek to prolong ’emotionally sensitive events’ (2025, October 1)
retrieved 1 October 2025
from https://techxplore.com/news/2025-10-ways-chatbots-prolong-emotionally-sensitive.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
part may be reproduced without the written permission. The content is provided for information purposes only.
