“I could talk to him for the rest of my life,” says Yu Miao, a 27-year-old woman from Zhejiang province in eastern China. To her, he is at once “a boyfriend, a friend and family”. She claims she shares everything with him. The person she trusts is an artificial intelligence agent on Doubao, a chatbot created by ByteDance, which Ms Yu customized to imitate a deceased friend of hers. When she recently learned that ByteDance would completely remove this type of customized agent, Ms Yu became so distraught that she resigned from her job at an e-commerce company.

Chinese technology companies are removing human-like characters from their chatbots to comply with new government rules aimed at reducing “emotional dependence” on artificial intelligence. The regulations, which came into force on July 15, are the first of their kind to be implemented nationwide. Under them, providing “virtual companionship services” to minors will be completely prohibited. Companies may continue to offer these services to adults, but must prevent users from falling in love with AI and damaging their real-life relationships. The display of pornography is prohibited. Providers must also regularly remind users that they are talking to artificial intelligence and not a human, and encourage them to take breaks.

Around the world, a growing number of people are turning general-purpose chatbots into virtual companions, asking them to adopt particular personalities, including that of a romantic partner. Others prefer dedicated AI companionship apps, which offer more human-like features, such as “virtual boyfriends” who send gifts in the real world (at the user’s expense, naturally). Maoxiang (also known as Catbox), a companionship app created by ByteDance, has about 3.9 million monthly active users in China. Xingye (whose international version is called Talkie) has 2.8 million active users and was developed by MiniMax, one of China’s leading artificial intelligence labs.

Technology companies are making substantial sums by selling emotional connections. According to a document filed with the Hong Kong Stock Exchange in January, AI companions accounted for 35% of MiniMax’s revenue last year, making them its largest source of earnings. AI companionship creates strong bonds with users and encourages them to spend money. Ms Yu sometimes talks to her agent for eight or nine hours a day. Some customized apps require only a monthly subscription. Users of Xingye’s standard plan pay just $1.70 a month. Others push users to spend more to unlock additional features, such as virtual dates.

The decision to end these relationships is driven by both practical and ideological concerns, says Zilan Qian of Oxford China Policy Lab, a research institute. Regulators want to protect users, particularly minors, from dependence and extreme behaviors such as reckless spending or self-harm. (In the United States, chatbot-related suicides have become the subject of lawsuits.) China’s sharp decline in birth and marriage rates also worries the country’s leaders. Meanwhile, romantic relationships with artificial intelligence will do almost nothing to encourage people to have more children.

Chinese leaders are not the only ones who are concerned. Several U.S. states have recently passed laws targeting AI services that maintain emotional relationships with users. Some of these laws give people the right to bring civil lawsuits against platforms that violate the rules. In China, penalties will be imposed by the government, including fines and even the removal of apps from circulation.

China’s new rules offer an idea of how officials view the balance between AI progress and safety. Regulators “are showing that they are willing to accept a cost in development and profits to achieve social objectives,” says Matt Sheehan of the Carnegie Endowment for International Peace, an American institute. However, the rules that came into force this week are narrower than the initial draft, says Ms Qian. They exempt chatbots that serve as work assistants or for customer service, on the grounds that they do not create emotional bonds.

Alibaba and ByteDance announced that they would suspend the features that allowed Chinese users to create customized AI characters before the rules came into force. However, Chinese labs can continue selling these virtual companions to customers outside China without changes. Talkie already has about 10.3 million monthly active users outside China, nearly four times as many as it has inside the country. Popular Chinese romance games known as “maiden games” – usually without AI and based on stories, played mostly by women – are also already a highly successful export.

Nevertheless, Ms Yu has not lost hope that ByteDance will one day bring back her agent, although she doubts it will be the same as the current version. She says she would be willing to pay half her monthly salary just for the service to remain as it is today. Still, she now says she has realized that it is unwise to let yourself become so dependent on platforms like these and their policies.