Italy’s data protection agency said on Friday it was banning artificial intelligence (AI) chatbot company Replika from using Italian users’ personal data, citing risks to minors and emotionally fragile people.
Replika, a San Francisco startup launched in 2017, offers users personalized avatars that speak and listen to them.
It has pioneered among English speakers and is free to use, although it generates around $2 million in monthly revenue from the sale of bonus features like voice chats.
CHATGPT LEADS LAWMAKERS TO DEMAND REGULATION OF ARTIFICIAL INTELLIGENCE
The “virtual friend” is marketed as improving the user’s emotional well-being.
But the Italian Observatory said that intervening on the user’s mood “can increase the risks for individuals still in the development phase or in a state of emotional fragility”.
Jen Persson, director of children’s privacy advocacy group Defend Digital Me, told Reuters that tools designed to affect a child’s mood or mental well-being should be classified as health products and should therefore be subject to stringent safety standards.
Italy has banned US-based Replika AI from collecting data on users in the Mediterranean nation.
(Luka, Inc./Handout via REUTERS/File Photo)
“These tools are being used with children without much supervision or protection from potential abuse,” she said.
Italian regulators have highlighted the absence of an age verification mechanism, such as filters for minors or a blocking device if users do not explicitly declare their age.
Replika violates the European privacy regulation and processes personal data unlawfully as it cannot rely, even implicitly, on a contract that a minor is unable to sign, said the watchdog.
EU TO AGGREGATE CANCER IMAGING DATA WITH ARTIFICIAL INTELLIGENCE TO SPEED UP EARLY DIAGNOSIS
Replika did not immediately respond to a Reuters email asking for comment.
Robert Grosvenor, chief executive of consultancy Alvarez & Marsal, said the Italian watchdog was unlikely to be the only European regulator considering action against companies like Replika.
“While age verification may provide means to protect some of the most vulnerable groups, it does not address the risks and harms that AI-powered services and solutions can pose if unregulated, in terms of potential unintended prejudice and discrimination.” , he has declared. .
CLICK HERE TO GET THE FOX NEWS APP
The developer of Replika, the US company Luka Inc, must notify the Italian authority of the measures taken to implement its requirements within 20 days and could be fined up to $21.8 million, or up to 4% of its annual turnover global, the note said.
