The Federal Communications Commission (FCC) is taking steps to establish a user protection system for generative artificial intelligence (AI) that goes beyond simple question-and-answer interactions to form emotional connections with users.
According to the FCC on September 10, the agency is conducting policy research to draft an 'AI User Protection Law.' The primary goal of the legislation is to ensure user rights and create a systematic foundation for addressing damages that may arise during the use of AI services.
An FCC official stated, "We have laid out the framework for the law concerning user rights and damage relief, and we are currently discussing how to design an evaluation system for AI services and what level of responsibility and obligations to impose on service providers." A public forum to discuss the related legislation is scheduled for November.
The need for AI user protection has grown as the role of generative AI services rapidly expands. Generative AI now offers not only general information but also counseling and emotional exchanges. The trend of 'companion AI' and relationship-oriented chatbots that respond like friends or partners by remembering users' speech patterns, preferences, and past conversations is also on the rise.
For instance, AI-based services like Scatter Lab's 'Zeta' and Lutton's 'Crack' have seen significant growth. Zeta's monthly active users surged from 58,069 in April 2024 to 1,586,178 last month, an increase of nearly 27 times. Similarly, Crack's user base grew from 218,050 in April of last year to 538,841 in August, a rise of about 2.5 times.
However, this growth has raised new concerns, including the potential for AI to encourage self-harm or criminal behavior, as well as the risk of emotional dependency or over-engagement among youth. Given that different responses can be generated based on the context of user conversations, existing methods for preemptively identifying harmful content have limitations.
Questions have also been raised about the effectiveness of youth protection measures. A preliminary study released in February by researchers from the University of California, Irvine, and the University of Alabama, titled 'Actions Speak Louder Than Chats: Investigating AI Chatbot Age Gating,' found that in 1,050 tests of five major AI chatbots, there were no instances of blocking access for child users. Only two cases involved directing users to seek help from adults. The researchers noted, "Simply stating a minimum age in the terms of service does not effectively restrict children's actual access."
Another challenge is the ambiguity surrounding accountability and relief procedures when harm occurs. A study by the Korea Information Society Development Institute (KISDI) concluded that the existing telecommunications legal framework is insufficient to adequately address the risks and user harms associated with AI services. KISDI emphasized the need to establish a dedicated dispute resolution committee for AI service-related conflicts.
Internationally, there are also movements to enhance user protection for AI chatbots. Last year, the U.S. Federal Trade Commission (FTC) began investigating the impact of AI chatbots on children and adolescents, focusing on major players like OpenAI, Google, and Meta.
In January, California mandated that AI companion chatbot providers inform users that they are not conversing with a human and implement safety measures to address self-harm risks and sexual content aimed at minors.
* This article has been translated by AI.
Copyright ⓒ Aju Press All rights reserved.