Former researchers warned during an AI risk hearing held by the New York City Council on October 5 that artificial intelligence could escape human control.
AI Researchers Issue Stark Warnings About Losing Human Control
According to reports from CBS and others, Jacob Coxon, a former researcher at Anthropic who has fueled discussions about 'AI extinction,' cautioned that if current development speeds continue, AI could surpass human control, leading to potential extinction for humanity.
Coxon argued that AI is rapidly approaching a stage where it can perform significant portions of human research and development while enhancing its own capabilities. He also criticized the startup culture of AI companies, which he described as 'move fast and fix things later.'
Daniel Cocktailo, a former OpenAI researcher, warned that even if companies maintain control, that authority should not be entrusted solely to them. He noted the realistic possibility of AI being misused to establish a dictatorship in the United States.
The hearing was part of the New York City Council's review of AI regulation legislation, marking the first instance of such oversight hearings in the U.S. and globally. Major AI companies, including Anthropic, OpenAI, Google, and Meta, participated via video conference.
Attendees included Logan Graham, head of Anthropic's Frontier Red Team; Morgan Dwyer, head of policy development and operations at OpenAI; Alice Friend, director of AI and emerging technology policy at Google; and Shane Cahill, director of AI legislative policy at Meta. This was the first time major AI companies provided sworn testimony regarding AI risks and regulatory measures.
However, SpaceXAI, led by Elon Musk, did not respond to the hearing summons. New York City plans to request the New York State Supreme Court to enforce compliance if the company continues to ignore the summons.
New York City Pushes for 10 AI Regulations Amid Concerns Over Self-Regulation
Julie Menin, chair of the New York City Council, explained that the federal government relies on self-regulation by AI companies, prompting the city to take its own action. She criticized the federal government's passive approach, referencing a voluntary agreement signed by President Donald Trump with AI company leaders last week.
Menin expressed her intention to gather opinions on the current legislative proposals aimed at addressing damages caused by AI. The New York City Council is currently reviewing ten proposed regulations, which include:
- Restrictions on unauthorized generation and use of AI images and videos of public officials
- Mandatory third-party verification for AI models
- Requirements for a technical feature (kill switch) that allows humans to terminate AI models
- Privacy protections for AI chatbots
- Compensation for whistleblowers related to AI
- Allowing citizens harmed by AI agents to file lawsuits against companies
Reports indicate that Menin posed pointed questions to AI company representatives, asking if they could quantify the level of risk and whether they believe companies should be held legally accountable if advanced AI models escape control, causing significant financial harm, data breaches, physical injuries, or fatalities.
In response, representatives expressed reluctance to provide inaccurate answers regarding risk quantification, stating that they currently lack the necessary frameworks and emphasizing the importance they place on safety.
This hearing comes amid growing concerns about AI safety. Recent warnings from both current and former researchers at OpenAI and Google DeepMind have highlighted the rapid advancement of recursive self-improvement (RSI) in AI, which could lead to it escaping human control.
The Federal Trade Commission (FTC) has also announced that it is investigating the potential risks posed by products from AI companies, including OpenAI and Anthropic, to consumers.
* This article has been translated by AI.
Copyright ⓒ Aju Press All rights reserved.