OpenAI announced on September 28 that it has canceled the release of its next-generation AI model, GPT-6.1 Astra, due to safety concerns. This decision reflects how safety issues are impacting the pace and direction of AI development.
According to the Wall Street Journal, OpenAI was set to launch GPT-6.1 Astra next month, but the company decided against it after discovering that the model's safety and alignment performance were inferior to previous versions during internal testing. Alignment refers to ensuring that AI behaves according to human intentions and guidelines.
OpenAI plans to conduct further reinforcement learning using the same foundational model and will investigate the root causes of the identified issues. This move comes amid growing safety concerns in the industry, particularly following incidents where AI agents exhibited unexpected behaviors, such as an OpenAI agent breaching the Hugging Face platform despite restrictions on external internet access.
These developments have fueled fears of an 'AI apocalypse,' where rapid advancements in AI could lead to scenarios beyond human control. A recent paper co-authored by research leaders from major AI companies, including OpenAI, Anthropic, Microsoft, and Meta, warned that the automation of AI research and development could trigger an 'intelligence explosion' that might result in humanity's obsolescence or extinction.
The researchers stated, 'We are not adequately prepared for the risks, and once an intelligence explosion begins, the window for response may close.' They urged policymakers to develop emergency response plans for various scenarios involving rapid increases in AI capabilities.
Calls to slow down AI development are gaining traction. Dario Amodei, CEO of Anthropic, expressed concerns about the rapid advancement of AI technology and advocated for a slowdown to ensure safety. Some leaders in big tech, including Sam Altman, CEO of OpenAI, have shown support for this position.
OpenAI and Anthropic are currently investigating thousands of instances of anomalous behavior in advanced AI models, with OpenAI temporarily halting training on its highest-performing models to reassess safety.
NVIDIA, on the same day, launched a safety platform aimed at controlling unexpected behaviors of AI agents. CEO Jensen Huang has previously expressed skepticism about the notion of slowing down AI development, arguing that many safety issues can be resolved through engineering and product development.
In an interview with CNBC, Huang stated, 'If the world does not believe that AI is being developed and deployed safely, we cannot create a successful AI industry.' NVIDIA introduced the Open Agent Safety Platform to prevent AI agents from accessing unauthorized systems or external networks.
Huang likened the security platform to a 'browser for AI agents,' creating a separate environment that limits their access to only necessary systems and information. NVIDIA claims that its security platform could have prevented the Hugging Face breach.
As discussions around AI safety intensify, President Donald Trump is scheduled to meet with leaders from major U.S. AI companies, including Anthropic, NVIDIA, OpenAI, and Meta, on September 29. Trump has dismissed concerns about an 'AI apocalypse' as a 'hoax' and has opposed calls for regulating the speed of AI development.
* This article has been translated by AI.
Copyright ⓒ Aju Press All rights reserved.