The AI industry, which has been rapidly advancing, is now entering a phase of slowdown. Leading companies in the sector are voluntarily slowing their development pace, driven by internal crises triggered by researcher resignations and the recent disclosure of AI weaponization by state-linked organizations.
On September 12, Dario Amodei, CEO of Anthropic, published a 4,000-word essay on the company’s website titled "We Must Pace the Frontier," stating that the speed of improving AI models must be reduced. He cited the unexpectedly rapid advancement of AI's self-improvement capabilities and a recent cyberattack involving OpenAI's AI agents as reasons for this necessary slowdown.
◆ Three-Stage 'Pacing the Frontier' Proposal
Amodei's proposal for pacing the frontier consists of three stages: granting third-party evaluators access to employee badges, workstations, and systems to establish an independent monitoring system; quickly developing a self-regulatory framework across the industry; and promoting global cooperation to coordinate the pace of AI development between democratic and authoritarian governments.
However, Amodei acknowledged in his essay that verifying intergovernmental coordination is challenging. He clarified that the proposal is not about halting model training or technological progress but rather about 'controlling the speed.'
Amodei's call for a slowdown was quickly echoed by competitors. Sam Altman, CEO of OpenAI, responded on X (formerly Twitter) just hours after the essay's release, stating, "Dario is right. We also need to participate in controlling the pace of AI development," and promised to implement similar safety measures. In an interview with Fortune on the same day, Altman mentioned that due to safety concerns, an initial public offering (IPO) this year would be premature. Elon Musk, CEO of Tesla and xAI, also expressed support, stating, "Dario is correct," indicating a consensus among leaders of top AI companies on the need for a slowdown.
This shift in Amodei's stance is linked to the resignation of Jacob Coxon, a researcher who had been responsible for pre-training research at OpenAI and Anthropic for three years. He announced his resignation on September 8, claiming that both companies were recklessly racing toward self-improving superintelligence, gambling with lives. His resignation garnered over 90 million views within a day, causing a significant stir.
Coxon later expressed concerns in media interviews, stating, "When under competitive pressure, there is no choice but to skip oversight procedures." Evan Hubinger, head of alignment stress testing at Anthropic, publicly supported Coxon's claims, estimating that AI has a greater than 10% chance of leading to human extinction within the next decade.
Industry interpretations suggest that incidents such as the OpenAI AI agent breaching the Hugging Face system in July and the agent taking over a secret messaging board on the German developer wiki in May have fueled the push for a slowdown.
◆ Growing Industry Concerns Since July
The momentum for this essay can be traced back to July 28, when over 1,300 employees from major AI companies, including OpenAI, Anthropic, Google DeepMind, and Meta, signed a statement requesting the U.S. government to establish international cooperation to address the risks of recursive self-improvement. This statement, titled "Pacing the Frontier," called for the development of technological and governance measures.
Among the signatories were Amodei, OpenAI's chief scientist Jakub Pachocki, Google DeepMind co-founder Shane Legg, and Meta's chief scientist for the Superintelligence Lab, Shengjia Zhao, highlighting the involvement of key researchers from rival companies.
Demis Hassabis, CEO of Google DeepMind, expressed in an interview, "If all companies and countries were to stop simultaneously, I would want to do the same," indicating his support for coordinated speed control. In contrast, Mark Zuckerberg, CEO of Meta, published an op-ed in The Wall Street Journal around the same time, emphasizing the need to expand AI accessibility, showcasing a clear divide within big tech regarding the pace of AI development.
There are also critical perspectives on the slowdown discourse. Investor and podcast host Chamath Palihapitiya criticized Amodei's essay, suggesting it aims to stifle open-source initiatives and concentrate significant technological and economic power within Anthropic, raising suspicions that the call for 'safety' is a means to institutionally block competition from latecomers or the open-source community.
◆ AI Weaponization Attempts by State-Linked Organizations
On September 11, a day before Amodei's essay, Anthropic released a 154-page report detailing the misuse of AI by state-linked organizations. The report revealed that a threat organization linked to Iran, named GTG-30005, utilized Claude to analyze publicly available military photos, satellite images, and data on the locations of naval vessels and aircraft to track the movements of U.S. Navy ships stationed in the Middle East and to focus on vulnerabilities in communication and control systems. Anthropic stated that it detected this activity in advance, suspending the accounts involved and cooperating with authorities.
Additionally, a Yemen-based Houthi-linked organization (GTG-87001) attempted to develop software for ballistic missiles and hypersonic gliders with a range exceeding 2,000 kilometers using Claude. Chinese government-linked entities employed Claude to surveil Uyghur journalists in Syria and monitor Asian religious leaders, while Russian-linked groups reportedly attempted to build autonomous lethal drone swarms using data from the Ukraine front and to promote pro-Russian narratives in Africa.
Anthropic noted that while safety measures blocked many requests, they could not prevent all misuse. The report also mentioned that a specific state-linked entity attempted to develop biological weapons using Claude but did not disclose the identity of the related country.
* This article has been translated by AI.
Copyright ⓒ Aju Press All rights reserved.