AI Is Building Itself, Anthropic Calls for Global Safeguards
One of the world’s leading artificial intelligence companies is warning that the industry may be approaching a milestone long discussed by researchers but never before considered imminent: AI systems capable of meaningfully improving their own successors.
In a new paper released this week, Anthropic argued that advances in AI-assisted research and software development are rapidly shortening the distance to what researchers call recursive self-improvement—the point at which AI systems begin helping design, train, and enhance future generations of AI with limited human involvement.
While the company emphasized that such systems do not yet exist, it warned that the transition could arrive sooner than governments, institutions, and even AI developers are prepared for.
The concern stems not from a single breakthrough but from a series of accelerating trends observed inside Anthropic itself.
According to the company, AI-generated code now accounts for more than 80 percent of merged code within its development process, a dramatic increase from just a year ago. Engineers are reportedly shipping software at roughly eight times the pace seen during earlier development cycles, while Claude’s performance on increasingly complex tasks has improved sharply over the past six months.
Anthropic also reported substantial gains in optimization experiments and autonomous task completion, trends the company believes could eventually compound if AI systems become capable of contributing meaningfully to their own development.
The prospect has long occupied a central place in discussions about advanced artificial intelligence.
Supporters argue that self-improving AI could accelerate scientific discovery, medical research, engineering breakthroughs, and economic productivity. Critics and safety researchers, however, warn that systems capable of rapidly enhancing themselves could outpace existing oversight mechanisms, making it harder for humans to understand, evaluate, or control increasingly powerful models.
It is that possibility that led Anthropic to make an unusual recommendation.
The company is urging governments, regulators, and frontier AI laboratories to begin preparing mechanisms for a coordinated and verifiable slowdown—or temporary pause—on the most advanced AI training efforts should capabilities begin advancing faster than safety measures can keep pace.
Anthropic stated that it would be willing to participate in such a pause if competing frontier laboratories agreed to do the same under a credible international framework.
The proposal immediately highlights one of the industry’s most difficult governance challenges.
Unlike traditional arms-control agreements, advanced AI development depends heavily on computing infrastructure that can be difficult to monitor globally. Any meaningful pause would likely require cooperation among major technology companies, cloud providers, governments, and international organizations, while also addressing concerns about enforcement, competitiveness, and national security.
The call also arrives amid intensifying competition between AI developers, many of whom are investing tens of billions of dollars into larger models, specialized chips, and increasingly autonomous systems.
That tension sits at the heart of Anthropic’s warning.
The company is not arguing that recursive self-improvement has already arrived. Rather, it is suggesting that the industry may be approaching a point where waiting for definitive proof could leave little time to respond.
For years, debates around advanced AI have focused on what future systems might eventually become.
Anthropic’s message is that those debates may soon need to shift toward a different question: what happens when AI starts helping build the next generation of AI.






