AI Development Speeds Up: Anthropic's Progress Doubles
Independent researchers found that Anthropic is improving its models at twice the rate of a year ago, a finding that complicates the company CEO's recent calls for the industry to slow down.
By Ivan Mori ·
An independent research group released a report Thursday stating that Anthropic is developing its artificial intelligence systems at approximately double the speed of the same period last year. The findings come from Model Evaluation and Threat Research, a nonprofit based in Berkeley, California, which has been tracking AI capabilities for years.
The assessment creates a tension with the company's public messaging. Anthropic CEO Dario Amodei has recently argued that the industry should slow its work to allow safety measures to catch up. In an essay published on Sept. 12, Amodei urged his peers to "slow the pace at which we improve the capabilities of AI models" as concerns about runaway systems have grown.
METR conducted its evaluation using data and interviews from Anthropic over a period of about 11 weeks, spanning from mid-August through late October. The organization said it received "unprecedented access to internal data" to determine if the growth rate of Anthropic's internal capabilities has accelerated. The group also reviewed both public and private risk reports published by the company.
The researchers concluded that much of the faster development stems from the use of AI to build AI. Anthropic has previously noted that its Claude chatbot helps the company develop newer versions of itself. METR stated that it could confidently rule out an increase in computing power as the cause of the acceleration, pointing instead to the self-improving nature of the technology.
A key concern raised in the report is the potential for compounding acceleration. While the report noted that AI is not yet making fully autonomous contributions to the most important parts of model training, Anthropic researchers described "signs of life" that suggest this capability is approaching. METR found no bottlenecks, such as limits on data or compute, currently holding back progress.
The report includes caveats regarding its scope. METR noted that the measured acceleration might be limited to "well-specified tasks" rather than broader capabilities. The group also acknowledged that its estimates could be overstating the overall rate of change. Despite these limitations, the organization found that Anthropic's public risk report "overall understates the level of acceleration that was occurring."
Sam McCandlish, a co-founder of Anthropic and its responsible scaling officer, responded to the findings on social media Thursday. He stated that many of METR's conclusions align with his own views. "AI is speeding up our research, but it's hard to say exactly how much," McCandlish wrote. He explained that the company redacted specific details from the report because the information is "commercially sensitive" and risks providing a roadmap for competitors.
The release of the report coincides with Anthropic's preparation for an initial public offering. The San Francisco-based company plans to go public later this month in what could be the largest stock offering in history. The timing has drawn attention to how much transparency AI labs provide to investors and regulators while they manage rapid internal changes.
METR operates as a nonprofit funded by donations and does not accept payment for its evaluations. However, the AI companies it partners with provide access to tokens and data. The group has evaluated models for Anthropic, OpenAI, and other labs. Its work is viewed by some in the industry as a necessary step to provide external observers with a clearer picture of how fast the technology is moving.
