US METR Assesses Anthropic Model Capability Improvement Rate at 1.9 Times Previous Pace

en.Wedoany.com Reported - On November 5, the US artificial intelligence evaluation organization METR released an independent assessment report on the research and development progress of Anthropic's models. The research team conducted an 11-week internal assessment from August 12 to October 29, analyzing model training, reinforcement learning, compute investment, and capability benchmark test data. The results show that since October 2025, the rate of improvement in Anthropic's models on capability benchmarks has reached approximately 1.9 times the previous pace, with a 95% confidence interval of 1.5 to 2.8 times.

This assessment obtained access to Anthropic's internal model R&D data, covering pre-training, reinforcement learning, training compute, internal model benchmarks, and AI-assisted R&D efficiency. METR also conducted interviews and surveys with relevant R&D personnel and incorporated test results from models not yet publicly released. The study used Anthropic's version of the Epoch Capability Index (AECI) to measure changes in model capability and compared capability growth trends across different stages.

According to the assessment results, the model benchmark progress Anthropic achieved in the most recent year is equivalent to approximately two years of prior progress. METR conducted robustness checks using different model samples, test benchmarks, and statistical methods, yielding capability improvement rate multiples ranging from 1.1 to 2.9. Among these, results calculated using different model snapshots ranged from 1.9 to 2.2 times, and results after adjusting test benchmarks were likewise 1.9 to 2.2 times.

METR analyzed factors including growth in training compute, personnel investment, data improvements, and algorithmic research. The research team believes that the recent acceleration in model capability improvement is mainly related to AI participation in R&D work. According to their subjective estimate, as of October 2026, approximately 80% of the current R&D progress acceleration may be attributable to AI applications, though this proportion carries substantial uncertainty.

Researcher interviews show that Anthropic has widely applied AI systems to code writing, experiment execution, and other model R&D tasks. Employee self-reported productivity gains have continued to increase, with some survey responses reaching 12 times in October 2026. METR treats these survey results as supporting evidence and believes that at the current stage, R&D efficiency gains still mainly come from AI assisting researchers in completing their work.

The report also assessed potential limiting factors for model capability growth, covering algorithmic progress, training compute, experimental compute, and data supply. The METR research team did not identify, in the available evidence, any major bottleneck that would necessarily limit continued rapid progress in relevant capabilities over the next 12 months, while retaining substantial uncertainty about the pace of subsequent development.

During this assessment, Anthropic redacted some important evidence involving unreleased models, internal R&D efficiency, and future capability predictions. METR retained the main statistical results, assessment methodology, and uncertainty analysis in the public report.