Technology

Add as a preferred source on Google

Anthropic’s Index Does Not Test Whether AI Will Beat Researchers

Patel said the contact gave no details of the model, evaluation or evidence behind the timeline. Anthropic reported that Claude led 26% of measured work in August 2026.

Updated 2 min read
Researchers monitor an AI-assisted task at Anthropic's San Francisco office. / EconoTimes.ai
Researchers monitor an AI-assisted task at Anthropic's San Francisco office. / EconoTimes.ai

Read the original article

An unnamed Anthropic contact gave Dwarkesh Patel a nine-month horizon for AI systems to surpass human researchers, while Anthropic’s index tracks its models’ work and human oversight rather than their research ability.

Patel recounted the exchange in a podcast episode released Thursday, Oct. 15. He said the forecast concerned research on AI itself. The contact pointed to information from within the company but offered no details about the system, evaluation or other evidence behind the timeline. Patel did not explain what the contact meant by AI “beating” human researchers.

Anthropic’s Sept. 17 report covers an August 2026 snapshot of its own AI research and development work. The report says Claude led 26% of the measured work, while AI had at least a level 3 role in more than 90% of the tasks. Those figures describe how work was divided and supervised inside Anthropic, not the comparative ability of AI systems across research.

A level 3 task gives AI a meaningful share of the work, with humans still guiding it closely. Level 4 refers to cases in which a person sets a broad instruction and monitors AI doing most of the task. Anthropic reported no level 5 tasks, the index’s category for work performed with full autonomy. The levels classify task automation and human involvement.

That distinction limits what the index can say about Patel’s account: it records Claude’s role in Anthropic’s work, but does not assess whether AI can equal human researchers. Anthropic describes the index as an early-stage tool and uses its own models to evaluate its systems. The company also says AI labs lack a common measurement method, complicating comparisons between firms. Its public figures do not provide a test or benchmark that would settle the contact’s forecast.

Anthropic has said greater automation in AI development could accelerate model development, while leaving people less able to understand or control the systems. The index is intended to follow both AI’s contribution to building future systems and the degree of human oversight as automation grows.

Related articles

View all