Anthropic's Opus 5.5 model has achieved a 2.3x compute efficiency in P-Zero Research's TasteVal benchmark, a significant leap in AI's ability to design and evaluate experiments. This progress has prompted Anthropic co-founder Jack Clark to move his prediction for recursive self-improvement (RSI) forward to 2027.
TL;DR
- Anthropic's Opus 5.5 model achieves a 2.3x compute efficiency in P-Zero Research's TasteVal benchmark.
- This progress accelerates the timeline for recursive self-improvement (RSI) in AI, with Jack Clark now predicting it could happen as early as 2027.
- The benchmark measures AI models' ability to design and evaluate experiments, a critical capability for autonomous AI research.
What happened
P-Zero Research introduced the TasteVal benchmark to measure AI models' ability to design and evaluate experiments. The benchmark tests models on eight tasks related to AI development, with a fixed coding agent running experiments on a single Nvidia H100 GPU. The human baseline was set by 24 experts from leading AI labs like OpenAI, Google DeepMind, Nvidia, and Microsoft Research.
Anthropic's Opus 5.5 model achieved a compute multiplier of 2.3, reaching expert-level scores with less than half the compute. The compute multiplier measures how efficiently a model can match expert scores, with a higher multiplier indicating greater research taste. The top model's performance suggests a significant improvement in AI's ability to conduct independent research.
Based on these results, Jack Clark, co-founder of Anthropic, has moved his prediction for recursive self-improvement (RSI) forward to 2027. RSI refers to AI systems developing their own successors, a critical milestone in AI development. Clark had previously expected this capability to emerge in 2028.
Why it matters
The TasteVal benchmark measures a critical capability that skeptics had flagged as a barrier to major AI breakthroughs. AI researcher Herbie Bradley had argued that models lacked the research taste, creativity, and long-term planning skills necessary for autonomous AI research. The benchmark's results push back against this assessment, demonstrating significant progress in AI's ability to plan and evaluate experiments.
The implications of this progress are significant for AI development. Faster and cheaper AI research could accelerate breakthroughs, but it also raises concerns about the verifiability of research conducted by AI models. As models take on more of the research process, it becomes harder for humans to verify their work, posing challenges for oversight and safety.
The debate around AI development is shifting, with major AI labs acknowledging the potential for RSI to emerge sooner than expected. Anthropic CEO Dario Amodei has called for a controlled slowdown in AI development, while OpenAI acknowledges that fully autonomous RSI isn't happening yet and says it should only be pursued if human oversight is guaranteed.
Key facts
- Anthropic's Opus 5.5 model achieved a compute multiplier of 2.3 in P-Zero Research's TasteVal benchmark.
- The compute multiplier measures how efficiently a model can match expert scores, with a higher multiplier indicating greater research taste.
- The benchmark tests models on eight tasks related to AI development, with a fixed coding agent running experiments on a single Nvidia H100 GPU.
- The human baseline was set by 24 experts from leading AI labs like OpenAI, Google DeepMind, Nvidia, and Microsoft Research.
- Jack Clark, co-founder of Anthropic, has moved his prediction for recursive self-improvement (RSI) forward to 2027 based on the benchmark's results.
- The compute multiplier has roughly doubled every three months since December 2025, according to P-Zero Research.
- The TasteVal benchmark measures a critical capability that skeptics had flagged as a barrier to major AI breakthroughs.
- The implications of this progress are significant for AI development, raising concerns about the verifiability of research conducted by AI models.
Context
The TasteVal benchmark arrives in the middle of a debate kicked off by major AI labs about the potential for recursive self-improvement (RSI) in AI. Anthropic has acknowledged that RSI hasn't been achieved yet but could come sooner than most institutions are prepared for. OpenAI acknowledges that fully autonomous RSI isn't happening yet and says it should only be pursued if human oversight is guaranteed.
The benchmark shifts what counts as progress in AI development. For a long time, the focus was on how well models solve well-defined tasks like math problems or coding challenges. Now, the question is increasingly whether they can make the decisions that push research forward.
The progress measured by the TasteVal benchmark has significant implications for AI development. Faster and cheaper AI research could accelerate breakthroughs, but it also raises concerns about the verifiability of research conducted by AI models. As models take on more of the research process, it becomes harder for humans to verify their work, posing challenges for oversight and safety.
