A study presented at the Fourteenth International Conference on Learning Representations (ICLR 2026) found that larger AI models completed more turns successfully, while reasoning extended the length of tasks completed in one response.
The authors—Akshit Sinha, Arvindh Arun, Shashwat Goel, Steffen Staab and Jonas Geiping—used controlled, synthetic assignments with the needed information and action sequence specified in advance. They assessed execution while leaving planning out of the task, allowing them to compare how models handled longer interactions. High accuracy on a lone action did not guarantee the same consistency over a longer exchange.
The researchers found that larger models maintained stronger turn-level performance, even as accuracy declined over later turns. In tests where a model’s earlier answers contained errors, later answers were more prone to mistakes, a pattern the authors call self-conditioning.
That pattern did not appear in the same way in tests of Qwen3 models running in thinking mode. Separately, the paper measured how complex a single-turn task a model could handle while maintaining at least 80% accuracy. GPT-5 reached a task complexity of 2,176 steps at that threshold, compared with 432 for Claude-4 Sonnet.
The authors suggest that next-token training may lead models to continue from error-filled histories, but present this as a possible explanation. They also say reasoning before acting uses more computing power and fills more of a model’s context window; the paper does not identify who specifically bears those costs.
The evaluation used synthetic tasks. The authors say progress on such tests is necessary but not enough to establish performance in open-ended real-world work. The paper also cites METR’s estimate that software-task horizons for frontier models have doubled about every seven months.
