Prime Intellect closes 82% of gap to human researchers in large autonomous experiment

Prime Intellect ran more than one hundred autonomous trials on ten frontier 2026 models, each in an isolated environment on 8 H200 cards for up to 8 days. The task was to improve a training recipe for a GPT with 124 million parameters, allowing the models to modify only the optimizer’s hyper-parameters, with no network access and no shortcuts. The best runs succeeded in closing 82% of the gap to the peak achieved by dozens of human researchers over several months.

The benchmark was a clean nanoGPT optimizer track, a narrow, well-known code base that enables a fair comparison. Every model started from the same initial point and was evaluated solely on its choices in the hyper-parameter space. The hardware was identical for all runs: 8 H200 cards and a time budget of up to 8 days per run, with no internet and no ability to import tricks from new papers, forcing experimental reasoning inside the sandbox.

The models tested included Fable 5, Opus 5, GPT-5.6 Sol, Kimi K3, Grok 4.5, GLM 5.2, Muse Spark 1.1, DeepSeek V4 Pro, Grok 4.6, Muse Spark 1.2, and Qwen 3.8. The distinction between the stronger and weaker models was not theoretical knowledge—each knew the literature—but experimental judgment: which experiments to run, how to navigate the benchmark’s built-in noise, and when to revert to previously discarded ideas because the recipe had changed.

Prime Intellect’s harness provides each model with a persistent IPython kernel, a research workspace where personal tools can be built. Kimi K3 leveraged this to create a suite of utilities: controlled versions of the optimizer, loss-curve comparisons, and Newton-Schulz tuning. When a cleaner update did not perform as expected, the hypothesis was revised and the process continued. This was not merely “using tools” in a narrow sense; it was the construction of a research workflow from scratch.

Looking ahead, Prime Intellect envisions multi-agent harnesses in which small open-weight models handle monitoring, logging, and technical execution while the frontier model focuses on strategic decisions. This approach is expected to dramatically lower experiment costs and improve outcomes by reducing GPU cycles spent on infrastructure work. At the same time, the plan is to extend these “fast-track scripts” to other parts of the training stack and to scale up the runs themselves.