MIT and Sakana AI show self-improving code agents work at one-tenth the cost

Researchers from MIT and Sakana AI have demonstrated that self-improving code agents can work — and at roughly one-tenth the compute cost of the previous leading method.

The approach, called SIFT (Self-Improvement via Fast Tree-search), lets a model write changes to its own code, rank them with an LLM judge, and send only the most promising candidates for full benchmark evaluation. The bottleneck is the benchmark run itself, so the pre-filter slashes compute time: under 50 CPU hours and under five wall-clock hours for the top configuration running o3-mini.

On the Polyglot benchmark, the o3-mini configuration hit 35.1% after 30 tree-search expansions. For comparison, DGM, the prior method showing similar self-improvement, reached 30.7% only after 80 nodes. The gap isn't just in percentage points: SIFT did it in one-tenth the CPU hours DGM required.

Running the full search with Qwen3-30B cost 224 CPU hours and $34 in API spend — again, about one-tenth of DGM's baseline. The savings come from the same principle: the judge filters out the vast majority of mutations before they reach an expensive benchmark run.

Judge quality determines the outcome. On TerminalBench, the researchers showed how critical the choice of judge is. With gpt-5.4-high as a pairwise judge, the search found an agent scoring 36.7%, up from a 29.2% baseline. gpt-5 (without the high suffix) yielded 34.5%, and the top-ranked candidate was not the best agent the search produced — proof that the ranking is imperfect and that judging quality caps the ceiling.

The practical takeaway: iterative self-improvement of code agents is feasible on a sane compute budget, no thousand-GPU clusters required. The current limit isn't the algorithm but the quality of the judge model, and the next generation of judge models will push that ceiling higher. The paper is available at academy.dair.ai/papers/self-improvement-via-fast-tree-search.