DisCo agent lifts ML benchmark scores by distilling open-source operational knowledge

A research agent called DisCo improves performance on ML benchmarks without swapping the underlying model. It does this by adding a layer of operational knowledge distilled from open-source code repositories.

The problem DisCo solves is familiar to anyone who has tried to run an autonomous agent on a real ML task: the model knows the method but not the "tricks" that make it work in practice — hidden parameters, version dependencies, debugging patterns. That knowledge exists in repositories and papers, but it is written for humans and too large to load at inference time.

Two distillation paths

DisCo operates on two complementary tracks. The first, task-agnostic, scans the open ecosystem and distills popular repositories into compact, verified skills. The second, task-oriented, generates dedicated skills for a specific task.

The output of the first track is the AREX-Skill library — more than 5,000 verified skills distilled from 1,000 popular ML repositories, organized into 20 domains and 178 capability families.

Numbers with a fixed base

Holding the backbone (GPT-5.5), the research harness, and the compute budget constant, the skill-equipped agent scores 134.3% higher on MLE-bench, 34.4% higher on PaperBench, 9.2% higher on FrontierCS, and 14.0% higher on PassNet compared with the same agent without skills.

The gaps on MLE-bench and PaperBench are especially large. Both benchmarks test the ability to carry out full end-to-end ML engineering tasks, not just answer knowledge questions.

Library without citations, for now

The paper (arXiv:2609.02749) remains a preprint without peer review, and Semantic Scholar shows no models or datasets citing it yet. The list of similar papers recommended by the library's bot — all from 2026 — shows how fast the field is moving: Repo2Skill-Evo on skill obsolescence, Demystifying Agent Skills on the limits of the approach, and further work on self-evolution and safety. That context matters: DisCo is not operating in a vacuum but inside a dense research conversation attacking the same questions from different angles.

Available to run locally

The code is on GitHub, and a DisCo CLI enables quick installation (`curl -LsSf | bash`). For those who prefer listening over reading, tensorbrife turns Hugging Face papers into daily podcasts. The direct-read identifier for the agent: `hf papers read 2609.02749`.