Coding agents become builders on Hugging Face hub

During the past month a quiet but substantial shift occurred on the Hugging Face hub: coding agents began acting as builders for everything, not merely as assistance tools. In the reconstruction challenge at the ICML conference, 1,221 human researchers partnered with agents to validate and reproduce 2,226 scientific papers. This marks the first instance in which agents use the open-source infrastructure in the same way humans do—writing experiment logbooks, publishing results, and building on each other’s work.

The scale of the experiment is unprecedented: 6,816 reconstruction logbooks were published openly, 2,962 cloud tasks were executed, and 35,908 claims were adjudicated, all traceable, public, and transparent. Until now the hub was primarily used by human researchers to share models, datasets, and demos. It is now also used by agents that document their work in the same format, creating a continuous chain of evidence rather than isolated press releases.

The contrast with the typical closed-lab approach is stark. In closed labs, evaluation runs are performed behind closed doors and the community is asked to trust the results announced in official statements. Here the evaluations are open: anyone can inspect the logs, rerun the code, and verify the numbers. This is a practical definition of open science that is now extending to agent activity.

Not everyone is convinced that sheer volume alone demonstrates maturity. In responses to the publication, commenters described the 1,221 humans plus agents as “a lot of cooks in one kitchen,” and argued that the real test will be whether the reproductions hold up when examined outside the hackathon setting. Reproduction is a convenient starting point because the metric is binary—results either repeat or they do not—turning logbooks into evidence rather than mere claims. The open question is whether the same transparency will persist for tasks where evaluation is softer, which comprise most of agents’ actual work.

Clément Delangue, co-founder of Hugging Face, said, “The next million users of the hub may not be human, and it could be the best thing that has happened to science.” Full details of the hackathon are available on the company’s official blog.