One model learns from team logs and rewrites the agent playbook

A new paper from Stanford and Together AI shows a three-model team that learns from past conversation logs how to divvy up checking and challenging roles solves problems none of them could crack alone. On math and physics benchmarks the team hit 66.7% accuracy versus 48.8% for the strongest solo model — and it beat an Oracle pick from the individual outputs, meaning the collaboration produced correct answers that didn't exist in any single run.

Why discussion and voting fall short

The standard multi-agent playbook today: let models debate, then vote. In practice that mostly surfaces an answer someone already had; it doesn't create new knowledge. The researchers found the bottleneck isn't model capability — it's missing structure. Without explicit roles — who checks, who challenges, who plays devil's advocate — the conversation converges too fast on a wrong consensus.

Learning from just 15 examples

The twist is a meta-model that reviews the team's past runs, spots failure patterns, and rewrites the protocol: who double-checks whom, when to inject a counter-challenge, how to hand off between solution stages. That learning took only 15 practice problems — negligible compared to standard training data — and it was enough to fundamentally shift the collective behavior.

Results that speak for themselves

On math and physics benchmarks the coordinated team averaged 66.7% against 48.8% for the best solo model. More important: it cleared the Oracle selection bar, a perfect hindsight pick from the answers the models had already generated individually. The interaction itself forged reasoning paths no single model could reach — not just better filtering of what was already there.

Explicit roles over a fixed script

The practical takeaway: instead of running a fixed debate-and-vote script, define explicit roles — checker, challenger, synthesizer — and let a meta-model update the division of labor based on run history. That turns the pipeline into something that improves over time, no heavy fine-tuning or fresh labeled data required.