P-Zero researchers launch "research taste" benchmark, and models already surpass experts at 1/30th the cost

P-Zero Research has released TasteVal, a benchmark that measures "research taste": a model's ability to plan experiments and interpret their results across eight real AI development tasks. Scores are expressed relative to a human-expert baseline — marked at 1x — and the metric is the compute a model consumes versus the experts per attempt. In plain terms: how cheaply and quickly the model reaches the same insight a researcher would spend far more resources to obtain.

According to P-Zero's data, progress was sluggish from GPT-4 in 2023 through the end of 2025. In December 2025 the slope shifted: scores have been doubling every 3.0 months since. If that pace holds, the benchmark becomes one of the more interesting indicators to watch — not because it measures knowledge or language, but because it measures the skill that determines how fast AI can start improving itself.

Claude Opus 5.5 currently sits at 2.30x the expert baseline, meaning it delivers better results at less than one-third the cost. It leads Fable 5.1 and GPT-6 Astra. The more striking figure is the cost: roughly 1/30th of a human expert attempt. That does not mean the model is "smarter" than the researchers; it means it is more efficient at reaching the same specific experimental conclusion the benchmark tests.

Jack Clark noted yesterday that this curve is one reason he expects a significant milestone in 2027. The logic is straightforward: if models improve at planning and interpreting experiments — the core skill of AI research — the self-improvement loop shortens. TasteVal does not measure general intelligence; it measures specific research productivity. And in productivity, the numbers are starting to look different.

The benchmark is new, the eight tasks are a narrow sample, and the expert comparison covers only those exact tasks, not end-to-end open research. There is no independent replication yet, and no full disclosure of the scoring methodology beyond the compute ratio. As with any new metric published by an interested party, it is worth waiting for external validation before drawing conclusions about timelines.