Open models close raw quality gap with closed models in months

Tri Dao presented at ICML a concise assessment: open models are now only a few months behind leading closed models, while the serving stack, kernels, inference engines and speculative decoding continue to double their operational advantage. The remarks were made at the flagship research conference and reflect a trend that researchers have been monitoring for several quarters.

The serving stack components are not decorative. Hardware-tailored kernels shorten first-token latency, inference engines such as vLLM or SGLang maximize throughput at fixed memory, and speculative decoding enables generating multiple tokens in parallel without sacrificing accuracy. Each element improves independently, and when combined the gains compound rather than merely sum, which is why the raw quality gap does not tell the whole story.

The estimate of “a few months” is significant because not long ago the community discussed a gap of a year or more between comparable closed and open versions. The current pace results from higher-quality open data, leaked or published training recipes, and shorter release cycles from communities such as Llama, Qwen or Nemotron. When the base model advances quickly, fine-tuning and deployment timelines shrink as well.

Responding to a Together AI post, a commentator said an open model with a suitable serving stack is “useful beyond measure” compared with a closed model of the same tier, even if it arrives months later. The argument rests on full control over fine-tuning and deployment, without dependence on an external API, rate limits or a single provider’s policy changes. For organizations running production workloads this is the difference between capacity planning and gambling on a third-party service’s availability.

Dao did not release concrete benchmark figures, nor specify which tasks or datasets were used to measure the “months” gap. The definition of “quality” also remains unspecified—whether it refers to MMLU, HumanEval, human evaluation or a composite metric. Without numbers and methodology, the claim remains a directional indicator rather than quantitative proof.