MBZUAI's Institute of Foundation Models releases six fully open models, 0.9B to 375B

The Institute of Foundation Models (IFM), the frontier lab launched by Mohamed bin Zayed University of Artificial Intelligence in May 2025, did not stop at a single model and a benchmark table. Last week it shipped K2 Horizon — six models at 375B-A23B, 36B-A4B, 32B, 7B, 3.7B and 0.9B — all under Apache 2.0, with the complete training corpus, intermediate checkpoints, training code, configs and detailed logs. IFM calls it the largest open release in the field's history, and the claim is hard to dispute: all six are already on Hugging Face with FP8 and GGUF builds, day-zero support in vLLM, SGLang and Ollama on Nvidia, AMD and Cerebras hardware. Hosted access is available through Compass, Cerebras and Nebius on the institute's platform.

What sits behind the numbers

The six share a core architecture, vocabulary, training methodology, interfaces and deployment tooling — consistency designed to let teams prototype on 3.7B and scale to 375B-A23B without swapping serving stacks. Each model was pre-trained on roughly 20 trillion tokens, with about 17% of the corpus comprising problem-solving trajectories that show explicit reasoning, and roughly 10 trillion synthetic tokens. Post-training data was folded in from mid-training rather than held for the end, and IFM researchers say more than 100 million unique synthesized tasks were created. Tool definitions were presented in JSON, XML and Markdown during training so the model would learn semantics rather than syntax; Markdown became the inference default, about 18.5% more efficient than JSON on the institute's measurements.

MoVA and Uno: two architectural and inference innovations

Mixture-of-Value Attention (MoVA) extends expert routing into the multi-head attention layers, not just the feed-forward networks, opening a second axis for capacity growth while staying compatible with FlashAttention, GQA and sparse attention. The result, K2-Horizon-MoVA-36B-A4B — 36B total parameters, roughly 4B active per token — lands slightly below the dense 32B model under identical conditions but leads the comparison set with 58.6 on Terminal-Bench 2.1 and 26.8 on tau3-Banking. In parallel, Uno freezes the autoregressive parameters and trains a small set of diffusion parameters that learn only to generate efficiently; through what IFM calls "diffusion distillation" the adapters emit token blocks in parallel, delivering a 3× speedup without quality loss. The adapters are released as LoRA, currently for the 7B and 0.9B models.

Benchmarks: the small models are the sharp story

The largest model, 375B-A23B, posts 70.2 on Terminal-Bench 2.1, 1,441 Elo on GDPVal-AA, 67.7 on MCPMark and 87.3 on GPQA Diamond