Microsoft names Maia its primary inference engine, targeting multi-gigawatt rollout

Satya Nadella has formally declared that the Maia chip family will become the central inference infrastructure for every Microsoft product, with a deployment target of several gigawatts. The move is designed to cut reliance on Nvidia hardware, and Microsoft says Maia 300 is intended to replace AMD chips as well, not only Nvidia's. The headline claim: 40% more tokens per megawatt versus leading GPU-based systems.

Maia 300, the third generation slated for 2027, uses a multi-chiplet architecture. Instead of a single large die housing compute, memory controllers and I/O, those functions are split across separate chiplets packaged together. The separation frees die area on the compute chiplets for math units and lets each chiplet be fabricated on the process node that suits it best. Each accelerator packages eight HBM4 stacks delivering 18.0 TB/s peak bandwidth, a 950 W TDP, and on-die SRAM that serves as a scratchpad to relieve memory pressure.

Microsoft identifies data-movement cost, not raw compute, as the bottleneck in large-scale inference. To address it, Maia 300 supports low-precision formats such as enhanced MXFP4, which compresses the KV cache — the attention state that grows with every token — and enables longer context windows within the same power envelope. A full rack holds 96 accelerators at 167 kW with 27.6 TB of HBM, and the company claims up to 7x the token throughput of a Maia 200 rack.

The predecessor, Maia 200, is already handling production traffic including Microsoft Copilot. Internal figures show it delivers up to 1.7x more tokens per watt than a leading GPU infrastructure. The shift to Maia 300 targets long-context reasoning, low-latency chat and multi-step agent workloads — scenarios where memory efficiency and bandwidth matter more than raw FLOPS.

The harder constraint is supply. Microsoft is negotiating with TSMC for more than 300,000 Maia 300 units for 2027 delivery, though the figure remains in negotiation and has not been signed as a purchase order. Nadella's characterization of the program as "our biggest bet on custom silicon" marks a strategic inflection: starting next calendar year the company will expand Maia deployment to multi-gigawatt scale, making it the default inference engine across all its services.