Meta launches MetaRoCE, a clean-sheet RDMA protocol for massive model training

Meta unveiled MetaRoCE this week, a clean-sheet RDMA transport built for AI workloads on standard Ethernet. Unlike standard RoCE, which assumes in-order frame delivery and leans on priority-based flow control (PFC) to prevent loss, MetaRoCE treats the fabric as lossy and pushes reordering, path selection, and recovery onto the NIC. The company is releasing the specification, a reference software implementation, and a compliance test suite through the Open Compute Project (OCP).

Not yet deployable in production

The artifacts are not expected until October 2026, likely at the annual OCP Global Summit. For now this is an architecture decision, not a procurement one: Meta has validated the protocol on AMD Pensando programmable NICs, and additional vendor implementations are in progress. Anyone looking for a drop-in solution will have to wait.

The problem: the fabric sees packets, the NIC sees intent

Meta has scaled its clusters to hundreds of thousands of GPUs across multiple data centers and regions. At that scale the network sits on the critical path of every training step. Collective operations such as all-reduce and all-to-all synchronize thousands of accelerators, and the slowest transfer sets the pace for the entire job. Standard RoCE is the bottleneck: it expects in-order delivery, depends on PFC, and suppresses packet spraying — the deliberate spreading of packets across multiple paths — which actually delivers better performance in large-scale, multiplane networks.

Six design decisions that change the picture

MetaRoCE flips the model: intelligence moves to the endpoint, and the fabric collapses into thin logical paths, each with real-time telemetry, per-path RTT, ECN state, and utilization. Out-of-order delivery becomes the default: packets are sprayed across many paths and arrive out of order by design. Every packet carries its destination, so data lands directly in its final location on arrival — no reorder buffer, no head-of-line blocking. Multipathing is built in: each path uses a different UDP source port as entropy for ECMP, and the NIC can change that port at any moment to steer traffic off a bad path. Because each path maintains its own window and round-trip estimate, traffic can distinguish congestion from failure and rebalance explicitly.

Loss tolerance instead of zero loss

MetaRoCE treats the fabric as lossy — no PFC, no pause frames. A gap in the 256-bit selective acknowledgment bitvector signals loss rather than reordering, triggering a precise retransmit of the missing packet on the path that lost it. Congestion control runs on both ends: ECN-based AIMD on the sender side combined with fair-share rate hints on the receiver side. With every acknowledgment the receiver returns the slice of inbound bandwidth it allocated to that sender, so senders converge to the right rate directly instead of hunting for it. Incast — where many senders target one receiver simultaneously — resolves in one or two round trips.

Topology independence and minimal switch requirements

MetaRoCE asks the fabric for two things every modern switch already supports: ECN marking and ECMP. It requires no packet trimming, in-network telemetry, credit-based flow control, or protocol changes in the switch. The work builds directly on Meta's 2024 RoCE-at-scale effort and the broader evolution of its infrastructure.