Nvidia releases Nemotron-3 Ultra teacher model for competitive programming

Nvidia announced over the weekend the NVIDIA-Nemotron-Labs-Teacher-Competition-Coding model, a teacher model dedicated to competitive programming that is part of a suite of more than ten teacher models feeding the Multi-Teacher On-Policy Distillation (MOPD) stage in the Nemotron 3 Ultra recipe. The model has been available as a standalone checkpoint on Hugging Face since 14 August under the OpenMDW-1.1 license, which permits both commercial and non-commercial use.

The model is built on a hybrid Latent Mixture-of-Experts (LatentMoE) architecture that interleaves Mamba-2 and MoE layers with selected attention layers. In the LatentMoE mechanism, tokens are projected into a smaller latent dimension for routing and computation within the experts, improving accuracy per byte. The model contains 550 billion parameters, of which 55 billion are active in each forward pass. It also incorporates Multi-Token Prediction (MTP) layers with shared weights across prediction heads, enabling the generation of multiple tokens in a single inference step and improving both speed and quality.

Pre-training used the NVFP4 recipe, a quantization-aware approach shared with the Nemotron 3 family, where most linear layers employ NVFP4 for weights, activations and gradients, while selected layers (latent projections, MTP, QKV/Attention projections and embeddings) remain in BF16 or MXFP8 for stability. The pre-training cut-off point is September 2025. After that, supervised fine-tuning (SFT) and code-focused reinforcement learning (RL) were applied, producing a post-trained ultra-teacher model that excels in competitive programming benchmarks; post-training continued until May 2026.

Consistent with the rest of the family, the model first produces a reasoning trace before the final answer, and the depth of reasoning can be controlled via a flag in the chat template. According to Nvidia, the model is especially suited for solving hard algorithmic problems with execution-verified validation, acting as a teacher or grader in data-generation pipelines, and generating verified code solutions. Supported languages include English, French, Spanish, Italian, German, Japanese, Korean, Hindi, Brazilian Portuguese and Chinese, though the primary focus remains English and code.