2-bit NVIDIA Nemotron 3.5 Lightning runs tool calls for 10 minutes using 22 GB VRAM

In Unsloth Desktop, the 2-bit variant of Nemotron-3.5 Lightning can execute tool calls continuously for 10 minutes while consuming only 22 GB of video memory. During the run the model cites more than eighty websites, executes code and identifies 10 real-world locations worldwide. Unsloth Dynamic 2.0, which accompanies the system, attains higher accuracy and precedes most leading quantizers.

The model was created by NVIDIA Corporation, with development spanning from December 2025 to May 2026. Initial training cut-off data were taken in September 2025, and post-training cut-off data in May 2026. Nemotron ™ belongs to an open family of models whose weights, data and recipes are publicly available to improve efficiency and accuracy of custom AI agents.

The BF16-full version, NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16, comprises thirty billion parameters, of which three billion are active, and is built on a hybrid Mixture-of-Experts architecture that integrates Mamba-2, MoE and selected Attention layers. It also incorporates speculative decoding methods that speed text generation. The model is commercial-ready and includes an NVFP4 variant tuned for optimal latency and throughput.

For rapid deployment on an H100 unit, vLLM (Nightly v0.27.1) can be used with the following command:

```

export MODEL_CKPT=nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16

vllm serve --model $MODEL_CKPT \

--max-num-seqs 128 \

--enable-prefix-caching \

--async-scheduling \

--mamba-backend flashinfer \

--mamba-ssm-cache-dtype float16 \

--enable-mamba-cache-stochastic-rounding \

--mamba-cache-philox-rounds 5

```

The model is released under the OpenMDW-1.1 license, permitting commercial use in accordance with the download terms.