MiniMax H3 runs on single 4090 with 4.44× speedup over dense execution

The researchers of MiniMax released a dedicated runtime package for the H3 model that enables running the full BF16 FL2VA checkpoint, the 33B-parameter model, the Qwen3-VL conditioner and VAE on a single RTX 4090 with 24 GB memory. The core achievement is a speedup of 4.44× end-to-end compared with dense execution on the same hardware, thanks to a sparse-attention kernel (Sol-Attn) adapted to the SM89 architecture, a caching mechanism called TeaCache, and regional compilation with torch.compile.

The trick is offloading layer-wise components via SGLang: at any moment only the active component sits on the GPU, while the rest wait in host memory. The full configuration (rtx4090_fullopt.toml) fixes all parameters—from prompt and seed through 50 measured denoising steps to the SM89 profile and cache—so the run reproduces exactly the benchmark that was published. Requirements are PyTorch 2.11 with CUDA 12.8, Triton 3.6, the FL2VA checkpoint supplied through the environment variable H3_MODEL_PATH, and ffmpeg/ffprobe for final video encoding.

Sol-Attn runs with tau=1.0, diagonal threshold, an exact KV queue for prefix, compressed prefix queries, and the first ten steps and first two blocks in full compression. The first sparse forward step reported an effective block density of 19.65 % in dimensions [1, 38247, 56, 128]. TeaCache, with threshold 0.10, five saved steps and one cooling step, performed 14 block-stack calculations and 35 reuse calls out of 49 decisions. The Real-QKV gate passed the declared error tests (maximum absolute error 0.0625, average 0.000289, L2 relative 0.000609).

The 4.44× acceleration was measured against a “no-loss” baseline where neither TeaCache nor Sol-Attn sparsity were enabled, i.e., the comparison is with a fully dense run using the same layer offload and regional compiler. No claims were made about bitrate or video quality, no perceptual metrics or embeddings were measured, and timings were taken from a single hot sample without variance characterization. In other words, this is an engineering benchmark of runtime, not a proof of generation quality.

The run is executed from the repository root with `python3 scripts/run.py config/minimax_h3/rtx4090_fullopt.toml`; the script also supports pointwise overrides via `--set` and exporting a Slurm package via `launch_config.py`. The runtime package saves out.mp4, benchmark.json and launch logs under a runs directory. The attached commits are: Sana 6fb7eb1, SGLang 6fa3f9d.