Liquid AI releases visual draft model for LFM2.5-VL-3B, claims up to 3.13× decoding speedup

Liquid AI has released LFM2.5-VL-DSpark, a draft model that attaches to LFM2.5-VL-3B to accelerate speculative decoding in vision-language models. The draft adds only 280 million parameters — an 8.9% increase over the full model — and ships with immediate support in llama.cpp, MLX-VLM, and SGLang. The company reports decoding speedups of up to 3.13× on Apple Silicon devices and up to 2.66× on H100 GPUs, with end-to-end gains ranging from 1.56× to 2.62× depending on platform and task.

How the draft works

The architecture mirrors the text-only drafts in the LFM2.5-DSpark series: the draft samples hidden states from a fixed set of layers in the target model and conditions on them to predict a block of k candidate tokens. The visual version's novelty is that image patches and text tokens are projected into a shared representation before those layers, so the draft operates on vectors of the same dimension regardless of input modality. The inference algorithm itself remains unchanged.

Training methodology and architecture selection

Liquid trained the draft on a vision-language SFT data mixture with weights biased toward expected workloads. Ablations across 3, 4, and 5 layers led to a simplified Attention-only draft with 4 layers and a block size of 9. The run spanned 10 epochs; the acceptance rate improved with additional training tokens until it hit diminishing returns. For deployment the company recommends a block size of 8 or 9, depending on hardware.

On-device performance

On an M5 Max running MLX, decoding runs 2.30× to 3.13× faster depending on the task — general VQA, text-in-image, captioning, charts, complex reasoning, and multi-turn dialogue — with end-to-end improvements of 1.56× to 2.62×. On an M3 Ultra with llama.cpp the numbers are more modest: 1.57× to 2.14× for decoding, 1.30× to 1.77× end-to-end. The gap stems from differences in GPU architecture and memory management between the two Apple Silicon generations.

H100 performance

Using the same draft, SGLang on H100 delivers decoding speedups of 2.04× to 2.66× and end-to-end gains of 1.64× to 2.27×. The spread between decoding acceleration and overall improvement is consistent across both platforms, and that is not accidental.

Amdahl's law in practice

In the company's words: prefill is largely compute-bound, and its cost grows quadratically with prompt length. In VLMs a vision encoder produces hundreds of visual tokens before the language backbone even starts. On edge devices compute is limited, so prefill consumes a larger share of total wall-clock time. Speculative decoding accelerates only the decoding stage — not image encoding, not prefill. When the unaccelerated stages already dominate the clock, even a large decoding speedup translates into a limited overall gain. This is Amdahl's law in action: total throughput is bounded by the portion that does not speed up.

Deployment and usage

In SGLang a build with DSpark support for LFM2 targets is required (PR #40651), and launch uses the `--speculative-*` flags with block size 9. In llama.cpp a compatible build is needed (PR #29339) and launch uses `--spec-type draft-dspark` and `--spec-draft-n-max 8`. The block size is read from the draft's config.json.