Liquid AI releases experimental drafter accelerating vision-language decoding up to 3.13x on Apple silicon

Liquid AI has announced LFM2.5-VL-3B-DSpark, an experimental drafter model that sits atop its LFM2.5-VL-3B vision-language model and promises up to 3.13× faster decoding on Apple silicon and up to 2.66× on Nvidia H100 GPUs. The drafter adds roughly 280 million parameters — an 8.9% increase in deployed parameters — without altering the primary model's output. Weights are available now on Hugging Face in Safetensors and GGUF formats, with support arriving Sunday in SGLang, MLX-VLM and llama.cpp. The release carries the LFM Open License v1.0, which permits free commercial use only for companies with annual revenue below $10 million.

Speculative decoding works by having a small model propose several tokens ahead; the large model then verifies the entire block in a single forward pass and keeps only the tokens it agrees with. The key insight here is that the drafter is modality-agnostic: once tokens reach the hidden layers, text and image patches are simply tensors. This lets Liquid AI reuse the same inference algorithm that powers its text-model DSpark drafters, without writing a separate vision code path.

The drafter is a simplified attention-only model with 4 layers and a block size of 9; the recommended setting at runtime is 8 or 9 depending on hardware, with 8 used on Apple silicon. The embeddings and LM head are shared with the target model, so the drafter does not carry its own. Training ran for ten epochs on common vision-language task data, and all ablations and training runs were conducted exclusively on AMD hardware.

Benchmarking followed the MMSpec suite across 6 task categories — general VQA, Text VQA, image captioning, Chart VQA, complex reasoning and multi-turn dialogue — at batch size one, temperature zero, with 16-bit weights for the vision encoder and backbone. Data was collected on Pipette, Liquid's public benchmarking infrastructure. The "up to" figures come from different tasks: on an M5 Max, the 3.13× decoding speedup came from COCO Captioning, while the 2.62× end-to-end gain came from MMMU-Pro. Acceptance rates were similar across both Apple software stacks, indicating they depend on the drafter and workload rather than the runtime. On SGLang running on a single H100, the throughput advantage held at every concurrency level measured, though the gap narrows as concurrency increases.

Under greedy decoding the primary model verifies every proposed token, so output is identical to the base model. At non-zero temperatures with coupled sampling, speculative decoding preserves the target model's output distribution, a result proven by Leviathan et al. Temperature does affect speed: higher temperatures spread probability mass over more candidate tokens, the drafter and target agree less often, and both acceptance rate and throughput drop.

End-to-end gains are smaller at the edge because speculative decoding accelerates only the decoding stage. Image encoding and prefill run at the same speed, and a vision-language model must encode the image and then process hundreds of visual tokens alongside the prompt. On edge devices with less compute than a datacenter GPU, prefill occupies a larger share of total latency — Amdahl's law in action: total speedup is bounded by the unaccelerated portion. This explains cases such as TextVQA on M5 Max, where a 2.69× decoding improvement translates to only 1.56× end-to-end.

To run the model, SGLang version 0.5.19 or later is required. Launch LiquidAI/LFM2.5-VL-3B with the `--speculative-algorithm DSPARK` flag.