Qwen3.8-Flash-Next runs locally on 75 GB memory and outperforms Claude Opus 4.6 Max

The new Qwen model, a 125 billion-parameter MoE variant, is now available for local execution via Unsloth’s GGUF format, requiring no graphics card. According to the company, the 1-bit compressed version occupies 75 GB of system (or unified) memory and recovers 80 % of the original accuracy while being 79 % smaller than the full BF16 version that weighs 355 GB. The notable point is that the architecture permits inference in ordinary system memory at speeds approaching VRAM, a material shift for anyone with a unified-memory machine or an NVIDIA DGX Spark station.

Qwen3.8-Flash-Next is built on the new Qwen4 architecture and supports a context window of 262,144 tokens, a substantial jump from the previous generation. The multimodal model includes a hybrid-thinking mechanism with two separate default modes: a thinking mode with temperature 1.0 and top_p 0.95, and a regular instruction mode with temperature 0.7 and top_p 0.80. The key distinction lies in presence_penalty—zero in thinking mode, 1.5 in regular mode—forcing the model to diversify when it is not “thinking” out loud.

The aggressive compression relies on treating the new PLE (Per Layer Embeddings) and Ngram layers as random-access lookup tables that remain at a minimum of 4-bit, because deeper quantization would destroy their accuracy. All other weights are reduced to 1-bit. The result is a UD-Q4_K_XL version that weighs 111.3 GB with KLD 0.0447 and 93.48 % accuracy; UD-IQ4_XS drops to 93.7 GB with KLD 0.079 and 91.09 % accuracy; the smallest entry, UD-Q3_K_XL, stands at 90 GB with 90.3 % accuracy.

Unsloth’s requirement table lists tiers of 75, 79, 90, 112, 200, 270 and 355 GB, referring to total memory (RAM + VRAM, or unified memory). The smallest 1-bit version runs on 75 GB, but a device with at least 96 GB is advisable to leave headroom for the operating system and long contexts. Users can also offload the PLE/Ngram layer to an SSD and use mmap, reducing simultaneous RAM and VRAM usage—a useful trick for systems limited to 64 or 80 GB.

The model supports a reasoning_effort parameter with four levels: xhigh (default), medium, low and none. In Unsloth Desktop the switch is made via --chat-template-kwargs '{"reasoning_effort":"medium"}' (PowerShell requires escaping the quotes). A Preserve Thinking mechanism retains the chain of thought from the previous message, increasing token consumption but potentially improving accuracy in ongoing conversations; in Unsloth it is toggled via dedicated Think and Preserved Thinking switches.

The claim that the model “outperforms Claude-4.6-Opus (Max)” comes from Qwen’s launch announcement, not from an independent third-party test. The KLD and accuracy figures for the various quantizations were published by Unsloth, which received day-zero access. No results from LMSYS Chatbot Arena, MT-Bench, HumanEval or SWE-bench are available yet. Until independent comparisons arrive, these numbers remain manufacturer estimates—interesting, but not a substitute for external measurement.