PrismML releases Ternary Bonsai 2 27B: 27B model at 5.9 GB, 98% performance

PrismML has shipped Ternary Bonsai 2 27B, a ternary-weight version of Qwen3.8 27B that occupies 5.93 GB against the 53.8 GB of the FP16 original. The company reports 98.2% average performance retention across 20 benchmarks, support for both text and image input, and a context window of 262 thousand tokens. The official demo runs Cline coding agents and computer-use workflows on an RTX 5090. The release follows the first Bonsai 27B by two months; that model's ternary variant preserved roughly 95% of baseline performance.

The Qwen3.8 27B architecture is unchanged: 27.36 billion parameters split into 24.35 billion in the language backbone, 2.54 billion in embeddings and the LM head, and 0.47 billion in the vision tower. The backbone uses hybrid attention — approximately 75% linear-attention layers and 25% full-attention. Ternary weights cover embeddings, attention projections, MLP projections, and the LM head; only 26.2 million parameters (0.0976%) remain in higher precision, specifically the recurrent state path and normalization weights. In the GGUF format the vision tower is packed separately as a 0.63 GB file loaded only on image input.

Each ternary weight takes one of three values: −1, 0, or +1. Every group of 128 weights shares a single FP16 scale. A ternary value carries log2(3) ≈ 1.585 bits; adding the 16-bit scale per 128 weights yields 1.71 bits per weight, rising to 1.72 bits with the high-precision tensors included. Practical kernels require dense packing, and the whitepaper describes two GGUF schemes: PTQ1_0 packs trits densely at 1.76 bits per weight (5.93 GB), while PQ2_0 stores each trit in a 2-bit slot at 7.25 GB, cheaper to decode. Weights are also stored in a rotated basis: PrismML applies a blockwise Hadamard rotation of size 1,024 before ternary assignment, and the runtime applies the matching transform to activations before every matmul. The company cites SpinQuant as the source of the idea but does not disclose how the ternary values themselves are determined.

The sharper comparison is against conventional quantization. An IQ2_XXS build of Qwen3.8 27B averages 75.2 at 7.3 GB. On AIME26 Bonsai 2 scores 95.83 versus 78.6; on LiveCodeBench v6 the gap is 90.07 versus 70.05. All tests ran in thinking mode with EvalScope and vLLM on H100. The 98.2% average masks uneven drops: vision retains 96.3%, knowledge and reasoning 96.9%. In long-horizon agent tasks the fall-off is steeper — 52.8 versus 69.7 on Terminal-Bench 2.1 and 60.8 versus 80.6 on SWE-bench Verified (roughly 75% retention) — and neither benchmark was included in the 20-benchmark average. Reasoning effort also matters: at medium effort the average is 79.3 versus 82.6 for the FP16 baseline, and low effort is unsupported. All results are PrismML's own and have not been independently reproduced.

Measured decode speeds (batch-1, optimized kernels, 16 September 2026): RTX 5090 reaches 142.5 tokens/s at 0.582 mWh/token; RTX 4090 hits 96.7 with PTQ1_0; a 72 W L4 delivers 32.1; M5 Max and M5 Pro laptops manage 46.8 and 27.7 respectively. No single packing wins everywhere: PTQ1_0 is faster on Ada and L4, PQ2_0 on Blackwell, Hopper, Ampere, and Apple Silicon, and for prompt processing on all platforms. The research team also claims 40% better energy efficiency than an 8B full-precision model. Execution requires PrismML's fork of llama.cpp — the upstream rejects the PTQ1_0 and PQ2_0 types. The supported path is the Bonsai-demo repository: run setup.sh, then scripts/start_llama_server.sh for chat.