Bonsai 2 27B: Qwen3.8 in 5.9 GB at 98% of Full Precision

On September 17, 2026, PrismML released Ternary Bonsai 2 27B — a 1.76-bit-per-weight build of Alibaba’s Qwen3.8-27B that fits in 5.93 GB and, on the company’s 20-benchmark suite, keeps 98.2% of the full-precision model’s score. July’s Ternary Bonsai 27B, built on the older Qwen3.6 base, retained about 95%. The new build ships under Apache 2.0 on Hugging Face, decodes at 27–47 tokens per second on Apple Silicon laptops, and — through a community WebGPU port — loads directly in a browser tab.

PrismML’s pitch is that extreme compression is now close to free. “What is compelling about Bonsai 2 27B is how little capability is lost despite such a dramatic reduction in footprint,” said Ion Stoica, the UC Berkeley professor who advises the company. Founder and CEO Babak Hassibi described the release as “closing the quality gap while keeping the same deployment advantages.”

What Changed From Bonsai 27B

The recipe is the same as July’s: every weight is −1, 0, or +1, with one shared FP16 scale per group of 128, giving about 1.72 bits per weight in principle and 1.76 as packed. The language model shrinks from 53.8 GB in FP16 to 5.93 GB, a 9.1× reduction; an optional 0.63 GB vision tower adds image input, and the 262K-token context is preserved. Two refinements account for the gain in retained quality.

First, selective precision. The July build ternarized nearly everything; Bonsai 2 keeps 26.2 million parameters — the recurrent state path of Qwen3.8’s linear-attention layers plus normalization weights — in full precision. That is 0.0976% of the model, about 52 MB, and moves the effective rate from 1.71 to 1.72 bits. Second, a rotated weight basis: weights are stored after a blockwise Walsh–Hadamard rotation (block size 1,024, fixed ±1 signs), with the matching transform applied to activations at runtime. Rotations spread outlier values across a block so ternary rounding loses less; the whitepaper says the runtime cost is negligible next to the matrix multiply, though at batch size 1 it remains “one of the larger non-matrix-multiply costs of a decode step.”

Because Qwen3.8-27B is a hybrid-attention model (about 75% linear, 25% full attention), PrismML wrote custom CUDA and Metal kernels that read the packed ternary weights directly. Stock llama.cpp cannot load these files; you need PrismML’s fork or a runtime that has integrated it — LM Studio, Jan, Ollama, vLLM, Docker Model Runner, or MLX on Apple devices.

The Numbers

Evaluated in thinking mode at the highest reasoning effort, Ternary Bonsai 2 27B averages 83.9 against 85.4 for Qwen3.8-27B in FP16 — and 83.6 for the previous-generation Qwen3.6-27B at full precision. The 5.9 GB build, in other words, edges out the 54 GB model it replaced. By category it holds 96.57 on math (vs. 97.06), 81.58 on coding (vs. 82.17), 77.57 on agentic tool calling (vs. 79.74), and 78.59 on vision (vs. 81.64), while instruction following rises to 82.66 from 81.25.

The contrast PrismML draws is with conventional 2-bit quantization. An IQ2_XXS build of the same model is larger (7.3 GB at 2.2 bits) yet averages 75.2, and it fails unevenly: 78.6 on AIME26 and 70.05 on LiveCodeBench, where Bonsai 2 scores 95.83 and 90.07. That selective collapse on hard reasoning while surface fluency survives is, the whitepaper argues, “easy to miss in casual testing and expensive to discover in production.” On the company’s “intelligence density” metric (score per GB, chart above), Bonsai 2 lands at 0.444 versus 0.275 for IQ2_XXS and 0.051 for FP16.

Long-horizon agentic work, the weak spot PrismML flagged in July, is the clearest advance. On Terminal-Bench 2.1 and SWE-bench Verified — reported for a Bonsai model for the first time — the ternary build scores 52.8 and 60.8 against 69.7 and 80.6 at full precision, roughly three-quarters retention. On τ²-Bench it reaches 80.2, up from 73.6 for the previous release. Launch demos show it driving Cline coding-agent loops and computer-use tasks on a single RTX 5090, where it decodes at 142.5 tokens per second; an M5 Max MacBook reaches 46.8, an M5 Pro 27.7, and an M4 Pro about 18.

In the Browser

The Hugging Face WebML community, which wrote custom WebGPU kernels for the 1-bit Bonsai 27B in July, has published a Space that runs Ternary Bonsai 2 27B entirely in the browser — a single self-contained HTML page with no server-side inference. It needs a WebGPU-capable browser (Chrome, Edge, or recent Safari) and enough GPU memory for the roughly 6 GB of weights plus KV cache, which a recent laptop can supply. A 27B-class model running as a web page, with nothing installed and no data leaving the machine, is a clear marker of how far the local-inference floor has moved in six months.

What This Means

Two caveats belong next to the headline. The 98.2% figure is measured at the highest reasoning effort; the whitepaper’s medium-effort table shows wider gaps (AIME25 74.58 vs. 86.25, GPQA Diamond 75.56 vs. 84.34), so users who cap thinking budgets for latency will see more degradation. And unlike July, there is no 1-bit Bonsai 2 yet: the Bonsai-2 collection lists only ternary builds, so the 3.9 GB phone-class footprint still means the older model.

Even so, the direction is clear. Six months ago the question was whether an extreme-quantized 27B model was usable at all. Now a 6 GB file matches the previous-generation full-precision model and runs a coding agent on a laptop. For students and researchers that is a capable multimodal reasoning model with a 262K context, no API key, and no per-token bill — and, for the first time, one you can try by opening a tab.

This post was drafted with AI assistance and reviewed by RITS staff.

Sources

• PrismML — Introducing Bonsai 2 27B: Near-Lossless Compression in a 9x Smaller Footprint
• PR Newswire — PrismML Launches Bonsai 2 27B, Its Most Capable Model Yet
• PrismML — Ternary Bonsai 2 27B whitepaper (PDF)
• Hugging Face — prism-ml/Ternary-Bonsai-2-27B-gguf model card
• Hugging Face — Bonsai-2 collection
• Hugging Face Spaces — Run Ternary-Bonsai-2-27B locally in your browser on WebGPU