Bonsai 27B: A 27B-Class AI Model That Runs on a Phone

On July 14, 2026, PrismML released Bonsai 27B — a pair of extreme-quantization builds of Alibaba’s Qwen3.6-27B that the company calls the first 27B-class model capable of running on a phone. The 1-bit variant fits in 3.9 GB and runs on an iPhone 17 Pro, while a higher-quality ternary variant fits in 5.9 GB for laptops. Both ship under the Apache 2.0 license, free to download, and keep the bulk of the original model’s benchmark performance despite shrinking it by roughly 9–14×.

The pitch is straightforward: a model class that until now lived in the cloud can move onto the device — private by default and available without a network round trip. PrismML frames Bonsai 27B not as a lightweight chat toy but as a model “built to do real work: reasoning through complex tasks, planning multi-step workflows, writing and debugging code.”

Two Variants, One Base Model

Both builds start from Qwen3.6-27B, the dense 27-billion-parameter multimodal model Alibaba released in April, and compress its weights far below the usual 4-bit floor:

• Ternary Bonsai 27B — weights constrained to three values {−1, 0, +1} with FP16 group-wise scaling, landing at a true 1.71 bits per weight and a 5.9 GB footprint. It retains 94.6% of the full-precision model’s score (80.49 vs. 85.07 across a 15-benchmark suite). This is the “laptop-class quality” build.
• 1-bit Bonsai 27B — binary {−1, +1} weights at 1.125 bits per weight and just 3.9 GB, retaining 89.5% of full precision (76.11 average). This is the “phone-class footprint” build.

Getting a 27B model onto a phone is a memory problem before it is anything else. As PrismML notes, a 12 GB iPhone only exposes about 6 GB to any single app — so at roughly 4 GB, the 1-bit build “is the first to pass through with room to work.” On an iPhone 17 Pro it runs at about 11 tokens/second; on an M5 Max MacBook it reaches up to 87 tok/s (1-bit) and 58 tok/s (ternary), and up to 163 tok/s on an RTX 5090.

How the Compression Works

Unlike BitNet-style approaches that pretrain a low-bit network from scratch, Bonsai is a post-training quantization: each weight is stored as a small integer t_i times a shared FP16 scale s_g for its 128-weight group (w_i = s_g · t_i). The low-bit representation spans embeddings, attention projections, MLP projections, and the LM head, while normalization parameters stay in higher precision. A 4-bit vision tower keeps the model multimodal, and a 4-bit KV cache shrinks the memory cost of its 262K-token context window from about 17.2 GB down to 4.3 GB.

PrismML is pointed about the “true bits per weight” framing — 1.71 and 1.125 with “no high-precision escape hatches behind a low-bit label.” That is a jab at conventional sub-4-bit quantization, which can advertise a low average while quietly keeping sensitive layers at higher precision. The company measures the payoff with an “intelligence density” metric — roughly, accuracy per gigabyte — where the 1-bit build reaches 0.53 per GB, about 10× the full-precision baseline and roughly 2.7× the best competing low-bit build.

What This Means

The honest caveat is in the numbers. Compression is not free: the 1-bit build sheds the most on the hardest tasks, dropping to 66.0 on agentic/tool-calling and 59.6 on vision, versus 80.0 and 72.6 for the full-precision model. The ternary build holds up far better and is the one to reach for when quality matters. Even so, both degrade gracefully rather than collapsing — independent analysis notes that a conventional 2-bit build (IQ2_XXS) scores a respectable 88.9 on MMLU-Redux while falling apart on AIME, LiveCodeBench, and agentic tasks, exactly the selective failure Bonsai is designed to avoid.

Practically, Bonsai 27B slots into the existing local-AI stack: it runs through llama.cpp, MLX, vLLM, Ollama, LM Studio, and Jan, with iOS access via the Locally AI app. For students and researchers, that means a genuinely capable reasoning-and-coding model that fits on a laptop — or a phone — with no API key, no usage bill, and no data leaving the device. It is the clearest sign yet that the frontier of “local AI” is no longer about tiny models, but about squeezing full-size ones down to size.

This post was drafted with AI assistance and reviewed by RITS staff.

Sources

• PrismML — PrismML Announces 1-bit Bonsai 27B: The First 27B Model to Run on a Phone
• PrismML — Bonsai 27B technical overview
• Hugging Face — prism-ml/Ternary-Bonsai-27B-gguf model card
• MarkTechPost — PrismML Releases Bonsai 27B: 1-bit and Ternary Builds of Qwen3.6-27B
• 9to5Mac — PrismML releases Bonsai 27B, claiming first major AI model of its size fit for iPhone