Bonsai 27B (PrismML): What 3.9 GB Actually Costs You

Every write-up of PrismML's Bonsai 27B reprints the same four numbers: 3.9 GB, 5.9 GB, 90%, 95%. Nobody has done the two pieces of arithmetic that decide whether the thing will actually load on your machine. So we did them — the shipped file sizes against the marketed ones, and the KV cache against an iPhone's memory budget. Both change the answer.

What Bonsai 27B actually is

PrismML released Bonsai 27B on 14 July 2026 under Apache 2.0, with weights on Hugging Face under the prism-ml organisation and the runtime forks and demo code on the company's GitHub. It is not a model trained from scratch. It is Qwen3.6 27B, compressed end-to-end to extremely low-bit weights, with the low-bit representation running across embeddings, attention, MLPs and the LM head. The vision tower is handled separately at 4 bits.

The parameter count you see quoted — 27.8 billion — breaks down as roughly 27.3B language weights (about 24.8B across a 64-block backbone plus about 2.5B of embeddings and LM head) and a 0.46B vision tower. Architecture is Qwen3.6's hybrid attention, roughly 75% linear and 25% full, with a growing full-attention KV cache on only 16 of the 64 layers. That last detail matters more than it sounds, and we will come back to it.

Two builds ship. The 1-bit build uses binary weights in {−1, +1}, packed as one sign bit plus a single FP16 scale amortised over each group of 128 weights — 1.125 effective bits per weight, marketed at 3.9 GB, which PrismML frames as about 14.2x smaller than the roughly 53.8 GB FP16 baseline. The ternary build uses {−1, 0, +1} with one shared FP16 scale per group of 128 — 1.71 effective bits per weight, marketed at 5.9 GB, about 9.4x smaller.

Both keep the base model's 262,144-token context. Both do text, vision and tool calling. PrismML bills the 1-bit build as the first 27B-class model to run on a phone — their framing, not an independently adjudicated first.

One attribution to get straight, because it is being reprinted loosely: a widely shared Medium write-up says the community WebGPU build credits its kernels to two hosted frontier models. Read carefully, that claim is about the browser build's WebGPU kernels, not about the kernels PrismML ships — and the Medium piece cites no source for it. PrismML's own pages credit forks it maintains (llama.cpp for CUDA and Metal, MLX for Apple Silicon, mlx-swift for iOS) and make no AI-authorship claim anywhere on prismml.com, docs.prismml.com or the four Hugging Face cards. If you repeat the claim, scope it to the WebGPU demo and label it uncited.

Marketed size vs. the file you download

Here is the first arithmetic problem. 3.9 GB and 5.9 GB are information-theoretic figures: parameter count multiplied by effective bits per weight, divided out. They are what the model would weigh in a format that could store 1.125 or 1.71 bits per weight exactly. No shipping runtime can.

Neither GGUF nor MLX has a native 1.58-bit slot. Ternary weights therefore get padded into 2-bit slots, and the ternary build overshoots its headline number by roughly 20%. PrismML says this out loud, but only in the fine print of the ternary model card, which concedes a deployed footprint of about 7.2 GB against an ideal size of 5.9 GB — and adds that it exceeds the iOS per-app memory budget of roughly 6 GB, making it laptop and GPU only. The 1-bit build is the honest one here: 3.53 GiB on disk is fractionally under its 3.9 GB headline.

Vision is the second omission. Both headline numbers are language-model-only. The mmproj vision tower is a separate download: about 0.63 GB in 4-bit HQQ, about 0.93 GB in BF16 reference. A 3.9 GB multimodal model is not a 3.9 GB multimodal download.

Table: DataLLM Lab, compiled from PrismML's launch page, the docs file table and the four prism-ml Hugging Face cards, July 2026; the Q4_K_M size is the published file size on the Unsloth Qwen3.6-27B GGUF repo, the Q4_K_XL figures are PrismML's own baseline. Bonsai sizes and retention figures are vendor-published. Retention is the exact ratio, not PrismML's rounded headline.

The practical consequence: if you provisioned 6 GB of headroom for the ternary build based on the number in every headline, it will not load. That is not a rounding quibble. It is a 1.3 GB miss on a 5.9 GB claim.

The phone math nobody prints

Second arithmetic problem, and the reason this article exists: the 3.9 GB phone footprint and the 262,144-token context cannot both be true at once.

Weights are the small half of the bill. PrismML publishes one KV cache figure — about 4.3 GB at the full 262K window with near-lossless 4-bit KV quantization — and states that enabling the 4-bit cache shrinks the context-dependent term about fourfold, which puts the FP16 cache at roughly 17.2 GB. It also publishes measured peaks for the 1-bit llama.cpp build: 5.2 GB at 4K context, 5.6 GB at 10K, 11.6 GB at 100K, dropping to about 6.8 GB at 100K and about 9.4 GB at the full 262K with 4-bit KV. The MLX build runs consistently higher: 5.9 GB at 4K, 6.3 GB at 10K, 12.2 GB at 100K.

Those numbers are internally consistent, and you can back out a per-token cost from them: roughly 64 KiB per token at FP16, roughly 16 KiB at 4 bits, plus about 1.2 GB of fixed runtime overhead. (That is unusually cheap for a 27B model — the hybrid attention means only 16 of 64 layers carry a growing full-attention cache.) Extrapolating linearly from PrismML's published points gives a usable model of what fits:

Table: DataLLM Lab. The 4.3 GB 4-bit cache and the 6.8 GB / 9.4 GB peaks are PrismML-published; the 17.2 GB FP16 figure is implied by PrismML's own statement that 4-bit KV cuts the context term about fourfold. The 8K, 32K and 100K KV figures are modeled by linear extrapolation, calibrated so the model reproduces all four published llama.cpp peaks to within about 0.1 GB. Treat them as estimates, not vendor specs — and note that PrismML's MLX peaks, which are the ones that matter on iOS, run roughly 0.7 GB higher than the llama.cpp ones at the same context.

Read the last column. On an iPhone 17 Pro Max — 12 GB of RAM, of which PrismML says roughly 6 GB is available to any one app — the 1-bit build fits at 8K context on the llama.cpp numbers and is already tight there on MLX, and is out of budget well before 100K. The 262K window is real — on a workstation. On a phone, you have a 27B-class model with a few-thousand-token working context. That is still a remarkable thing. It is not the 262K on-device agentic loop the launch framing implies.

Where the quality actually goes

PrismML's 15-benchmark thinking-mode suite, run in-house with EvalScope and vLLM on H100 under identical decoding and scoring, averages 85.07 for Qwen3.6 27B at FP16, 80.49 for ternary and 76.11 for 1-bit. That is 94.6% and 89.5% — rounded in the marketing to 95% and 90%. Every one of these numbers is vendor-reported and, as of July 2026, nobody has replicated the suite independently.

The averages are the least interesting part. Compression does not degrade capabilities evenly, and the pattern is the opposite of what the marketing foregrounds.

The raw scores: math 95.33 / 93.40 / 91.66 across FP16, ternary and 1-bit; coding 88.74 / 85.96 / 81.88; knowledge and STEM 83.10 / 76.96 / 73.39; agentic tool calling 80.01 / 74.01 / 66.00; instruction following 78.47 / 71.77 / 65.74; vision 72.60 / 65.19 / 59.60.

At 1 bit, math keeps 96.2% of full precision. Agentic tool calling keeps 82.5% — a relative loss roughly 4.6x larger. Instruction following keeps 83.8%, vision 82.1%. The 90% average is carried almost entirely by math and code, and it masks the collapse of exactly the capability the launch page foregrounds: on-device agentic loops and tool calls. The single worst individual result in the suite is τ²-Bench, where the 1-bit build scores 61.34 against 82.90 at FP16.

PrismML is not hiding this, but you have to read two pages to see the contradiction. The GGUF model card lists as a stated limitation that agentic coding workflows are not yet a strong target of this release, and acknowledges measurable instruction-following gaps versus baseline — 65.74 against 78.47 on the category average, and 52.36 against 68.03 on IFBench. That sits directly against the launch page's multi-step agentic framing. Nobody has put those two statements next to each other before.

If you were planning to point this at a tool-calling harness, that is the number to plan around. Compare with what a full-precision open model gives you in the open-weights landscape for 2026.

What outside testers found

Independent evidence is thin, but it exists and points the same direction. Arman Jafarnezhad, an independent developer, ran 13 models over the same 98 questions across 7 categories and 3 difficulty levels, 3 runs each, on an NVIDIA Jetson Orin with about 30 GB of unified memory. He measured Qwen3.6-27B at 94.2%, ternary Bonsai at 86.2% and 1-bit Bonsai at 82.9%. He explicitly labels it a quick experiment and not a rigorous evaluation, and the hardware is not representative. Take it as directional signal only — but note the gap it shows is wider than PrismML's own suite implies, not narrower.

The Hacker News launch thread adds hands-on friction reports: one user wrote that he downloaded "two of the official ones in LM Studio, both 3.6gb, and neither loaded", another reported roughly 1 token per second on an old 4-core i5 under WSL2, and at least one questioned whether 27B is the right size target for a phone at all. Anecdotal, but the LM Studio loading issue is worth knowing before you plan an evening around it — the format needed PrismML's llama.cpp fork at launch — see our notes on LM Studio and unusual quant formats and on runtimes beyond Ollama.

On throughput, PrismML's own pages disagree with each other. The launch page reports up to 163 tok/s for 1-bit and 134 tok/s for ternary on an RTX 5090, and up to 87 tok/s (1-bit) / 58 tok/s (ternary) on an M5 Max. The GGUF card reports the same M5 Max at 66.4 tok/s and an H100 at 104.8 tok/s. The difference is runtime: the higher Apple numbers are MLX, the lower ones are llama.cpp on Metal. Always name the runtime when you quote a tok/s number for this model. On iPhone 17 Pro Max, PrismML reports about 11 tok/s generation with prompt processing around 111 tok/s, and warns that sustained phone generation is thermally constrained — a continuous battery-drain run settled at 10.8 tok/s and about 672 tokens per 1% of battery, roughly 67,000 tokens on a full charge.

PrismML also markets an intelligence-density metric — 0.53 score per GB for the 1-bit build, claimed as more than 10x the full-precision baseline and roughly 2.7x the best low-bit alternative available. It is a vendor-defined metric with no external standard and an unnamed comparison target. Treat it as positioning, not measurement.

Running it: MLX and llama.cpp

On Apple Silicon, MLX is the fast path (Mac, iPhone and iPad, via PrismML's MLX and mlx-swift forks):

mlx_lm.generate --model prism-ml/Bonsai-27B-mlx-1bit --prompt "..."

On NVIDIA and for GGUF generally, PrismML ships a llama.cpp fork with fused low-bit kernels for CUDA and Metal:

./llama-cli -m Bonsai-27B-Q1_0.gguf --mmproj mmproj.gguf -c 0 -p "..."

Add -ngl 99 to offload to GPU. The fused kernels consume the packed weights directly, applying group-wise scales inside the matmul rather than expanding the weights into a dense FP16 tensor — which is why the memory numbers above hold at inference time rather than only on disk. PrismML suggests temperature 0.7, top-p 0.95, top-k 20. Thinking mode is togglable per request and budgeted through thinking_budget_tokens.

The --mmproj flag is optional. The vision tower loads only when an image arrives, and images are downscaled to roughly 1,024 vision tokens by default on several backends to keep latency sane. There is also an OpenAI-compatible API surface and a WebGPU browser demo. If you are new to any of this, start with our guide to running LLMs locally in 2026, and for a sense of how the memory bookkeeping compares on a much larger open model, see GPT-OSS 120B hardware requirements or the walkthrough in running DeepSeek locally.

A decision rule

Sort by the memory you actually have, not by the headline:

• 24 GB or more of VRAM / unified memory. Do not use Bonsai. Run Qwen3.6 27B at Q4_K_M and keep the 4.6 points (ternary) to 9.0 points (1-bit) of suite average that compression gives up — 8 to 11 points on the one independent quality comparison we could find. The compression buys you nothing you need.
• 12 to 16 GB. Ternary Bonsai is a real option, and at this tier it is close to the only option: about 7.2 GB deployed against 16.8 GB for a Q4_K_M build of the same base, which does not fit at all. Provision for 7.2 GB, not 5.9 GB, and add the mmproj if you want vision.
• 8 GB or less, or a phone. The 1-bit build is the only one that fits, and only with 4-bit KV cache enabled. Budget for a working context in the low tens of thousands of tokens at most, and treat it as a short-context text and vision-Q&A model.
• Any tool-calling or agent loop. Weight the 82.5% agentic retention, not the 90% average — and note PrismML's own card says agentic coding is not yet a strong target of this release. If the workload is agentic, this is the wrong size of compromise; put a hosted model behind it.

What Bonsai 27B genuinely proves is that a 27B-class model can be squeezed to a bit and a fraction per weight and still do arithmetic and code at near-baseline. That is a real result. The marketing simply prices it in the wrong units: the size that matters is not the weight file, it is the weight file plus the cache plus the vision tower plus runtime overhead — and on the device the campaign is built around, that sum runs out of room long before 262K tokens do.

FAQ