Cactus Needle 3: One Set of Weights, an 8-29 MB Model at Any Depth

Cactus Compute shipped Needle 3 on September 17, 2026, a single set of weights that runs as anything from a 29-million-parameter, 8 MB model to a 121-million-parameter, 29 MB one, depending on how many layers you deploy. It is not a chat model. Needle 3 does three things — tool calling, structured extraction, and text embedding — and on Cactus’s own mobile tool-calling benchmark the full-depth model lands within 2.4 points of DeepSeek V4 Flash while running at up to 4,000 tokens per second on a Raspberry Pi 5. Weights, engines, and training code are released under Apache 2.0.

Advanced

One Set of Weights, Nineteen Models

The headline design idea is what Cactus calls a Laddered Simple Attention Network: the 20-layer model is trained so that every prefix from 2 to 20 layers is itself a deployable model. Four layers give you a 29M-parameter, 8 MB binary; eight layers give 52M; sixteen give 98M; the full stack is 121M and 29 MB. A developer picks the rung that fits the device with a single build flag (needle build --platform linux-arm64 --layers 8), and a fine-tuned adapter trained on the full model transfers down the ladder.

The “121M” figure overstates the compute. Most of those parameters sit in an engram: a hashed 2- and 3-gram memory of 18,432 slots, read by gather at layers 3, 7, 11, 15, and 19. Because a lookup is cheap, Cactus says the 121M model “does the arithmetic of a 50M one,” and that the architecture spends over 2x fewer MFLOPs per token than a transformer of the same configuration. The rest of the block is a Monarch Hadamard MLP in place of a dense feed-forward network (structured 32×32 Kronecker factors), grouped-query attention with RoPE, QK-normalization and a 3-tap causal convolution per channel, four residual lanes wired as multi-head hyper-connections with Sinkhorn-normalized writes, and a 1,024-token sliding window with global attention at layers 4, 9, 14, and 19. The tokenizer is an 8,192-piece SentencePiece BPE with a tied 768-wide embedding.

This is a meaningful step away from the original Needle, which RITS covered in May. That 26M model dropped feed-forward layers entirely on the theory that single-shot tool calling is a retrieval-and-assembly problem attention can handle alone. In July, the same team published a controlled study of attention-only transformers reporting that a standard transformer still leads by 0.47 nats at matched depth, and that the remaining gap traces to knowledge that lives in weights rather than in context. Needle 3 reads like the engineering answer to that paper: put the knowledge back, but in a hashed n-gram table and a low-rank MLP rather than a dense FFN.

Benchmarks: Strong on Tool Calls, Behind on Extraction

Cactus evaluated the shipped 2-bit binary (with its confidence gate on) against baselines run at f16 under vLLM, plus DeepSeek V4 Flash through its cloud API. Tool calling is scored as exact-match accuracy, extraction as field micro-F1, all on full test splits.

On Mobile Actions, the 121M Needle 3 scores 86.0% against 88.4% for DeepSeek V4 Flash, ahead of LFM2.5 1.2B (82.4%), Qwen3.5 0.8B (76.0%), FunctionGemma 270M (65.1%), and Apple’s 3B on-device foundation model (57.6%). The jump from Needle 2 (63.5%) is 22.5 points. On DroidCall, which requires multiple calls in the right order, Needle 3 scores 47.0% to DeepSeek’s 60.5%, with the 52M eight-layer rung (36.5%) still edging LFM2.5 1.2B (35.5%). The picture is less flattering on BFCL v4, a broader function-calling suite that penalizes irrelevant calls: Needle 3 lands at 50.2%, behind LFM2.5 1.2B (62.0%), LFM2.5 350M (59.1%), and Qwen3.5 0.8B (56.8%).

Extraction is where the small-model tax shows. Cactus describes Needle 3 as matching models “2-3x its size” on extraction, but on its own charts the full model trails LFM2.5 230M on DSTC8 (40.7% vs 53.0%), and on the two SNIPS suites it sits fifth behind LFM2.5 1.2B, Qwen3.5 0.8B, and LFM2.5 350M. Readers evaluating Needle for form or invoice parsing should treat the base model as a starting point for fine-tuning rather than a drop-in. That is also the pitch: on DroidCall, fine-tuning lifts every rung of the ladder by 18 to 36 points, and Cactus reports that from four layers (29M parameters) upward the tuned subnetwork passes DeepSeek V4 Flash.

What Ships, and What Doesn’t

The .cact file is memory-mapped and read in place, with Cactus Quants at 2.125 bits per weight. Every response carries a calibrated confidence score from a learned head, output is constrained by a byte-level grammar compiled from the developer’s tool schemas so it always parses, and a request that no declared tool can serve returns an empty list rather than a guess. Prebuilt engines under 1 MB exist for macOS, Linux (x86_64, arm64, armv7, riscv64, mipsel), Windows, Android, iOS, tvOS, watchOS, a WebAssembly build for the browser, and a WASI component. Existing Needle 2 deployments keep working with a generation=2 flag.

pip install cactus-needle

import needle

@needle.tool
def get_weather(city: str):
    "Get the current weather for a city."
    return {"city": city, "temp_c": 27, "sky": "clear"}

agent = needle.Needle(tools=[get_weather])
print(agent.run("what's it like in Lagos right now?")["results"])

Two caveats are worth reading before adopting it. First, “open” has an asterisk: the model was trained on 360 billion tokens of a proprietary structured dataset, local fine-tuning trains and exports at 4 bits, and the 2-bit post-training that produces the shipped binary runs only on the hosted Cactus Platform. You can reproduce the pipeline, but not the exact artifact. Second, the binary sends telemetry by default; the README documents turning it off with NEEDLE_TELEMETRY=0 and DO_NOT_TRACK=1, which matters for the air-gapped and privacy-first deployments the model is otherwise built for.

What This Means

Needle 3 is the clearest example yet of a small-model strategy that gives up on generality on purpose. Apple’s 3B foundation model, which shipped as a command-line tool in macOS 27 this week, is a general assistant that happens to run locally; Needle is a 29 MB component that does one job at a level a cloud model would otherwise handle. For a smartwatch, a robot controller, or an Android phone under $200, the difference between those two things is the difference between shipping and not shipping.

The ladder is the more transferable idea. Training once and deploying at any depth is a neat answer to the fragmentation problem in edge AI, where every device class has a different memory budget and every separate model size is a separate training run. If the approach holds up outside Cactus’s own benchmarks, expect other small-model teams to copy it.

Related Coverage

• Cactus Releases Needle: A 26M Distilled Model for On-Device Tool Calling — the original attention-only Needle, distilled from Gemini 3.1 Flash Lite, in May 2026.
• macOS 27 Ships a Foundation Model at the Command Line — Apple’s on-device model, one of the baselines in Cactus’s charts.
• DeepSeek V4.1-Flash: 552B Params, 890 Bytes of KV Cache Per Token — the cloud model Needle 3 is measured against.
• OpenBMB Releases MiniCPM5-2B, Topping Open Models Under 4B — a general-purpose small model, for contrast with Needle’s specialist approach.

This post was drafted with AI assistance and reviewed by RITS staff.

Sources

• Needle 3 release page — Cactus Compute — architecture, benchmark figures, fine-tuning results, platform table.
• cactus-compute/needle on GitHub — README, engines, training code, Apache 2.0 license, telemetry note.
• Needle 3 weights on Hugging Face — model card and per-platform engines.
• A Controlled Study of Attention-Only Transformers (arXiv:2607.18363) — the Cactus team’s July 2026 paper on Simple Attention Networks.
• The .cact format — Cactus blog — memory-mapped weights and Cactus Quants at 2.125 bits per weight.
• cactus-needle on PyPI — version 3.0.0 published September 17, 2026.
• Show HN: Cactus Needle 3 — launch thread.