Cactus Compute launches Needle 2, a 14 MB open-source 45-million-parameter model for tool calling

Cactus Compute (Cactus Compute) released Needle 2, an open model of 45 million parameters designed for tool calling, device control and embedded extraction, packaged as a single 14 MB binary. The weights are trained and laid out in CQ2-bit precision using proprietary quantization, and the entire model is signed inside the company’s C++ engine, eliminating the need for a runtime installation or download at inference time.

Reported throughput exceeds 500 tokens per second on Raspberry Pi 5, 400–1,500 tokens per second on Meta Quest 3S and Apple Vision Pro, and 300–700 tokens per second on phones priced under $200. The underlying premise is narrow and explicit: mapping a chaotic sentence to a stubbed function signature does not require global knowledge or open-ended prose, so 45 million parameters are sufficient, and the model targets hardware without GPU or NPU.

Needle 2 is built on what the team calls a Simple Attention Network. The design replaces the feed-forward network with a Hadamard MLP, retains GQA attention, adds an engram key-value memory drawn from compressed n-gram tables, and employs multi-lane hyper-connections. The network has 27 layers with a width of 512. The foundational research was posted as a pre-print on arXiv titled “A Controlled Study of Attention-Only Transformers”. Pre-training used a proprietary corpus of 115 billion tokens, plus an additional 38 billion tokens in in-the-wild training; for comparison, the team notes that LFM2.5-230M was trained on 19 trillion tokens. Compute cost is 70 MFLOPs per token, with 35 million of the 45 million parameters active in matrix multiplication, versus 460 MFLOPs for LFM2.5-230M, 540 MFLOPs for FunctionGemma 270M, and roughly 6,000 MFLOPs for Apple FM.

The weights never reside in RAM: the 2-bit codes are opened inside vector registers and mapped to scalar integer multiplications, keeping the compute path in int8. The binary probes the processor at startup and selects a kernel level—SDOT, NEON, AVX2, RISC-V vector, wasm SIMD, or scalar. A byte-level grammar that compiles your JSON summaries limits every emitted token; because the model knows which tokens are legal before logit generation, it skips up to 98 % of the vocabulary token space. Attention operates over a 256-token sliding window, and the system queue together with tool declarations are fixed as KV sinks, keeping memory around 28 MB regardless of conversation length. Five tools or fewer are routed directly; with more than five, a contrastive retrieval head embeds each schema once, ranks the query each turn, and returns only the top five—tools not selected become inaccessible. Every response carries a confidence value, the minimum between a post-hook head tuned to decoding token calls; out-of-scope requests return an empty call. The contract acts as a threshold: operate above it, ask again, or fall below it.

Needle 2 is delivered as ready-to-run binaries and a static library for macOS, Linux (x86-64, ARM64, ARMv7, RISC-V, MIPS32el), Windows, Android, iOS/watchOS/tvOS, and WebAssembly. According to Cactus Compute, Pebble already runs Needle locally in the Index 01 offline voice-assistant app. Target audiences include teams deploying firmware or apps on constrained hardware, wearable and IoT startups in early stages, mid-range consumer electronics manufacturers, robotics teams, and large device makers needing offline fallback. Industries span smart home, wearables, low-end mobile, in-vehicle driver-seat control, service robotics, kiosks and retail POS, routers and IP cameras, and monitored environments where audio must not leak. Applications cover voice-triggered actions on screenless devices, offline device control, field extraction from receipts and invoices, enum value tagging, and local routing that escalates to the cloud only under low confidence.

The team reports evaluation on five public benchmarks for tool calling using ordered strict exact match, a metric that requires exact alignment of function names, call order, and all arguments. Full score details were not released in this announcement.