Bonsai 27B: First 27B-Class Model to Run on a Phone
PrismML, a research-driven AI company founded by Caltech academics and backed by Khosla Ventures, Cerberus Capital Management, Google, and Samsung, has released Bonsai 27B, a multimodal large language model engineered to run locally on consumer smartphones and laptops. Built on the Qwen3.6 27B architecture, Bonsai 27B marks a significant breakthrough in model compression, delivering 27-billion-parameter capability within the memory constraints of modern mobile devices for the first time. The model utilizes a unified low-precision architecture that applies end-to-end quantization across the language network, embeddings, attention mechanisms, and MLP layers without relying on higher-precision fallbacks. It ships in two distinct variants: a ternary configuration using plus/minus one and zero weights with FP16 group-wise scaling, achieving 1.71 effective bits per weight and a 5.9-gigabyte footprint optimized for laptop deployment. The second variant employs binary plus/minus one weights at 1.125 effective bits per weight, compressing the model to 3.9 gigabytes to fit within the operational memory budget of an iPhone 17 Pro. Both versions support a 262,000-token context window, speculative decoding for accelerated inference, and a compact 4-bit vision tower for processing screenshots, documents, and camera feeds. Performance evaluations across fifteen benchmarks demonstrate strong retention of full-precision capabilities. The ternary variant preserves ninety-five percent of the baseline score, while the one-bit variant retains ninety percent. Notably, both configurations maintain high proficiency in mathematical reasoning, coding, structured tool calling, and instruction following, which are critical for autonomous agentic workflows. On an NVIDIA GeForce RTX 5090, inference reaches up to 163 tokens per second in the one-bit variant and 134 tokens per second in the ternary version. Apple M5 Max hardware achieves 87 and 58 tokens per second, respectively. The company reports an intelligence density exceeding 0.53 per gigabyte for the one-bit model, representing a tenfold improvement over full-precision baselines. This deployment milestone directly addresses the structural limitations of cloud-dependent AI systems, particularly for sustained agentic workloads that require hundreds of iterative steps. By executing locally, Bonsai 27B eliminates per-token API costs, ensures complete data privacy, and enables offline functionality. The architecture also facilitates hybrid cloud-edge systems, routing privacy-sensitive or routine tasks to the device while reserving frontier cloud models for complex computation. Developers can access the model today under the Apache 2.0 license, with native support for Apple MLX and NVIDIA CUDA. A limited-time developer preview API is also available to accelerate integration into mobile and desktop applications.
