Perplexity Open-Sources Lily, a Qwen3.6 Engine Tuned for Apple Silicon

Perplexity has open-sourced Lily, a Rust-and-Metal local inference engine built to run Qwen3.6-35B-A3B on Apple Silicon. The release makes available the runtime used for the local side of Perplexity Computer's Hybrid Compute workflow, with Perplexity reporting higher throughput than MLX-LM on an M5 Max. On Sept. 2, 2026, Perplexity published Lily in its public pplx-garden codebase. The project is a small server for one checkpoint: Qwen3.6-35B-A3B converted to MLX affine 4-bit weights. It exposes a minimal OpenAI-compatible HTTP API, always decodes greedily, and is licensed under Apache-2.0. The release is therefore an inference-engine opening rather than a broad local-model runtime: its README validates the exact 35B-A3B architecture and quantization layout at load time. The context is Perplexity's Hybrid Compute design, in which cloud models handle research, web search, reasoning, and planning while a local model works with private files, sensitive information, and on-device actions. Perplexity's Hybrid Compute announcement says the broader product is available to Pro, Max, and Enterprise subscribers, with three local models and support for Apple silicon Macs running macOS 15+ with at least 24GB of unified memory. Those product-level requirements should not be conflated with the standalone Lily package, whose current README lists Apple GPU family 10 or later, meaning M5 and newer, macOS 26 or later for Metal tensor operations, Rust 1.92, and the exact Qwen3.6-35B-A3B MLX affine 4-bit checkpoint with group size 64. Lily's narrower scope is a direct counterpoint to the general-purpose MLX and MLX-LM stack commonly used for local inference on Macs. Qwen3.6-35B-A3B has 35 billion total parameters but activates about 3 billion for each token; a router scores 256 expert subnetworks, selects eight, and adds one shared expert. The model also combines 10 full-attention layers with 30 Gated DeltaNet layers, creating uneven expert workloads, growing attention-cache traffic, and sequential recurrent-state updates. Its groupwise affine 4-bit format reduces roughly 70GB of bfloat16 weights to a 19.4GB checkpoint. Lily reconstructs those weights tile by tile inside grouped matrix multiplication, so the complete expanded weight array is never created in unified memory. Perplexity benchmarked the engine on one MacBook Pro with an M5 Max, a 40-core GPU, and 128GB of unified memory. Across 10 prompt lengths and 10 decode contexts from 256 to 128K tokens, using batch one and identical 4-bit checkpoint bytes, Lily averaged 4,156 prefill tokens per second versus 3,388 for MLX-LM, a 1.23x advantage, and 170.0 decode tokens per second versus 126.4, a 1.35x advantage. At a 4K-token prompt and 4K-token decode context, the reported figures were 5,749.9 versus 4,737.5 prefill tokens per second and 186.6 versus 140.9 decode tokens per second. Lily was faster at every recorded point, with measured advantages ranging from 1.12x to 1.42x for prefill and 1.31x to 1.37x for decode. A teacher-forced comparison across 192 positions found perplexity only 0.04% higher and the same top-ranked token at 96.35% of tested positions. The measurements are detailed in Perplexity's Apple Silicon analysis, but they describe one hardware configuration rather than a universal result across Macs or workloads. Early developer discussion was broadly positive about the reported speed, while questions focused on performance on M5 Pro systems, clearer explanations of the engine, and whether local inference could replace the wider Computer workflow or its cloud-connected capabilities. Those concerns match the public server's boundaries: it supports text-only messages and stream: false, while rejecting sampling parameters, streaming responses, tools, response formats, multimodal content, and speculative decoding. Lily is consequently best understood as a focused local serving layer, not a standalone replacement for Perplexity Computer's orchestration, cloud research, or privacy gate. Perplexity's stated outlook is to broaden support across models, chips, and serving workloads, making the release an early example of an open inference runtime optimized simultaneously for a model architecture and a hardware platform.