5 stories tagged on-device-llm, newest first.
18 points by anemll | 3 comments
A tweet from ANEMLL reporting a 27B-parameter Qwen model running on the iPhone 18 Neural Engine at 15–70 tokens per second using 1.5-bit quantization.
501 points by HenryNdubuaku | 169 comments
Product page for Cactus Needle 2, an open 14MB agentic LLM for tool calling, device use, and structured extraction on phones, wearables, and tiny devices.
626 points by franze | 137 comments
Landing page for apfel, an MIT-licensed tool exposing Apple's built-in on-device LLM on Macs as a CLI, OpenAI-compatible server, and chat.
689 points by anemll | 314 comments
An X post from @anemll showing a 400B-parameter MoE language model running locally on an iPhone at about 0.6 tokens per second via a Flash-MoE app.
409 points by rafaelcosta | 194 comments
Blog post describing how an iPhone 16 Pro Max produces garbage output when running LLMs with Apple's MLX framework, and investigating the cause.