72 stories tagged local-llm, newest first.
135 points by erikkannike | 95 comments
Blog post introducing a fine-tuned Qwen3-4B-Instruct model, trained on a MacBook Air, that avoids common AI writing tics and runs locally via Ollama.
308 points by albinstman | 85 comments
Llmspeed is a web tool that simulates the token generation speed of local LLMs on chosen hardware, model, and quantization settings before buying hardware.
356 points by Anon84 | 83 comments
GitHub repository for PS5LM, a port of llama.cpp that runs GGUF language models natively on a jailbroken PlayStation 5.
102 points by niztk | 55 comments
Landing page for Pulse Local, a free Mac budgeting app that answers spending questions using small AI models running entirely on the user's machine.
7 points by Ir0n | 0 comments
A page from meshapplied.com about running a Qwen LLM at 255 tokens per second on a single Nvidia RTX 5090 GPU; no page content was captured.
85 points by joshcsimmons | 19 comments
GitHub repository for LemonSeed Studio, an iPad IDE running local LLM inference on an AMD GPU over Thunderbolt using the Linux amdgpu driver embedded in DriverKit.
892 points by snehesht | 402 comments
GitHub repository for Strata, an inference engine offering one-click install to run Qwen3.8-Flash-Next locally on consumer hardware with an OpenAI/Anthropic-compatible API.
187 points by anerli | 88 comments
GitHub repository for Magnitude, an open source Rust inference engine for local AI agents that autotunes GPU kernels to your hardware, claiming up to 2x faster than llama.cpp.
14 points by turantekin | 7 comments
Landing page for Parrot, a free open-source Mac meeting recorder with a live AI copilot that suggests answers during calls, using local or cloud models.
614 points by bashbjorn | 194 comments
A NobodyWho blog post showing how to build "Jev," a meme-inspired chatbot or LLM project, in 25 lines of Python.
683 points by ilreb | 285 comments
OpenJev is a browser-only demo that runs a local language model on your GPU to compare reading option probabilities directly versus decoding token by token.
417 points by felineflock | 171 comments
A Level1Techs forum post presenting technical experiments on how quantization and implementation differences make locally run LLMs perform worse than reference versions.
361 points by aratahikaru5 | 122 comments
Landing page for Nativ, an open-source macOS app for downloading and running open AI models locally on Apple Silicon without accounts or cloud.
331 points by alekq | 224 comments
A blog post critiquing OpenCode, an open-source AI coding agent, listing annoying behaviors and alarming security design problems based on its source code.
882 points by vforno | 223 comments
GitHub repo for Colibrì, a dependency-free C engine that runs the 744B GLM-5.2 MoE model on a ~25GB-RAM machine by streaming experts from disk.
390 points by livestyle | 175 comments
GitHub repository by jamesob collecting notes, runners, and tools for running state-of-the-art LLMs locally.
1124 points by stared | 699 comments
A Quesma blog post by Piotr Migdał arguing that the dense Qwen 3.6 27B model is the best choice for running LLMs locally for development, with impressions and setup guidance.
559 points by TechTechTech | 266 comments
Unsloth documentation guide explaining how to run Z.ai's GLM-5.2 open model locally using Unsloth Dynamic quantized GGUFs.
433 points by alphabettsy | 227 comments
Blog post by Alex Ellis comparing local Qwen models with Claude Opus, arguing local models are a different tool rather than an Opus replacement, based on real project use.
1195 points by cloudking | 501 comments
Ask HN thread asking whether anyone has fully replaced Claude or GPT with a locally run model for daily coding, and what hardware and token speeds they get.
464 points by kkm | 114 comments
A blog tutorial on running Gemma 4 and Qwen3.6 locally on macOS with llama.cpp, MTP speculative decoding, and multimodal support as a coding agent.
659 points by cafkafk | 266 comments
Blog post about running Google's Gemma 4 language model locally on a 2016-era Intel Xeon server.
451 points by asenna | 127 comments
Blog post describing how the author used Gemma 4 running locally on a five-year-old MacBook to index a year of personal video footage overnight.
627 points by zambelli | 225 comments
GitHub repository for Forge, a Python framework adding guardrails to improve reliability of self-hosted LLM tool-calling and multi-step agentic workflows.
411 points by caust1c | 173 comments
Blog post by antirez reflecting on the rapid popularity of DwarfStar 4 (DS4), his local AI inference project, and its future plans beyond DeepSeek v4 Flash.
525 points by shintoist | 156 comments
A blog post describing a setup for running local LLMs such as Qwen via LM Studio on an Apple M4 Mac with 24GB of memory.
461 points by tamnd | 133 comments
GitHub repository for ds4 by antirez, a local inference engine for running the DeepSeek 4 Flash model on Apple Metal, released under the MIT license.
471 points by Zetaphor | 122 comments
Opinion blog post arguing against using Ollama, recounting its history as a llama.cpp wrapper, attribution disputes, cloud pivot, and recommending alternatives.
436 points by MattHart88 | 190 comments
GitHub repository for Ghost Pepper, an open-source macOS hold-to-talk speech-to-text app using local WhisperKit and LLM cleanup models.
380 points by vbtechguy | 94 comments
A blog post walking through setting up Google Gemma 4 26B for local inference on macOS using LM Studio 0.4.0's headless lms CLI, and using it with Claude Code.
517 points by AbuAssar | 107 comments
Homepage for Lemonade, AMD's open source local LLM server that runs models on GPUs and NPUs on Windows, Linux, and macOS.
588 points by redundantly | 303 comments
Ollama blog post announcing a preview of Ollama running on Apple's MLX framework for Apple Silicon Macs.
447 points by yogthos | 247 comments
GitHub repository for ATLAS (Adaptive Test-time Learning and Autonomous Specialization), a system for running local LLMs on consumer GPUs for coding tasks.
346 points by matthiaswh | 154 comments
Ente's blog post announcing the first release of Ensu, an offline app for running LLMs locally on your device with full privacy and control.
379 points by mft_ | 119 comments
GitHub repository for Flash-MoE, a project for running a 397B-parameter mixture-of-experts model on a laptop using Metal inference code.
364 points by brainless | 67 comments
Announcement page for Unsloth Studio (Beta), an open-source no-code web UI for training, running, and exporting open AI models locally.
404 points by Vaslo | 128 comments
Home Assistant community forum post describing a user's setup of a fully local voice assistant using Home Assistant Assist and llama.cpp, replacing Google Home.
1375 points by ricardbejarano | 326 comments
CanIRun.ai is a web tool that estimates which AI models your machine can run locally, using browser APIs to detect hardware and grading each model's expected performance.
424 points by Curiositry | 138 comments
Unsloth documentation guide explaining how to run Alibaba's Qwen3.5 model family locally using GGUF quantized versions, with updates on improved quantization.
410 points by lostmsu | 219 comments
VentureBeat article on Alibaba's open-source Qwen3.5-Medium models (122B and 35B) claiming Claude Sonnet 4.5-level performance on local hardware.
372 points by fugu2 | 197 comments
Blog post by Tim Plaisted explaining how to connect Claude Code to a local open-source LLM when the usage quota is exhausted.
331 points by tmaly | 133 comments
Ask HN thread asking how people implement retrieval-augmented generation locally with minimal dependencies for code or complex documents.
551 points by mpweiher | 304 comments
A Substack newsletter post guiding software engineers on running local coding models as an alternative to paying for Claude Code, covering tooling and setup steps.
326 points by jmorgan | 158 comments
Ollama blog post from September 24, 2025 announcing a web search feature for its models and API.
400 points by jerryliu12 | 115 comments
GitHub repository for Dayflow, an MIT-licensed macOS app that builds an automatic timeline of your day by analyzing screen content with local or cloud VLMs.
981 points by mkagenius | 261 comments
A blog post describing how the author built a fully local, offline AI workspace with LLM chat and code execution, with no cloud dependency.
404 points by amram_art | 221 comments
Ollama's Turbo preview page, a paid service for running open models on datacenter-grade hardware through the Ollama app and API.
511 points by BUFU | 263 comments
Ollama blog post from July 30, 2025 announcing its new desktop app for running local language models.
369 points by runjuu | 186 comments
Landing page for Refine, a macOS grammar-checking app powered by local AI models that keeps writing on-device as a private alternative to Grammarly.
386 points by codelion | 62 comments
Show HN post introducing AutoThink, a technique that adaptively allocates thinking tokens by query complexity and uses steering vectors to improve local LLM reasoning.
331 points by LorenDB | 73 comments
Ollama blog post from May 15, 2025 announcing a new engine for running multimodal models locally.
475 points by redman25 | 104 comments
Documentation page in the llama.cpp GitHub repository describing multimodal (vision) input support via libmtmd and the tools that use it.
1061 points by felixrieseberg | 257 comments
Download page for Clippy, a desktop app with a 1990s Windows-style interface for chatting with large language models running locally via llama.cpp.
339 points by marban | 203 comments
A personal blog post describing how the author built a budget computer for running AI models locally and privately.
496 points by kgwgk | 97 comments
GitHub repository for llama.vim, a Vim plugin that provides local LLM-assisted code and text completion using llama.cpp.
525 points by prettyblocks | 271 comments
Ask HN thread asking what people have built with tiny 0.5B–3B parameter language models, such as those available on Ollama.
535 points by rbanffy | 302 comments
A Nature technology feature on why researchers are running small, openly available AI models locally on laptops, with guidance on getting started.
325 points by tcarambat1010 | 68 comments
GitHub repository for AnythingLLM, an open-source desktop and Docker AI application with built-in RAG, agents, embeddings, and vector database.
319 points by nkaz123 | 78 comments
GitHub repository for Pi-C.A.R.D, a locally running AI voice assistant on a Raspberry Pi that can converse and describe camera photos.
593 points by nilsherzig | 118 comments
GitHub repository for LLocalSearch, a fully local meta search engine that uses chains of LLM agents to answer questions without OpenAI or Google API keys.
578 points by tosh | 194 comments
Ollama blog post from March 14, 2024 announcing preview support for running models on AMD graphics cards.
496 points by xyc | 210 comments
Landing page for RecurseChat, a macOS app for chatting with local LLMs offline, with full-text search and ChatGPT history import.
337 points by pentagrama | 137 comments
Ollama blog post from February 15, 2024 announcing a preview release of Ollama for Windows.
341 points by samlhuillier | 83 comments
GitHub repository for Reor, an open-source desktop note-taking app that runs local LLMs and vector search for semantic linking and RAG Q&A over markdown notes.
594 points by Casteil | 169 comments
Ollama blog post from February 2024 announcing OpenAI Chat Completions API compatibility for running local models like Llama 2.
544 points by adamhowell | 130 comments
Ollama blog post from January 23, 2024 announcing official Python and JavaScript client libraries installable via pip and npm.
584 points by JohnTheNerd | 156 comments
A personal blog post describing how the author built a fully local, LLM-powered voice assistant to control their smart home without cloud services.
406 points by victormustar | 120 comments
Landing page for LM Studio, a desktop app for Mac and Windows to discover, download, and run local LLMs such as LLaMa and Falcon in GGUF format.
618 points by jmorgan | 187 comments
Ollama blog post from August 1, 2023 explaining how to run an uncensored Llama 2 model locally using Ollama.
573 points by bfirsh | 153 comments
Replicate blog post explaining how to run Meta's Llama 2 model locally on M1/M2 Macs, Windows, Linux, and phones without an internet connection.
392 points by antouank | 97 comments
GitHub repository for privateGPT, an open-source tool for querying your own documents with GPT-style LLMs locally and privately without data leaks.
564 points by rain1 | 233 comments
A GitHub Gist with instructions for running Meta's Llama 13B model on a 6GB GPU using llama.cpp's CUDA/cuBLAS layer offloading.