Llama.cpp on a jailbroken PS5: run any GGUF model natively on the console(github.com)
356 points by Anon84 | 83 comments
GitHub repository for PS5LM, a port of llama.cpp that runs GGUF language models natively on a jailbroken PlayStation 5.
#gguf on Hacker News
9 stories tagged gguf, newest first.
Janas: Run LLMs larger than RAM by streaming experts from SSD(github.com)
207 points by khalidkhair | 77 comments
GitHub repository for Janas, a pure C inference engine that runs LLMs larger than RAM by streaming MoE experts from SSD, on CPU or GPU.
Show HN: Janus – Go binary that runs GGUF models via Vulkan on AMD/Intel/Nvidia(github.com)
92 points by Maverick617 | 17 comments
GitHub repository for Janus, an MIT-licensed AI model API router written in Go with a Vulkan-based runner for GGUF models on AMD, Intel, and Nvidia GPUs.
New Cyber-OSINT model released(twitter.com)
39 points by soltanov | 9 comments
X post announcing REDCELL-26B-A4B, a locally runnable MoE model fine-tuned on OSINT and cyber threat intelligence instructions, published as GGUF on Hugging Face.
GLM-5.2 – How to Run Locally(unsloth.ai)
559 points by TechTechTech | 266 comments
Unsloth documentation guide explaining how to run Z.ai's GLM-5.2 open model locally using Unsloth Dynamic quantized GGUFs.
Unsloth Studio(unsloth.ai)
364 points by brainless | 67 comments
Announcement page for Unsloth Studio (Beta), an open-source no-code web UI for training, running, and exporting open AI models locally.
How to run Qwen 3.5 locally(unsloth.ai)
424 points by Curiositry | 138 comments
Unsloth documentation guide explaining how to run Alibaba's Qwen3.5 model family locally using GGUF quantized versions, with updates on improved quantization.
Show HN: Sweep, Open-weights 1.5B model for next-edit autocomplete(huggingface.co)
499 points by williamzeng0 | 110 comments
Hugging Face model card for Sweep Next-Edit 1.5B, an open-weights GGUF-quantized model that predicts a developer's next code edit for autocomplete.
LM Studio – Discover, download, and run local LLMs(lmstudio.ai)
406 points by victormustar | 120 comments
Landing page for LM Studio, a desktop app for Mac and Windows to discover, download, and run local LLMs such as LLaMa and Falcon in GGUF format.