35 stories tagged mixture-of-experts, newest first.
103 points by armcat | 19 comments
BottleCap AI blog post announcing ThinkingCap-Gemma-4-26B-A4B, a fine-tune cutting thinking tokens by 39% on average with near-unchanged benchmark accuracy.
307 points by funnym0nk3y | 104 comments
Mistral AI documentation page for Mistral Large 4, an open-weight multimodal MoE model with 52B active and 1.05T total parameters, listing pricing, context and API features.
499 points by theanonymousone | 264 comments
OpenRouter model page for DeepSeek V4.1 Flash 1017, a sparse mixture-of-experts LLM snapshot with pricing, 1M context, and API access details.
884 points by bigwheels | 446 comments
Meta research blog post announcing Muse Spark Open, an open-weight large language model Meta calls the most capable US open-weight model to date.
271 points by gmays | 72 comments
Liquid AI blog post announcing LFM2.5-24B-A2B, a sparse language model with about 2B active parameters designed for fast on-device agents.
207 points by khalidkhair | 77 comments
GitHub repository for Janas, a pure C inference engine that runs LLMs larger than RAM by streaming MoE experts from SSD, on CPU or GPU.
133 points by AnneWodell | 33 comments
OpenRouter model page for StepFun's Step 5 Preview, a 600B-parameter sparse MoE model with 1M context, multimodal input, and $1/$2.70 per million token pricing.
1961 points by Philpax | 1170 comments
Mistral documentation page for Mistral Large 4, an open-weight multimodal Mixture-of-Experts model with 49B active and 1.05T total parameters.
64 points by apt-get | 4 comments
Mistral AI post announcing Mistral Large 4, a 1T-parameter multimodal open-weights model with 49B active parameters, available via API with weights due end of October.
![Screenshot of aleph-alpha.com: Kolibri – Tech Report [pdf]](https://hn.urlbox.com/shots/renders/2026/10/3/afb8e7e8-ce88-431e-bc45-8ebfad65531a.jpg)
104 points by yu3zhou4 | 4 comments
Aleph Alpha technical report introducing Kolibri, an open-weights English–German Mixture-of-Experts language model with 78.1B total and 3.46B active parameters.
39 points by soltanov | 9 comments
X post announcing REDCELL-26B-A4B, a locally runnable MoE model fine-tuned on OSINT and cyber threat intelligence instructions, published as GGUF on Hugging Face.
1085 points by Philpax | 541 comments
Z.ai blog post announcing GLM-5.3-Flash, a natively multimodal 320B-parameter (18B active) model with hybrid sparse/linear attention, claiming strong coding and agentic results.
1004 points by explosion-s | 431 comments
OpenRouter model page for DeepSeek V4 Pro 0813, a mixture-of-experts LLM GA release, listing pricing ($0.435/$0.87 per 1M tokens), 1M context, and providers.
692 points by Philpax | 161 comments
Hugging Face model card for Qwen3.8-2.4T-A95B, a 2.4-trillion-parameter mixture-of-experts text generation model from Qwen in Transformers format.
318 points by marcobambini | 152 comments
GitHub repository for waste, a dependency-free embeddable C inference engine that runs the 2.78-trillion-parameter Kimi K3 model by streaming weights from NVMe.
845 points by gitpusher42 | 300 comments
GitHub repo for TurboFieldfare, a Swift/Metal inference engine running 4-bit Gemma 4 26B-A4B on M-series Macs in about 2 GB RAM by streaming experts from SSD.
882 points by vforno | 223 comments
GitHub repo for Colibrì, a dependency-free C engine that runs the 744B GLM-5.2 MoE model on a ~25GB-RAM machine by streaming experts from disk.
1124 points by stared | 699 comments
A Quesma blog post by Piotr Migdał arguing that the dense Qwen 3.6 27B model is the best choice for running LLMs locally for development, with impressions and setup guidance.
575 points by indigodaddy | 348 comments
Simon Willison's blog post reviewing DeepSeek's V4 preview models, Pro and Flash, 1M-context MoE open-weights LLMs under MIT license, and their pricing.
1209 points by cmitsakis | 496 comments
Qwen team blog post announcing the open-source release of Qwen3.6-35B-A3B, a mixture-of-experts model aimed at agentic coding.
689 points by anemll | 314 comments
An X post from @anemll showing a 400B-parameter MoE language model running locally on an iPhone at about 0.6 tokens per second via a Flash-MoE app.
379 points by mft_ | 119 comments
GitHub repository for Flash-MoE, a project for running a 397B-parameter mixture-of-experts model on a laptop using Metal inference code.
367 points by meetpateltech | 195 comments
Z.ai blog post announcing GLM-5, a 744B-parameter (40B active) language model aimed at agentic engineering, using DeepSeek Sparse Attention and asynchronous RL training.
323 points by scrlk | 108 comments
Hugging Face model card for GLM-4.7-Flash, a 31B-parameter MIT-licensed text generation model from Z.ai (zai-org).
340 points by meander_water | 136 comments
Blog post from Alibaba's Tongyi Lab introducing Tongyi DeepResearch, an open-source 30B MoE agent model, with benchmark results against OpenAI DeepResearch and others.
519 points by tosh | 201 comments
Qwen team blog post announcing Qwen3-Next, a model architecture aimed at improving LLM training and inference efficiency.
![Screenshot of arxiv.org: GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models [pdf]](https://hn.urlbox.com/shots/renders/2025/8/12/68eb07e3-6f81-44a7-9646-4891d3adb444.jpg)
382 points by SerCe | 71 comments
arXiv technical report from Zhipu AI and Tsinghua introducing GLM-4.5, a 355B-parameter open-source MoE language model for agentic, reasoning, and coding tasks.
345 points by ConteMascetti71 | 81 comments
GitHub repository for Kimi K2, Moonshot AI's mixture-of-experts large language model series, with docs, figures, license and README.
473 points by helloericsf | 66 comments
GitHub repository for DeepEP, DeepSeek's open-source expert-parallel communication library for Mixture-of-Experts model training and inference.
802 points by jasondavies | 320 comments
Databricks blog post from the Mosaic Research team announcing DBRX, a new open large language model claimed to be state-of-the-art.
589 points by georgehill | 272 comments
Mistral AI announcement post introducing Mixtral, a high-quality sparse mixture-of-experts language model.
![Screenshot of twitter.com: Mistral "Mixtral" 8x7B 32k model [magnet]](https://hn.urlbox.com/shots/renders/2023/12/8/7692ee30-ddfc-47ef-b630-0e0d65669ae0.jpg)
463 points by xyzzyrz | 199 comments
A Mistral AI post on X sharing a BitTorrent magnet link to download the weights of its Mixtral 8x7B 32k-context model.
360 points by 152334H | 162 comments
Blog post arguing that GPT-4's non-deterministic outputs at temperature zero are caused by its sparse mixture-of-experts architecture and batched routing.
569 points by bx376 | 488 comments
A Thread Reader unrolled 37-tweet Twitter thread by Yam Peleg purporting to reveal leaked architecture and training details of OpenAI's GPT-4.
322 points by georgehill | 167 comments
Tweet thread by swyx announcing a Latent Space podcast interview with George Hotz about tiny corp, including claims that GPT-4 is an 8x220B mixture model.