Cerebras Deploys Trillion-Parameter AI Models(cerebras.ai)
349 points by gmays | 127 comments
Cerebras blog post explaining how trillion-parameter models like Kimi K3 are split across multiple WSE-3T wafers for pipelined, high-throughput inference.
#kv-cache on Hacker News
4 stories tagged kv-cache, newest first.
Gemma 3 270M re-implemented in pure PyTorch for local tinkering(github.com)
399 points by ModelForge | 55 comments
GitHub directory in Sebastian Raschka's LLMs-from-scratch repo with notebooks re-implementing Google's Gemma 3 270M model in pure PyTorch, including a KV cache variant.
New LLM optimization technique slashes memory costs(venturebeat.com)
424 points by hochmartinez | 176 comments
VentureBeat article by Ben Dickson on a new LLM optimization technique that reduces memory costs by up to 75%.
Llama.cpp: Full CUDA GPU Acceleration(github.com)
640 points by gzer0 | 265 comments
A draft GitHub pull request to llama.cpp adding full CUDA GPU acceleration for all ggml tensors, including storing the KV cache in VRAM.