How Is LLaMa.cpp Possible?(finbarr.ca)
648 points by birriel | 210 comments
Blog post by Finbarr Timbers explaining the math of why llama.cpp can run LLaMA models locally on consumer hardware, focusing on memory bandwidth and compute limits.
#memory-bandwidth on Hacker News
1 story tagged memory-bandwidth, newest first.