Why Is Speculative Decoding Fast?(jbarrow.ai)
41 points by gmays | 5 comments
Blog post by Joe Barrow explaining that speculative decoding speeds up LLM inference by changing the kind of GPU work rather than reducing FLOPs.
jbarrow.ai on Hacker News
1 story on the front page, newest first. Scroll down to watch the site change over time.