Faster prompt lookup drafting in llama.cpp(jadidbourbaki.github.io)
72 points by pptadversary | 11 comments
Blog post by Hayder Tirmazi describing optimizations that make prompt lookup drafting in llama.cpp up to 42x faster (140x with Daniel Lemire's follow-up PR) with less memory.