LLM Inference Handbook(bentoml.com)
330 points by djhu9 | 18 comments
BentoML's LLM Inference in Production handbook, a glossary and guide covering inference concepts, performance metrics, optimization techniques, and operational best practices.
#prefix-caching on Hacker News
1 story tagged prefix-caching, newest first.