LLM Inference Handbook(bentoml.com)
330 points by djhu9 | 18 comments
BentoML's LLM Inference in Production handbook, a glossary and guide covering inference concepts, performance metrics, optimization techniques, and operational best practices.
#bentoml on Hacker News
2 stories tagged bentoml, newest first.
OpenLLM(github.com)
562 points by fzliu | 142 comments
GitHub repository for BentoML's OpenLLM, an open platform to fine-tune, serve, deploy, and monitor large language models in production.