bentoml.com on Hacker News

LLM Inference Handbook(bentoml.com)

330 points by djhu9 1 year ago | 18 comments

BentoML's LLM Inference in Production handbook, a glossary and guide covering inference concepts, performance metrics, optimization techniques, and operational best practices.

Benchmarking LLM Inference Back Ends: VLLM, LMDeploy, MLC-LLM, TensorRT-LLM, TGI(bentoml.com)

7 points by chaoyu 2 years ago | 0 comments

Is LMDeploy the Ultimate Solution? Why It Outshines VLLM, TRT-LLM, TGI, and MLC(bentoml.com)

11 points by helloericsf 2 years ago | 7 comments

Navigating the World of Large Language Models(bentoml.com)

39 points by sherlockxu 2 years ago | 3 comments