LLM Inference Handbook(bentoml.com)

330 points by djhu9 1 year ago | 18 comments

BentoML's LLM Inference in Production handbook, a glossary and guide covering inference concepts, performance metrics, optimization techniques, and operational best practices.

Hetzner is working on LLM Inference(sliplane.io)

126 points by jonas_scholz 3 months ago | 53 comments

Philosophy of Biff(biffweb.com)

43 points by jacobobryant 2 years ago | 20 comments

Show HN: Pulpie – Models for Cleaning the Web(usefeyn.com)

79 points by snyy 3 months ago | 17 comments

Lisp Programming with Vim (2019)(susam.net)

82 points by susam 2 years ago | 15 comments

Inference cost at scale with napkin math(injuly.in)

74 points by gmays 4 months ago | 15 comments