Polylith(polylith.gitbook.io)
85 points by 0x63_Problems | 13 comments
85 points by 0x63_Problems | 13 comments
91 points by ppymou | 8 comments
330 points by djhu9 | 18 comments
BentoML's LLM Inference in Production handbook, a glossary and guide covering inference concepts, performance metrics, optimization techniques, and operational best practices.
271 points by logicallee | 99 comments
MicroLLM Lab is a browser-based tool for running, chatting with, and benchmarking small quantized language models locally via WebGPU with no server.