LLM Inference Handbook (bentoml.com) | HN Screenshots

YHacker Newsscreenshots

Screenshot of bentoml.com: LLM Inference Handbook

First screen, captured 1 year ago.

Looks similar

Screenshot of souzatharsis.com: Taming LLMs – A Practical Guide to LLM Pitfalls with Open Source Software

Taming LLMs – A Practical Guide to LLM Pitfalls with Open Source Software (souzatharsis.com)

158 points by sebg 1 year ago | 26 comments

Screenshot of baseten.co: The efficient frontier of LLM inference

The efficient frontier of LLM inference (baseten.co)

137 points by philipkiely 1 month ago | 39 comments

Screenshot of hughjonesd.github.io: R interface to Apple's MLX library

R interface to Apple's MLX library (hughjonesd.github.io)

12 points by dash2 11 months ago | 0 comments

Screenshot of ml-explore.github.io: MLX: NumPy like framework for Apple Silicon by Apple

MLX: NumPy like framework for Apple Silicon by Apple (ml-explore.github.io)

131 points by dagmx 2 years ago | 12 comments

Screenshot of neutralino.js.org: Neutralinojs – Build lightweight cross-platform desktop apps with JavaScript

Neutralinojs – Build lightweight cross-platform desktop apps with JavaScript (neutralino.js.org)

106 points by thekode 3 years ago | 69 comments

Screenshot of github.com: OpenLLM

OpenLLM (github.com)

562 points by fzliu 3 years ago | 142 comments

GitHub repository for BentoML's OpenLLM, an open platform to fine-tune, serve, deploy, and monitor large language models in production.

llm openllm bentoml model-inference

Related stories

Screenshot of nanonets.com: LLM Structured Outputs Handbook

LLM Structured Outputs Handbook (nanonets.com)

335 points by vitaelabitur 9 months ago | 58 comments

A Nanonets cookbook guide explaining techniques for getting reliable structured outputs like JSON and XML from large language models.

structured-outputs llm json constrained-decoding

Screenshot of github.com: BitNet: Inference framework for 1-bit LLMs

BitNet: Inference framework for 1-bit LLMs (github.com)

357 points by redm 7 months ago | 163 comments

GitHub repository for Microsoft's BitNet, the official inference framework for 1-bit large language models.

bitnet 1-bit-llm quantization llm-inference

Screenshot of news.mit.edu: LLMs use a surprisingly simple mechanism to retrieve some stored knowledge

LLMs use a surprisingly simple mechanism to retrieve some stored knowledge (news.mit.edu)

336 points by CharlesW 2 years ago | 121 comments

MIT News article on research showing LLMs often retrieve stored facts via simple linear functions, and a technique to probe what a model knows about subjects.

large-language-models interpretability knowledge-retrieval mit

Screenshot of yitay.net: Training LLMs from ground zero as a startup

Training LLMs from ground zero as a startup (yitay.net)

360 points by swyx 2 years ago | 92 comments

Yi Tay's blog post on the experience of training large language models from scratch at a startup (Reka), covering compute, infrastructure, and lessons learned.

llm-training reka startups gpu-compute

Screenshot of github.com: Llama2.c: Inference llama 2 in one file of pure C

Llama2.c: Inference llama 2 in one file of pure C (github.com)

659 points by anjneymidha 3 years ago | 158 comments

GitHub repository by Andrej Karpathy for llama2.c, a minimal implementation for running Llama 2 model inference in a single file of pure C.

llama-2 c llm-inference karpathy

Screenshot of magazine.sebastianraschka.com: LLM architecture comparison

LLM architecture comparison (magazine.sebastianraschka.com)

357 points by mdp2021 1 year ago | 23 comments

Sebastian Raschka's Ahead of AI article comparing the architectures of modern LLMs, from DeepSeek-V3 to Kimi K2.

llm-architecture deepseek-v3 kimi-k2 transformers