vLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention (vllm.ai)

Screenshot of vllm.ai: vLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention

First screen, captured 3 years ago.

Looks similar

Web LLM (mlc.ai)

63 points by synergy20 3 years ago | 11 comments

Screenshot of blog.vllm.ai: vLLM large scale serving: DeepSeek 2.2k tok/s/h200 with wide-ep

vLLM large scale serving: DeepSeek 2.2k tok/s/h200 with wide-ep (blog.vllm.ai)

138 points by robertnishihara 9 months ago | 46 comments

Screenshot of baseten.co: The efficient frontier of LLM inference

The efficient frontier of LLM inference (baseten.co)

137 points by philipkiely 1 month ago | 39 comments

Screenshot of huggingface.co: LLM in a Flash: Efficient LLM Inference with Limited Memory

LLM in a Flash: Efficient LLM Inference with Limited Memory (huggingface.co)

223 points by ghshephard 2 years ago | 39 comments

Screenshot of bentoml.com: LLM Inference Handbook

LLM Inference Handbook (bentoml.com)

330 points by djhu9 1 year ago | 18 comments

BentoML's LLM Inference in Production handbook, a glossary and guide covering inference concepts, performance metrics, optimization techniques, and operational best practices.

llm-inference bentoml continuous-batching prefix-caching

Screenshot of souzatharsis.com: Taming LLMs – A Practical Guide to LLM Pitfalls with Open Source Software

Taming LLMs – A Practical Guide to LLM Pitfalls with Open Source Software (souzatharsis.com)

158 points by sebg 1 year ago | 26 comments

Looks similar

Web LLM(mlc.ai)

vLLM large scale serving: DeepSeek 2.2k tok/s/h200 with wide-ep(blog.vllm.ai)

The efficient frontier of LLM inference(baseten.co)

LLM in a Flash: Efficient LLM Inference with Limited Memory(huggingface.co)

LLM Inference Handbook(bentoml.com)

Taming LLMs – A Practical Guide to LLM Pitfalls with Open Source Software(souzatharsis.com)