vLLM large scale serving: DeepSeek 2.2k tok/s/h200 with wide-ep(blog.vllm.ai)138 points by robertnishihara 9 months ago | 46 comments
Screenshot pending…VLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention(blog.vllm.ai)8 points by jxmorris12 1 year ago | 1 comments
