vLLM large scale serving: DeepSeek 2.2k tok/s/h200 with wide-ep(blog.vllm.ai)

138 points by robertnishihara 9 months ago | 46 comments

Show HN: Letting LLMs Run a Debugger(github.com)

135 points by mohsen1 1 year ago | 28 comments

Apple's MLX adding CUDA support(github.com)

522 points by nsagent 1 year ago | 181 comments

A GitHub pull request (closed, WIP) in Apple's ml-explore/mlx repository adding an experimental CUDA backend to the MLX framework.

Show HN: liteLLM Proxy Server: 50+ LLM Models, Error Handling, Caching(github.com)

114 points by ij23 3 years ago | 27 comments

Vm0(github.com)

68 points by handfuloflight 9 months ago | 16 comments

Supporting org.apache.xml.security in graalVM(guust.ysebie.be)

23 points by whizzx 1 year ago | 3 comments